mu-law and gaussian output tests

Since the original code is trained on 9-bit audio (i.e 512 softmax output), there're a lot of background static that is present and unfortunately learned by the model.
Two paths will need to be investigated:

mu-law encoded audio (256 softmax output)

Gaussian output, similar to what's been discussed in the Clarinet paper. Can also test other distributions with bounded support (i.e Beta).

HI, in your experiments, which loss is better? and in beta loss, what is your value that loss decrease to?

Hi, currently I would recommend using the raw bits (9 or 10 bits). These are the easiest and fastest to train. I think audio can sound pretty good with 10 bit raw output, with maybe some post-processing work on the speech. Interestingly I had more trouble with the mulaw encoded output (I tried mulaw with 256 and 512 output dim) than raw bit output.
In terms of loss on Beta/Gaussian, the best average loss I've gotten is around -6.6. However this still exhibit some weird sharp noise in the speech, as you can hear from my beta distribution samples. I'm hoping this is just an issue of the training hyper-parameters, which takes a lot of testing to get optimal.
However it could also be that the model is too simple for getting it work with the more difficult outputs like Beta/Gaussian. The only models I've seen gaussian working on in open-source are the wavenet variant models, which have alot more parameters and fancy tricks than this current model.

@G-Wang The generated audio of the mulaw loss is bad?

@hdmjdp yes, not as good as raw bits.
Currently the audio quality and training speed is from best to worse:
raw bits (9 or 10 bits) > mulaw > gaussian/beta > mixture of logistic (this doesn't really work).

@G-Wang yes, in my experiments, the same as yours. but mol also can generate  wav.

@G-Wang Do you test the fc and rnn layer without upsample and resnet? can this module realtime in cpu?

@hdmjdp I haven't tried architecture change to see if the network works without upsampling. But that would be an option to investigate.
As for real time synthesis (on GPU only),  I have implemented partly batched synthesis here: https://github.com/G-Wang/WaveRNN-Pytorch/blob/aa46ff3c4c199c152d50f047346f38f059b47b0c/model.py#L226
you can take the mel spectrogram from for single sentence (say dimension 300 x 80), split it and batch it up (batch size depends on your GPU, say batch size of 10, so dimension 30 x 10 x 80), pass this batched mel (numpy array) through the batch_generate function to generate a batch of wav, and then join the batch of wavs together into a single wav. This will provide a speed up, in my test i used a batch size of 4 and 6 and i can get close to real time synthesis. (13 seconds for 9 second audio). With a higher batch size this may be possible.
I'm currently wrapping up exams so I won't have time to implement this fully until December, but feel free to give this a crack. I think the batch synthesis audio will sound fine because our model is only training on a very small sample length (hop_size x seq = 256 x 5).
I'm also looking into having mel overlaps when we batch to avoid any audio issues for wav generated at the edges of mels.

@G-Wang thanks very much, I will try it.

@G-Wang  I use "batch_generate", but got noise, what is wrong?

@hdmjdp did you ensure the melspectrogram is normalized between 0 and 1? you can see the audio.py to see how melspectrogram is normalized. if you can, try using the same melspectrogram that it was trained on.
I will take a look on my end to see where the bug is.

@G-Wang my code has bug. I have fix it. I want to implement the inference code with c. the rnn and fc computation is 17Gflops, i am not sure whether it can run in realtime.

You probabaly have to prune the RNN networks like they do in WaveRNN paper to get a small model.

@G-Wang I've tried gaussian output at my branch https://github.com/erogol/WaveRNN and here is the result on LJSpeech
https://soundcloud.com/user-565970875/gaussian-wavernn
the intonation is different than it's supposed to be and there is static noise for the parts of silences. Do you have any idea to go over this?
Now I plan to go with a mixture of logistic noise assuming the distribution is more complex than to approximate with a single Gaussian.
My other guess is that the variance bound is too large since it causes sample values all over the place with a single Gaussian
There is the gaussian branch https://github.com/erogol/WaveRNN/tree/gaussian

@erogol The low intonation is rather strange and it's the first I've seen. perhaps you haven't trained it enough? I recall for my gaussians I trained for many many many steps, like 500k to 1 mil steps.
My experience with getting a single Gaussian to work is that the variance tends to grow large and quite a few many samples are way outside the range of -1 and 1 and had to be clipped. I tried fixing the variance as a constant, and learning only the Gaussian mean, however the result was still not great.
I ended up going with Beta distribution, since it has bounded support between [0,1] and I can just scale it to between -1 and 1. You can hear the samples (here)[https://soundcloud.com/gary-wang-23/sets/wavernn-samples]. You can hear that it still contain the static artifacts you mentioned, I still haven't figured out why.
As for mu-law, I tried many variations (including the one you mentioned where you apply mu-law during preprocessing of audio). However for some reason, the mu-law performance is not great. I still need to get around to investigate this, I think it has something to do with converting the mu-law discrete output to real values before feeding it to the RNN that has some issues.
At the end of the day, I've found that 10-bit audio in the original implementation style gives me the best samples. You can hear some for yourself (here)[https://soundcloud.com/gary-wang-23/sets/obama_bernie_fun]. And the model trains the fastest this way as well.

Hey, how are you doing?
I've recently tried Gaussian distribution over multi-band wavernn.  I find that my method doesn't work at all, I can't even get any audio except noise when softmax distribution can get good results. And I also noticed the problem of variance. Then, when i tried to clip the variance, i found that it is actually equivalent to fitting the distribution of samples with mse loss because of the very small variance.  Do you think that's my problem with dealing with Gaussian distribution, or is Gaussian distribution really not suitable for modeling data that is actually discrete? (For example, 10bit [-32767, 32768],  totally 65535 discrete numbers). I haven't tried beta distribution yet. Do you have any suggestions or concluding comments? If I can get your reply, I will be very happy.
I wish you a happy life.

@WelkinYang sorry for late reply. You can see under my code's distributions.py where I've tried a few distributions, Gaussian, beta and mixture of logistics. The nice thing about beta over guassian is that beta distribution has a fix range support (range over which samples can be drawn from) from 0 to 1, which you can then scale to whatever range you'd like. With Gaussian, you will have to clip the samples to be within range since it technically has infinite support. Note I don't clip the variance, but simply clip the samples to be within range. In terms of actually ease of training, I've found learning good continuous distributions to be more difficult than discrete. I think mixture of logistics currently is the best for WaveRNN


Jobber Binaries Release

I am currently pursuing minimal Docker images. Is it possible that you can release binaries? This way I don't have to install the toolset for compiling the sources.
I am using amd64 platform.

In fact, I've been thinking about making binary distros.  What package manager do you use?

I am using:

apk on Alpine
yum on CentOS

I am quite fine with the binaries for jobber and jobberd already. I am pretty confident with the installation routine by now.

It is mandatory that I am able to install a fix version! Can you tell me how to build binaries from sources that would solve my issue!
Please take a look at my two installation routines inside my Dockerfile: https://github.com/blacklabelops/jobber-cron/blob/alpine/Dockerfile
Because go get does not support any versioning I wall always get the latest master sources. Now I download the sources of a specific release and want to compile it.
I get the following:
make: Entering directory '/tmp/jobber'
bash update-version.sh
fatal: Not a git repository (or any of the parent directories): .git
go install github.com/dshearer/jobber
can't load package: package github.com/dshearer/jobber: cannot find package "github.com/dshearer/jobber" in any of:
/usr/lib/go/src/github.com/dshearer/jobber (from $GOROOT)
/tmp/jobber/lib/src/github.com/dshearer/jobber (from $GOPATH)
make: *** [build] Error 1
cp: can't stat '/tmp/jobber/lib/bin/*': No such file or directory
Makefile:21: recipe for target 'build' failed
What would you propose? Would git clone on a release tag solve the issue?

Only for you! A Docker build building a binary Jobber release: https://github.com/blacklabelops/jobber-cron/tree/alpine/jobber-build
Maybe it's a valid idea to build a binary distribution and then adapt an installer without the whole build procedure?

I'll work on scripts that make a binary package for Alpine.

Looks like there is some fine progress! Is there a release date and where can I download them?

I'm slowwwly making progress...  I'm now building RPMs... which is not what you want!  But I can use the same structure for the Alpine builds.  I'll need another week or so.

In essence I want to be able to build a docker image from a release not only from master.
Either from:

A downloadable package.
Downloadable Source
Binaries

Maybe one of those is already working for Alpine?

I have FINALLY finished this!  An Alpine package is available on the releases page, under v<IP_ADDRESS>.
If you want, you can also build the package yourself.  On an Alpine host, checkout the sources, then cd into packaging/alpine_3.3/ and do "make pkg-local".
Let me know if this meets your needs.

Thanks for the binaries. I included an installation routine but have problems with some dependencies.
  It seems that
Alpine's developers dislike this, with reason.
When Go supports PIEs, we'll fix this.
*****************
var/cache/misc/jobber-1.1-r0.post-install: line 24: rc-update: not found
var/cache/misc/jobber-1.1-r0.post-install: line 25: service: not found
ERROR: jobber-1.1-r0.post-install: script exited with error 127

What does your docker file look like?

On Jun 19, 2016, at 12:56 AM, Steffen Bleul<EMAIL_ADDRESS>wrote:
Thanks for the binaries. I included an installation routine but have problems with some dependencies.
I tried to install them inside a Docker image, I get the following error message:
Install Command:
wget --directory-prefix=/tmp https://github.com/dshearer/jobber/releases/download/v1.1/jobber-${JOBBER_VERSION}-r0.x86_64.apk &&
apk add --allow-untrusted /tmp/jobber-${JOBBER_VERSION}-r0.x86_64.apk
Message:
(1/1) Installing jobber (1.1-r0)
Executing jobber-1.1-r0.post-install
**** WARNING ****
We are installing a non-PIE SUID-root program
(namely, /usr/bin/jobber).  It seems that
Alpine's developers dislike this, with reason.
When Go supports PIEs, we'll fix this.

var/cache/misc/jobber-1.1-r0.post-install: line 24: rc-update: not found
var/cache/misc/jobber-1.1-r0.post-install: line 25: service: not found
ERROR: jobber-1.1-r0.post-install: script exited with error 127
―
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

Dockerfile is here: https://github.com/blacklabelops/jobber-cron/blob/master/Dockerfile

I haven't had time to mess with your dockerfile yet. I see you're still making the jobber user. The package takes care of all that now.

On Jun 20, 2016, at 2:54 PM, Steffen Bleul<EMAIL_ADDRESS>wrote:
Dockerfile is here: https://github.com/blacklabelops/jobber-cron/blob/master/Dockerfile
―
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

This is the dockerfile I use to test the package: https://github.com/dshearer/jobber/blob/master/platform_tests/alpine_3.3/Dockerfile

On Jun 20, 2016, at 2:54 PM, Steffen Bleul<EMAIL_ADDRESS>wrote:
Dockerfile is here: https://github.com/blacklabelops/jobber-cron/blob/master/Dockerfile
―
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

What's inside your PKG_DIR=/pkgs?

The apk file.

On Jun 23, 2016, at 10:56 PM, Steffen Bleul<EMAIL_ADDRESS>wrote:
What's inside your PKG_DIR=/pkgs?
―
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

  It seems that
Alpine's developers dislike this, with reason.
When Go supports PIEs, we'll fix this.
*****************
var/cache/misc/jobber-1.1-r0.post-install: line 24: rc-update: not found
var/cache/misc/jobber-1.1-r0.post-install: line 25: service: not found
ERROR: jobber-1.1-r0.post-install: script exited with error 127

Those errors are because the package assumes that rc-init is installed.  jobber should still have been installed successfully though.

Does it seem to be working?

Currently, I am fine with checking out the tags and build the version myself. This way I do not have to ignore installation errors and be aware of any build problems.


Is there any way to get the schema that results from applying $merge or $patch definitions?

For example, if I have the following two schemas:
{
  "$id": "baseSchema",
  "type": "object",
  "properties": {
    "foo": {
      "type": "number"
    }
  }
}

{
  "$id": "schemaExtended",
  "$merge": {
    "source": { "$ref": "schemaExtended.json#" },
    "with": {
      "properties": {
        "foo": {
          "minimum": 0
        },
        "bar": {
          "type": "string"
        }
      }
    }
  }
}

Is there anyway to output the result of the merge operation?  Something like ajv.getMergedSchema(schemaExtended);, which would produce:
{
  "$id": "schemaExtended",
  "type": "object",
  "properties": {
    "foo": {
      "type": "number",
      "minimum": 0
    }
    "bar": {
      "type": "string"
    }
  }
}

The reason I ask is that most other json-schema implementations do not support merge-patch, so it would be nice if there was some method by which JSON schemas that use these keywords could be converted to regular JSON schemas.

This package implements $merge and $patch as custom keywords, it only uses resolved schemas in place, without generating full resolved schema.
It's relatively simple to generate the schema you want, using the same packages that are used here and json-schema-traverse.
It's worth publishing as a separate package I think - it could be used in ajv-cli package to add an additional command.


operator: drop build-image api

We now have the SeedImage resource to start and track image building tasks: drop the old build-image api.

Unit test failure is independent from this PR: wait for the fix in PR #390 .


[Enhancement] Support transform replace function in trino parser

Why I'm doing:
trino support function:
replace(string, search) → varchar
Removes all instances of search from string.

Ref: https://trino.io/docs/current/functions/string.html?highlight=replace#replace
What I'm doing:
Fixes #issue
What type of PR is this:

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[ ] Yes, this PR will result in a change in behavior.
[x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.2
[x] 3.1
[ ] 3.0
[ ] 2.5

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Minor fix for UIScrollView+CollectionKit.swift.

LGTM

@jindulys Lets delete the branch after the PR is merged to keep our branch count low.


Enable golangci-lint in GitHub Action and local

What type of PR is this? Enable golangci-lint for local dev and GitHub Action
Changes:

Install golangci-lint in the toolchain.sh
Add golangci-lint Makefile target, presubmit depends on it
Run golangci-lint job in Github Action, run this workflow for every time people send Pull Requests

Test

Test  toolchain.sh install golangci-lint in local ( CI!=true) and github action (CI==true) environment, in local it could install golangci-lint if it is not yet installed in your machine. And in github action environment it will still to install (Since we have separate github action job to run the golangci-lint by  uses: golangci/golangci-lint-action@v3 github plugin)
Test make presubmit running golangci-lint  in local ( CI!=true) and github action (CI==true) environment both work as expect. local env can run it and CI environment skip it
For GitHub Action happy path test, you could check this PR's "checks":

https://github.com/aws/aws-application-networking-k8s/actions/runs/7053275325/job/19200030069?pr=556

For GitHub Action unhappy path test, you could see  this PR https://github.com/aws/aws-application-networking-k8s/pull/555/files, and for the code that violate golangci-lint check, you will see the error message in the PR "Files changed" tab:

Will this PR introduce any new dependencies?: developers need to run make toolchain again to install golangci-lint in their local
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Can you add a task please to split toolchain for development and CI into different scripts. There is very little overlap in practice. In CI we might use specialized containers and more complex testing environment. And for development we might need more people oriented tools.

Ok, make sense,  will do


how to change to current Location position ?

how to change to current Location position ?

What do you mean? The current location position should be determined by the gps on the device. What are you trying to achieve?

This was reported before the first beta of v4. Assuming this is not relevant anymore.


KML file isn't renderer correctly.

Hi,
I am rendering a file using GMUKMLParser and GMUGeometryRenderer and the place marks icons and route style aren't render correctly. I have attached the file. In Android works fine.
Thanks
VALENCIA.kml.zip

Thanks for reporting. I haven't had a chance yet to look into this, but I plan on doing so. In the meantime, contributions are certainly welcome.

To match what's displayed on Android, I've filed #285 and #286 @ginesdeka Take a look and see if those are the issues that you meant. Note that the marker on Android is also the default one (red pin).

Please see #285 and #286 instead for this issue.


Access to cache is not specified in the VM message

VMs will soon have access to a shared cache. Access to this cache should be conditioned on properties of the VM message, so VM authors can specify if they allow it and so hosts can discriminate VMs that need cache from those that do not.
A limitation on the size of the cache would be useful as well.

Should VMs specify for how long the cache is kept ?


Gaps seem to be too big?

I got this VCF file:
##contig=<ID=ctg000001c,length=2880941>
ctg000001c      199338  .       A       <*>     0       DUP     END=2880941
ctg000001c      200045  .       A       <*>     0       Gaps    END=2880941
ctg000001c      200045  .       A       <*>     0       Gaps    END=2880941
ctg000001c      200226  .       A       <*>     0       DUP     END=2880941ctg000001c      282576  .       TT      T       19      PASS    RC=0;TYPE=del
<more results>

It seems to be saying there is a gap (unmapped reads) from 200 kbp to 2880 kbp (nearly the whole sequence). I know this is wrong, as the reads are from the same genome, and there are variants being called in the "Gaps" region. And the DUP regions also seem to go to the end of the sequence?
Is this a bug or am I misunderstanding?
PS. MapCaller v<IP_ADDRESS>

@tseemann Thank you for reporting this bug. I'll fix it asap.

I tried to fix this bug, but I was not sure if it works on your data. Please let me know if it happens again. It'd much better if I could test with your dataset. Thank you!
The latest version is v<IP_ADDRESS>.

It does not work on my data.
I will try and send you my data.
I tried -max_cnv 100000000 and it still reported them.

I've fixed the bug. It happened because I did not capture the correct size of cnv and unmapped region. Please update MapCaller to v<IP_ADDRESS>. Thank you!

I modified the rules for the identification of duplicated and unmapped regions. The previous rules were not complete. Please update MapCaller to v<IP_ADDRESS>. Thank you!

Seems to be fixed


OMF Format: Better 16/32 bit detection

When I try to import a 16 bit OMF library it suggest using 32 bit language definition for import as default.
I would like it to have a 16 bit language definition as default.
From the OMF standard the 16/32 bit definition is in SEGHEAD records.
So instead of stopping the fast scan after any comments it should stop after the first SEGHEAD.
My tests so far shows the SEGHEAD are all either 16 or 32 bit, no mixed cases.
My studies of the Query interface leaves me a bit puzzeled as how to do the selection based on 16/32 bit when doing a query.
I can't really see how that should be done. So if I could have a few pointers on that, I would be grateful.
A follow up question.
Watcom doesn't use the translator comment type, but have a special comment record telling how it was compiled.
But that record may be deep inside the library. So only by doing a full "scan" could that be detected.
But since language definition isn't used during processing (except the 16/32 bit) part, as far as I can see, would it be possible
to change the language definition in the parse phase, but before the parsed information is saved?
I see that Microsoft has a similar record type, but also sometimes have a translator record.

The import language suggestions are done by entries in the various process module's "opinion" files.  The only processor that has an entry for OMF is x86.opinion:
    <constraint loader="Relocatable Object Module Format (OMF)">
      <constraint compilerSpecID="windows">
        <constraint primary="i386" processor="x86"          endian="little" size="32" />
      </constraint>
      <constraint compilerSpecID="default">
        <constraint primary="i386" processor="x86"          endian="little" size="16" />
      </constraint>
      <constraint compilerSpecID="borlandcpp">
        <constraint primary="i386" secondary="borlandcpp" processor="x86"        endian="little" size="32" />
        <constraint primary="i386" secondary="codegearcpp" processor="x86"       endian="little" size="32" />
      </constraint>
      <constraint compilerSpecID="borlanddelphi">
        <constraint primary="i386" secondary="borlanddelphi" processor="x86"     endian="little" size="32" />
      </constraint>
      ```
The primary and secondary key parameters to `public static List<QueryResult> query(String loaderName, String primaryKey, String secondaryKey)` directly map to the above XML attributes.  For many opinion entries, these keys are values from the header that represent various machine types.  In other cases (like OMF), they are just strings that we define based on some other criteria.

Some follow up questions.
Inside the x86.opinion file there are references to various compiler specs like "windows", "borlanddelphi" and so on but the cpsec files are called names like "x86win", "x86delphi" and so on.
Is there a mapping I am missing or should the name align with the cspec file names?
Also are there to be a 16/32/64 version if the compiler generate code for all 3 types?
E.g. x86delphi_16, x86delphi_32 ...
Should windows be microsoft, visualstudio or something like that as multiple compilers can generate code for windows.

Inside the x86.opinion file there are references to various compiler specs like "windows", "borlanddelphi" and so on but the cpsec files are called names like "x86win", "x86delphi" and so on.
Is there a mapping I am missing or should the name align with the cspec file names?

They are defined in x86.ldefs.

Also are there to be a 16/32/64 version if the compiler generate code for all 3 types?
E.g. x86delphi_16, x86delphi_32 ...

I am a bit weak in this area, but it seems there is a different cspec file for each bit-version, each one using the same compiler id (i.e., windows).

Should windows be microsoft, visualstudio or something like that as multiple compilers can generate code for windows.

Yes, this is something we have been talking about improving.


Added tar.gz support for folder upload.

When first uploading files (no directory @ remote), it's MUCH more efficient to tar.gz files @ local, upload the archive and unzip it @ the remote.
For comparison, some 170 seconds uploads cut down to 1-2 seconds during my tests. (more than x100)
It still falls back to using parallel upload for single files or if it fails in one of the steps.

How does this compare to rsync and what kind of latency is there on your connection?

Hi,
I haven't compared it to rsync, since rsync is required to be installed @ the remote, while I'm using the base Azure Linux image and it's not installed there (as opposed to tar which comes with the basic Linux image), so it's irrelevant for me.
I'm located in Israel, and running azurerm on West Europe - that's the main cause for the latency. I guess if I were running a local vagrant things would look differently.
As for the exact latency, this is an example ping result:
Reply from XXXXXX: bytes=32 time=75ms TTL=48
Roy.

Hi @coderanger,
Are you going to accept this patch?
Do you have anything that should be fixed, or comments regarding it?
Thanks

Any update on this?

@coderanger ?

@coderanger ?

@coderanger ??????


x5t#S256 is encoded twice in JWKS endpoint

Describe the issue:
The x5t#S256 is encoded twice in JWKS endpoint.
Ex:
"x5t#S256": "TXpZeE1tRmtPR1l3TVdJMFpXTm1ORGN4TkdZd1ltTTRaVEEzTVdJMk5EQXpaR1F6TkdNMFpHUmxOakprT0RGa1pEUmlPVEZrTVdGaE16VTJaR1ZsTmc",

The actual certificate thumbprint is 3612ad8f01b4ecf4714f0bc8e071b6403dd34c4dde62d81dd4b91d1aa356dee6, but it's encoded twice as follows.

Base64 encode [1]
Base64URL encode at [2]

As per the spec, it should be encoded with base64 URL [3].
Environment information

Product Version: IS 5.11.0

[1] https://github.com/wso2-extensions/identity-inbound-auth-oauth/blob/v6.8.0/components/org.wso2.carbon.identity.oauth/src/main/java/org/wso2/carbon/identity/oauth2/util/OAuth2Util.java#L2979
[2] https://github.com/wso2-extensions/identity-inbound-auth-oauth/blob/v6.8.0/components/org.wso2.carbon.identity.oauth.endpoint/src/main/java/org/wso2/carbon/identity/oauth/endpoint/jwks/JwksEndpoint.java#L134C21-L134C45
[3] https://www.rfc-editor.org/rfc/rfc7515.html#page-12:~:text=The "x5t%23S256" (X.509 certificate SHA-256 thumbprint) Header
   Parameter is a base64url-encoded SHA-256 thumbprint

duplicate of #14899

Reopening because we need to fix the public.

This is fixed in the master. https://github.com/wso2-extensions/identity-inbound-auth-oauth/pull/2128


Acquire organization projects

Is there a way to get an organization's projects (and not a repositorie's projects)?

Found it, sorry :) http://github-tools.github.io/github/docs/3.1.0/Organization.html#listProjects


Hunter download error while building

Hi! I am facing an issue while building depthai-core. This is what I'm getting:
root@NanoPi-NEO-Plus2:~/depthai-core# cmake -H. -Bbuild -D BUILD_SHARED_LIBS=ON
-- [hunter] Initializing Hunter workspace (062a19ab13ce8dffa9a882b6ce3e43bdabdf75d3)
-- [hunter]   https://github.com/cpp-pm/hunter/archive/v0.23.258.tar.gz
-- [hunter]   -> /root/.hunter/_Base/Download/Hunter/0.23.258/062a19a
CMake Error at Build/Hunter-prefix/src/Hunter-stamp/download-Hunter.cmake:159 (message):
  Each download failed!

    error: downloading 'https://github.com/cpp-pm/hunter/archive/v0.23.258.tar.gz' failed
         status_code: 1
         status_string: "Unsupported protocol"
         log:
         --- LOG BEGIN ---
         Protocol "https" not supported or disabled in libcurl

  Closing connection -1

         --- LOG END ---
         error: downloading 'https://github.com/cpp-pm/hunter/archive/v0.23.258.tar.gz' failed
         status_code: 1
         status_string: "Unsupported protocol"
         log:
         --- LOG BEGIN ---
         Protocol "https" not supported or disabled in libcurl

  Closing connection -1

         --- LOG END ---
         error: downloading 'https://github.com/cpp-pm/hunter/archive/v0.23.258.tar.gz' failed
         status_code: 1
         status_string: "Unsupported protocol"
         log:
         --- LOG BEGIN ---
         Protocol "https" not supported or disabled in libcurl

  Closing connection -1

         --- LOG END ---
         error: downloading 'https://github.com/cpp-pm/hunter/archive/v0.23.258.tar.gz' failed
         status_code: 1
         status_string: "Unsupported protocol"
         log:
         --- LOG BEGIN ---
         Protocol "https" not supported or disabled in libcurl

  Closing connection -1

         --- LOG END ---
         error: downloading 'https://github.com/cpp-pm/hunter/archive/v0.23.258.tar.gz' failed
         status_code: 1
         status_string: "Unsupported protocol"
         log:
         --- LOG BEGIN ---
         Protocol "https" not supported or disabled in libcurl

  Closing connection -1

         --- LOG END ---
         error: downloading 'https://github.com/cpp-pm/hunter/archive/v0.23.258.tar.gz' failed
         status_code: 1
         status_string: "Unsupported protocol"
         log:
         --- LOG BEGIN ---
         Protocol "https" not supported or disabled in libcurl

  Closing connection -1

         --- LOG END ---

make[2]: *** [Hunter-prefix/src/Hunter-stamp/Hunter-download] Error 1
make[1]: *** [CMakeFiles/Hunter.dir/all] Error 2
make: *** [all] Error 2

[hunter ** INTERNAL **] Build project failed
[hunter ** INTERNAL **] [Directory:/root/depthai-core]

------------------------------ ERROR ------------------------------
    https://docs.hunter.sh/en/latest/reference/errors/error.internal.html
-------------------------------------------------------------------

CMake Error at cmake/HunterGate.cmake:87 (message):
Call Stack (most recent call first):
  cmake/HunterGate.cmake:97 (hunter_gate_error_page)
  cmake/HunterGate.cmake:345 (hunter_gate_internal_error)
  cmake/HunterGate.cmake:509 (hunter_gate_download)
  CMakeLists.txt:17 (HunterGate)

-- Configuring incomplete, errors occurred!

Sorry for the trouble here.  Will likely need @themarpe 's help on this.

Reposting the response from Community Discord:
@ans-qureshi seems like libcurl is outdated. Can you try updating it? Note: most likely offical apt of Ubuntu16 doesn't carry it, so you'll have to get it somewhere else, but updating it might cause issues for some other programs that also really on it. While not highly likely, still proceed with caution
wget https://curl.se/download/curl-7.77.0.tar.gz
./configure
make
sudo make install

The above should install the latest version. If it still won't work, some poeple recommend removing system curl sudo apt-get remove libcurl

Thanks, @themarpe !


依赖升级

npm WARN deprecated<EMAIL_ADDRESS>Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated<EMAIL_ADDRESS>This project has been renamed to 'tasksfile'. Install using 'npm install tasksfile' instead.
npm WARN deprecated<EMAIL_ADDRESS>core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
npm WARN deprecated<EMAIL_ADDRESS>fsevents 1 will break on node v14+. Upgrade to fsevents 2 with massive improvements.
npm WARN deprecated<EMAIL_ADDRESS>request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated<EMAIL_ADDRESS>Package no longer supported. Contact<EMAIL_ADDRESS>for more info.
npm WARN deprecated<EMAIL_ADDRESS>Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated<EMAIL_ADDRESS>CircularJSON is in maintenance only, flatted is its successor.
npm WARN deprecated<EMAIL_ADDRESS>Please upgrade to kleur@3 or migrate to 'ansi-colors' if you prefer the old syntax. Visit https://github.com/lukeed/kleur/releases/tag/v3.0.0\ for migration path(s).

暂缓用v14吧。应该很多依赖库都还没升级


[Feature Req]: Can we include fs module polyfill to the plugin?

This plugin is super helpful, so far most of the core modules are included as I can see. But I had to include the polyfill for fs module separately. Can we include that in the global plugin itself?

Which fs polyfill is used?

@Richienb there is no polyfill for fs in the browser, hence #45.

@afonsojramos What's the user experience with using 'fs': false?
Does it necessitate adding a note about it in the readme?

The point of this repo is to facilitate polyfills, and since there is no polyfill for fs, we might as well set it to false to set it as an empty implementation. Even some upgrade guides of webpack v4 to v5 suggest it.


Build and publish a new release please

Hello.
Could you please build, package and release a new version with latest commits (e.g. v2.1) to release page?
I would build it myself but I get this error:
...
Enabling urandom_seed
Enabling urngd
Enabling wpad
rm -rf /home/ado_linux/git/openwrt/build_dir/target-mips_24kc_musl/root-ath79/boot /home/ado_linux/git/openwrt/build_dir/target-mips_24kc_musl/root-ath79/tmp/* /home/ado_linux/git/openwrt/build_dir/target-mips_24kc_musl/root-ath79/lib/apk/db/*.post-install* /home/ado_linux/git/openwrt/build_dir/target-mips_24kc_musl/root-ath79/usr/lib/opkg/info/*.postinst* /home/ado_linux/git/openwrt/build_dir/target-mips_24kc_musl/root-ath79/usr/lib/opkg/lists/* /home/ado_linux/git/openwrt/build_dir/target-mips_24kc_musl/root-ath79/var/lock/*.lock
find /home/ado_linux/git/openwrt/build_dir/target-mips_24kc_musl/root-ath79/ -mindepth 1 -execdir touch -hcd "@1724003867" "{}" +
find: The relative path 'Files' is included in the PATH environment variable, which is insecure in combination with the -execdir action of find.  Please remove that entry from $PATH
make[2]: *** [package/Makefile:102: package/install] Error 1
make[2]: Leaving directory '/home/ado_linux/git/openwrt'
make[1]: *** [package/Makefile:172: /home/ado_linux/git/openwrt/staging_dir/target-mips_24kc_musl/stamp/.package_install] Error 2
make[1]: Leaving directory '/home/ado_linux/git/openwrt'
make: *** [/home/ado_linux/git/openwrt/include/toplevel.mk:248: world] Error 2

Alright, I was in WSL2 and was able to solve it by modifying PATH env variable by removing Windows paths, but still it would be nice if I could just download it :) Thank you.

+1

+1

Would it still be possible to publish a new release? It would be great to have an up-to-date package without having to compile it by oneself.


Exception when starting ledfx

Hi folks,
i just installed ledfx on my win10 machine (incl. Anaconda).
When starting ledfx, I am receiving the following message:
(ledfx) PS C:\Users\wolfg> ledfx --open-ui Loading configuration file from C:\Users\wolfg\AppData\Roaming\.ledfx Started webinterface at http://<IP_ADDRESS>:8888 Exception in thread sACN sending/sender thread: Traceback (most recent call last):   File "c:\users\wolfg\.conda\envs\ledfx\lib\threading.py", line 932, in _bootstrap_inner     self.run()   File "c:\users\wolfg\.conda\envs\ledfx\lib\site-packages\sacn\sending\output_thread.py", line 58, in run c:\users\wolfg\.conda\envs\ledfx\lib\site-packages\ledfx\effects\gradient.py:94: FutureWarning: arrays to stack must be passed as a "sequence" type such as list or tuple. Support for non-sequence iterables such as generators is deprecated as of NumPy 1.16 and will raise an error in the future.   gradient = np.hstack(self._color_ease(len(ease_chunks[i]), *color_pairs[i]) for i in range(n_colors-1))     [self.send_out(output) for output in list(self._outputs.values())   File "c:\users\wolfg\.conda\envs\ledfx\lib\site-packages\sacn\sending\output_thread.py", line 58, in <listcomp>     [self.send_out(output) for output in list(self._outputs.values())   File "c:\users\wolfg\.conda\envs\ledfx\lib\site-packages\sacn\sending\output_thread.py", line 79, in send_out     self.send_packet(output._packet, udp_ip)   File "c:\users\wolfg\.conda\envs\ledfx\lib\site-packages\sacn\sending\output_thread.py", line 95, in send_packet     self._socket.sendto(MESSAGE, (destination, DEFAULT_PORT)) socket.gaierror: [Errno 11001] getaddrinfo failed
Any ideas what is going wrong?
Thanks,
Wolfgang

@WolfgangHusen Please uninstall, uninstall anaconda and give the packaged application a go - https://ledfx.app/download

Thanks -  I uninstalled Anaconda and had to uninstall Python. After installing the packaged installation - it works.
Many thanks!


Getting this issue on running npm start... what is this??

Actual Behaviour
I am getting the following after locally installing the project, and using npm run start
`><EMAIL_ADDRESS>start /home/pranav1698/Desktop/open-source/open-event-webapp

node src/app.js

node[20721]: ../src/node_contextify.cc:628:static void node::contextify::ContextifyScript::New(const v8::FunctionCallbackInfov8::Value&): Assertion args[1]->IsString()' failed.  1: 0x8d20d0 node::Abort() [node]  2: 0x8d21a5  [node]  3: 0x9075f7 node::contextify::ContextifyScript::New(v8::FunctionCallbackInfo<v8::Value> const&) [node]  4: 0xb89f5b  [node]  5: 0xb8bef2 v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) [node]  6: 0x1cd66115c01d  Aborted (core dumped) npm ERR! code ELIFECYCLE npm ERR! errno 134 npm ERR<EMAIL_ADDRESS>start: node src/app.js`
npm ERR! Exit status 134
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/pranav1698/.npm/_logs/2018-09-24T19_10_04_409Z-debug.log
`
even though the dependencies are installed without any error,

Expected Behaviour
After the installation of dependencies, it should open up the site in the browser as expected but is not happening at this moment...

Steps to reproduce it
Probably a problem in the code

Log for the issue
Provided above

Screenshots of the issue
console error provided above

Would you like to work on the issue?

This is an installation related problem, the app is working fine.
Please ask installation related queries on the gitter channel, closing this for now.


Window explorer overlapping cairo shell

Explorer/Edge chromium over laping cairo shell after window 2004 update

System information:

Windows version: Build 19041
Cairo version: 0.3.7023.0
32-bit or 64-bit: 64
In Cairo Settings > Advanced, is Cairo set as shell?: No

Additional context
Add any other context about the problem here.
Screenshots
If applicable, add screenshots to help explain your problem.

Does the issue persist after reboot?

Happens mostly with Edge Chromium.
I have to exit cairo and restart to get the issue solve

When in this state, did other windows overlap Cairo when dragging them over it?

No while clicking the window, it gets overlaped
Sent from Yahoo Mail on Android
On Wed, Jun 24, 2020 at 8:58 PM, Sam<EMAIL_ADDRESS>wrote:
When in this state, did other windows overlap Cairo when dragging them over it?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

This should be resolved in the latest release.


Conditionally call xterm.js

The current method of calling xterm.js would result in xterm.js being included in the bundle and being loaded for every page.  Look into calling xterm.js only for the pages that will be displaying a console.

s/call/load/g

Already done.


Possibility to add markers without lat, long

I would like to be able to send a few draggable icons into the map but I can't find a way to send them to the map without knowing where the map is positioned.
Basically I would like to send the new icon to "center" of the map without knowing lat, long.
Is this possible?
Something like this;
{
"name": "c1",
"draggable": true,
"icon": "satellite",
"label": "C1",
"position": "center"
}

Hi - no this is not possible. You can either set the map to a known position and zoom then add the markers or if the map is in move to marker mode you could add them and the map will move to where yo add them.


Document compatible Kubernetes versions

Document compatible Kubernetes versions
fixes #1281

This change is

Preview:
https://deploy-preview-1319--competent-brattain-de2d6d.netlify.com/docs/started/k8s/overview/
Thanks @mameshini - please would you add a comment when this PR is ready for review.
One thing that struck me: Do you think it's worth moving the version info and table into a separate page, in the reference section of the docs? A couple of reasons:

First, and most importantly, putting the info on a separate page makes it easier to search for (SEO) and to find by navigating through the doc menu. People won't necessarily look for this info in the page about installing on existing K8s clusters.
Second, the table is likely to get quite large.

@sarahmaddox This PR is ready for review.
I updated it based on feedback I received from Jeremy.
We are providing this information to users who need to deploy on existing clusters, so it's certainly helpful to have the compatibility matrix in this section.  We can also copy it to the Reference section as well.  I will keep research about broader compatibility matrix, for example what versions of Istio, dex, etc are compatible with what Kubeflow versions.  But at this time I don't think I can accurately document it.

/assign @sarahmaddox

Thanks @mameshini - just one comment then it'll be LGTM.

Thanks!
/lgtm
/approve


iox-#1614 Move uninitialized array

Pre-Review Checklist for the PR Author

[x] Code follows the coding style of CONTRIBUTING.md
[x] Tests follow the best practice for testing
[x] Changelog updated in the unreleased section including API breaking changes
[x] Branch follows the naming format (iox-123-this-is-a-branch)
[x] Commits messages are according to this guideline

[x] Commit messages have the issue ID (iox-#123 commit text)
[x] Commit author matches Eclipse Contributor Agreement (and ECA is signed)

[x] Update the PR title

Follow the same conventions as for commit messages
Link to the relevant issue

[x] Relevant issues are linked
[x] Add sensible notes for the reviewer
[ ] All checks have passed (except task-list-completed)
[x] All touched (C/C++) source code files from iceoryx_hoofs are added to ./clang-tidy-diff-scans.txt
[ ] Assign PR to reviewer

Notes for Reviewer

This PR moves the UninitializedArray to iceoryx_hoofs/containers/include/iox/detail/ and removes the namespace containers.
Checklist for the PR Reviewer

[ ] Commits are properly organized and messages are according to the guideline
[ ] Code according to our coding style and naming conventions
[ ] Unit tests have been written for new behavior
[ ] Public API changes are documented via doxygen
[ ] Copyright owner are updated in the changed files
[ ] All touched (C/C++) source code files from iceoryx_hoofs have been added to ./clang-tidy-diff-scans.txt
[ ] PR title describes the changes

Post-review Checklist for the PR Author

[ ] All open points are addressed and tracked via issues

References

Closes #1614

Oh, the clang-tidy warning will disappear once you rebase to master :)


I have to add a static route

First thanks for your scripts!
I'm facing a problem, after the connection/server change I need to add (or update) a missing? static route to the upstream WG server endpoint.
On WG session establishment, my (internet) routes are as bellow, and the tunnel doesn't work.
Internet:
Destination        Gateway            Flags     Netif Expire
default            <IP_ADDRESS>        UGS         wg0
<IP_ADDRESS>        wg0                UHS         wg0
<IP_ADDRESS>      link#14            UH          wg0
<IP_ADDRESS>          link#7             UH          lo0
[…]

I have to manually create a static route, which I have to maintain myself in order to obtain :
Internet:
Destination        Gateway            Flags     Netif Expire
default            <IP_ADDRESS>        UGS         wg0
<IP_ADDRESS>        wg0                UHS         wg0
<IP_ADDRESS>      link#14            UH          wg0
<IP_ADDRESS>          link#7             UH          lo0
<IP_ADDRESS>/32  <IP_ADDRESS>        UGS         re0   # <--- added

The created WG local configuration has the Disable Routes ticked. The associated comment "This will prevent installing routes. Usually you only enable this to do own routing decisions via a local gateway and gateway rules.". I suppose this is why such route is not created for this tunnel.
I gave PIAWireguard.py  script a look and didn't found a part for automatic route management.
Do I miss something?
Thanks!

Hey,
I'm not sure why your default route after running the script is via WireGuard, because it shouldn't be. What should happen is the routing tables get two new routes. PIA's WG gateway IP and the IP of your assigned IP. Then you use the gateway you created to create Policy based routing rules (Firewall Rules).
I followed the guide on a test device this weekend, to making sure the guide works correctly, without error on OPNsense 20.7.7_1. Which update of OPNsense are you on?
I've also got it running on 3 other production OPNsense Firewalls without this issue occurring, so very strange.
Ticking "Disable Routes" is basically a requirement for this to all work correctly, otherwise you'll have routing issues with the Firewall itself, like your getting.
Example of my routes (taken the local network ones out)
Internet:
Destination        Gateway            Flags     Netif Expire
default            80-46-0-1.dynamic. UGS        igb0
<IP_ADDRESS>        wg1                UHS         wg1  # <--- added by WireGuard (PIA Gateway)
<IP_ADDRESS>       link#19            UH          wg1 # <--- added by WireGuard (Your PIA assigned IP)
<IP_ADDRESS>/20       link#1             U          igb0
<IP_ADDRESS>        link#1             UHS

FingerlessGloves

Hey,
I'm not sure why your default route after running the script is via WireGuard, because it shouldn't be. What should happen is the routing tables get two new routes. PIA's WG gateway IP and the IP of your assigned IP. Then you use the gateway you created to create Policy based routing rules (Firewall Rules).
I followed the guide on a test device this weekend, to making sure the guide works correctly, without error on OPNsense 20.7.7_1. Which update of OPNsense are you on?
I've also got it running on 3 other production OPNsense Firewalls without this issue occurring, so very strange.
Ticking "Disable Routes" is basically a requirement for this to all work correctly, otherwise you'll have routing issues with the Firewall itself, like your getting.
Example of my routes (taken the local network ones out)
Internet:
Destination        Gateway            Flags     Netif Expire
default            80-46-0-1.dynamic. UGS        igb0
<IP_ADDRESS>        wg1                UHS         wg1  # <--- added by WireGuard (PIA Gateway)
<IP_ADDRESS>       link#19            UH          wg1 # <--- added by WireGuard (Your PIA assigned IP)
<IP_ADDRESS>/20       link#1             U          igb0
<IP_ADDRESS>        link#1             UHS

FingerlessGloves

Thanks for your fast response. I wish all my ipv4 traffic to go through the vpn, so the default rule seems to make sense for me. We seems to have different configurations/goals, isn't it?

Thanks for your fast response. I wish all my ipv4 traffic to go through the vpn, so the default rule seems to make sense for me. We seems to have different configurations/goals, isn't it?

I may try to add a way to conditionally manage a static route in your script. I've never worked with the OPNsense api before (comming from PF since a few days). It may take me a long time :/ Do you have some input or tips to implement that? Thanks

I may try to add a way to conditionally manage a static route in your script. I've never worked with the OPNsense api before (comming from PF since a few days). It may take me a long time :/ Do you have some input or tips to implement that? Thanks

I think your going about it incorrectly atleast in my eyes, you could probably fudge something to get the routes on the OPNsense device to go via PIA but the issue would then be, if the tunnel drops the routes wouldn't remove themselves because of how WireGuard works, so when the script needs to get a new server it can't contact PIA to get another WireGuard server, which would happen at least every 2 months, due to PIA updating and restarting servers roughly in that period.
This is basic overview of how I would set this up to keep your devices using PIA only without messing with OPNsense too much.
Revert the PIA WireGuard setup back to how it is in the Guide/Script.
Go in to OPNsense WebUI, Firewall: NAT: Outbound. Change the Mode of the outbound NAT to manual.
Now create a New NAT rule, similar to the one below

Interface: WAN_PIAWG
SRC: internal network (I use ALIAS but you can use LAN network),
DEST: ANY
Translation / target: Interface Address
Leave the rest default

Now on your LAN rules, on the rule which allows traffic to the internet, set the gateway to WAN_PIAWG_IPv4.
Now because you don't have a NAT rule to NAT traffic from your LAN to WAN, you remove the risk of leaking if the PIA VPN tunnel comes down.
You should have 2 firewall rules, that says something like, if you want to restrict it more that's up to you.
Rule 1

SRC: LAN Network
DEST LAN Network
Gateway: Default
Description: All Lan to LAN

Rule 2

SRC: LAN Network
DEST !LAN Network
Gateway: WAN_PIAWG_IPv4
Description: All Internet traffic.

Make sure the rules are in this order.
I hope this idea makes sense, very basic way of having a network only use PIA for Internet access.
FingerlessGloves

I think your going about it incorrectly atleast in my eyes, you could probably fudge something to get the routes on the OPNsense device to go via PIA but the issue would then be, if the tunnel drops the routes wouldn't remove themselves because of how WireGuard works, so when the script needs to get a new server it can't contact PIA to get another WireGuard server, which would happen at least every 2 months, due to PIA updating and restarting servers roughly in that period.
This is basic overview of how I would set this up to keep your devices using PIA only without messing with OPNsense too much.
Revert the PIA WireGuard setup back to how it is in the Guide/Script.
Go in to OPNsense WebUI, Firewall: NAT: Outbound. Change the Mode of the outbound NAT to manual.
Now create a New NAT rule, similar to the one below

Interface: WAN_PIAWG
SRC: internal network (I use ALIAS but you can use LAN network),
DEST: ANY
Translation / target: Interface Address
Leave the rest default

Now on your LAN rules, on the rule which allows traffic to the internet, set the gateway to WAN_PIAWG_IPv4.
Now because you don't have a NAT rule to NAT traffic from your LAN to WAN, you remove the risk of leaking if the PIA VPN tunnel comes down.
You should have 2 firewall rules, that says something like, if you want to restrict it more that's up to you.
Rule 1

SRC: LAN Network
DEST LAN Network
Gateway: Default
Description: All Lan to LAN

Rule 2

SRC: LAN Network
DEST !LAN Network
Gateway: WAN_PIAWG_IPv4
Description: All Internet traffic.

Make sure the rules are in this order.
I hope this idea makes sense, very basic way of having a network only use PIA for Internet access.
FingerlessGloves

Thanks :+1: , I'll give it a try, probably tomorrow. I'll keep you informed.

Thanks :+1: , I'll give it a try, probably tomorrow. I'll keep you informed.

Sounds good, any questions just ask.
Make sure you got your phone's internet to hand encase you break something 🤭
Night

Sounds good, any questions just ask.
Make sure you got your phone's internet to hand encase you break something 🤭
Night

So, I've simply enabled the automatic gateway management by wireguard (https://github.com/kumy/OPNsensePIAWireguard/commit/9269e404df6f75d9afa19ea588df0f63dde070c8) and it now work as I wish.
My routes are now:
# (I have 2 tunnels)
root@gw:~ # netstat -rn4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
<IP_ADDRESS>/1          wg0                US          wg0
default            <IP_ADDRESS>        UGS         re0
<IP_ADDRESS>        link#14            UH          wg0
x.x.x.x            <IP_ADDRESS>        UGHS        re0
<IP_ADDRESS>          link#7             UH          lo0
<IP_ADDRESS>/1        wg0                US          wg0
y.y.y.y/32         <IP_ADDRESS>        UGS         re0
<IP_ADDRESS>/24     link#1             U           re0
<IP_ADDRESS>      link#1             UHS         lo0
<IP_ADDRESS>/24   link#2             U           re1
<IP_ADDRESS>    link#2             UHS         lo0
<IP_ADDRESS>/24   link#10            U      ath0_wla
<IP_ADDRESS>      link#10            UHS         lo0
<IP_ADDRESS>/32   wg1                US          wg1
<IP_ADDRESS>      link#13            UH          wg1
root@gw:~ #

I have not tested your last solution, but it seems a bit overkill to define the gateway in each of my rules…
Feel free to close the issue if you wish, thanks again for this project and for your time.

So, I've simply enabled the automatic gateway management by wireguard (https://github.com/kumy/OPNsensePIAWireguard/commit/9269e404df6f75d9afa19ea588df0f63dde070c8) and it now work as I wish.
My routes are now:
# (I have 2 tunnels)
root@gw:~ # netstat -rn4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
<IP_ADDRESS>/1          wg0                US          wg0
default            <IP_ADDRESS>        UGS         re0
<IP_ADDRESS>        link#14            UH          wg0
x.x.x.x            <IP_ADDRESS>        UGHS        re0
<IP_ADDRESS>          link#7             UH          lo0
<IP_ADDRESS>/1        wg0                US          wg0
y.y.y.y/32         <IP_ADDRESS>        UGS         re0
<IP_ADDRESS>/24     link#1             U           re0
<IP_ADDRESS>      link#1             UHS         lo0
<IP_ADDRESS>/24   link#2             U           re1
<IP_ADDRESS>    link#2             UHS         lo0
<IP_ADDRESS>/24   link#10            U      ath0_wla
<IP_ADDRESS>      link#10            UHS         lo0
<IP_ADDRESS>/32   wg1                US          wg1
<IP_ADDRESS>      link#13            UH          wg1
root@gw:~ #

I have not tested your last solution, but it seems a bit overkill to define the gateway in each of my rules…
Feel free to close the issue if you wish, thanks again for this project and for your time.

You only need to define the gateway once in your last rule, which should be the rule that lets your clients to the internet.
Yeah that change should do the trick.
How are you going to cater for the issue of contacting the PIA server list when the tunnel up but no handshakes or traffic?

You only need to define the gateway once in your last rule, which should be the rule that lets your clients to the internet.
Yeah that change should do the trick.
How are you going to cater for the issue of contacting the PIA server list when the tunnel up but no handshakes or traffic?

How are you going to cater for the issue of contacting the PIA server list when the tunnel up but no handshakes or traffic?

I do not need to handle it as from what I've seen, it is automatically managed by wg. The route swap seems done only when handshake is valid (and restored when tunnel goes down). I had this conf on my other vpn since some weeks, without problem (but I manage the other side of the tunnel). If I encounter problems I'll report it here.

How are you going to cater for the issue of contacting the PIA server list when the tunnel up but no handshakes or traffic?

I do not need to handle it as from what I've seen, it is automatically managed by wg. The route swap seems done only when handshake is valid (and restored when tunnel goes down). I had this conf on my other vpn since some weeks, without problem (but I manage the other side of the tunnel). If I encounter problems I'll report it here.

Cool sounds good, glad it's working 🙂
I shall close the issue 🙂

Cool sounds good, glad it's working 🙂
I shall close the issue 🙂

@kumy, Just a quick one I was just thinking.
What does your routing table look like if you specify a false PIA server and save the wireguard config?
On my WireGuard setup I've got WireGuard peers (RoadWarrior setup) they've had no handshake what so ever but their route is still in the routing table.
How does the WireGuard <IP_ADDRESS>/0 route cope with this?
I do need to create a LAB WireGuard again at somepoint.

@kumy, Just a quick one I was just thinking.
What does your routing table look like if you specify a false PIA server and save the wireguard config?
On my WireGuard setup I've got WireGuard peers (RoadWarrior setup) they've had no handshake what so ever but their route is still in the routing table.
How does the WireGuard <IP_ADDRESS>/0 route cope with this?
I do need to create a LAB WireGuard again at somepoint.

I'm on another task, I'll check in hour or two, sorry.

I'm on another task, I'll check in hour or two, sorry.

Just fired up a test OPNsense, test client. to test this.
The routing doesn't remove itself and the script can't change to a working server cause well there's no internet connection anymore...
Did you say you don't get this issue?

Just fired up a test OPNsense, test client. to test this.
The routing doesn't remove itself and the script can't change to a working server cause well there's no internet connection anymore...
Did you say you don't get this issue?

I'll check this use case tomorrow, family is still using internet actively…
You want me to test:

have my OPNsense wireguard connected to a server
edit the Endpoint -> Endpoint Address IP to something that doesn't host a WG server
save / apply / restart wg daemon?
check routes and connection status

Do we agree on the test case procedure?

I'll check this use case tomorrow, family is still using internet actively…
You want me to test:

have my OPNsense wireguard connected to a server
edit the Endpoint -> Endpoint Address IP to something that doesn't host a WG server
save / apply / restart wg daemon?
check routes and connection status

Do we agree on the test case procedure?

Correct and then run the script in debug mode in the CLI, to see what errors are produced.
That method is exactly how I tested it.

Correct and then run the script in debug mode in the CLI, to see what errors are produced.
That method is exactly how I tested it.

Effectively this specific use case doesn't work. Are you anticipating that the script may receive a configuration from the API and the remote server is down?
The PIAWireguard.py could be updated to handle this case. At the beginning,

check the internet connectivity to the API
if it fails and the tunnel is already configured
then disable the tunnel and process to the rest of the script

What do you think?

Effectively this specific use case doesn't work. Are you anticipating that the script may receive a configuration from the API and the remote server is down?
The PIAWireguard.py could be updated to handle this case. At the beginning,

check the internet connectivity to the API
if it fails and the tunnel is already configured
then disable the tunnel and process to the rest of the script

What do you think?

Yeah your gonna need to add a ping check, then if no response bring the tunnel down. Of course this check is only needed if your trying to run the script in your setup, so I'm not going to update the main repo. As modifying the default route of the OPNsense in my opinion isn't the best way of setting this up.
I wouldn't worry about PIA API being down, I would think that's in some form of HA setup. Plus PIA have everything service monitored.
Your going to want to implement some form of VPN kill switch for your clients, so when it's brought down, only the OPNsense itself has internet access, though the normal WAN.

Yeah your gonna need to add a ping check, then if no response bring the tunnel down. Of course this check is only needed if your trying to run the script in your setup, so I'm not going to update the main repo. As modifying the default route of the OPNsense in my opinion isn't the best way of setting this up.
I wouldn't worry about PIA API being down, I would think that's in some form of HA setup. Plus PIA have everything service monitored.
Your going to want to implement some form of VPN kill switch for your clients, so when it's brought down, only the OPNsense itself has internet access, though the normal WAN.


error--- no matching function for call to 'std::basic_ofstream

From<EMAIL_ADDRESS>on March 31, 2011 01:35:23
What steps will reproduce the problem? 1.build.py --install 2. 3. What is the expected output? What do you see instead? What version of the product are you using? On what operating system? 2.0 Please provide any additional information below. in ocelot/executive/implementation/PassThroughDevice.cpp   line 604
std::ofstream file(stream.str());
should be
std::ofstream file((stream.str()).c_str());
the same in ocelot/graphs/implementation/DivergenceDrawer.cpp:187
Original issue: http://code.google.com/p/gpuocelot/issues/detail?id=50

From<EMAIL_ADDRESS>on April 08, 2011 14:27:33
Fixed in r1164 , please verify.
Status: Fixed


📝 Correction | Canary Mail

Description
Canary Mail is also available for Android and MacOS, but it's only listed for iOS currently.
Why I am making the suggestion
n/a

My connection with the software
N7a

[x] I will keep the issue up-to-date if something I have said changes or I remember a connection with the software.

Source: https://canarymail.io/

AFAIK it is only listed for iOS because that is the best option in their garden, but it is not the best solution, K-9 is way better and it is libre, that is why it is recommended. Once some libre project is created for their platform Canary will probably be delisted. I would close this issue.

Good to know

Yes what @LongJohn-Silver said. While we do prefer open source recommendations, when none exist, options are limited.


Support hashable-1.4

https://hackage.haskell.org/package/hashable-<IP_ADDRESS>/changelog

I was wondering whether we should update constraints like (Eq k, Hashable k). @phadej has convinced me not to do this for now in https://github.com/haskell-unordered-containers/hashable/issues/232#issuecomment-956200669. Instead, we can consider requiring hashable >= 1.4 in a few months and enabling -Wredundant-constraints then.

Ping @treeowl. I'd like to make a release including this patch soon.

I checked that CI indeed picks up hashable-<IP_ADDRESS> as a dependency. Ping @treeowl .
In the meantime, @sjakobi do you mind adding 9.2.1 job to CI?

Sounds like a plan.

@phadej I've checked locally that this branch builds with GHC 9.2.1.


Add Docker to this project

this is a way to implement docker for your project.

Thank you for your first PR and my apologies for taking a while to respond.
Would it be possible to add layer caching to the PR for faster building?
Also what are your thoughts on using a different base such as:

scratch
alpine
bullseye-slim

My goal with these questions is to explore how we can create faster and more efficient Docker builds.

I'm currently learning docker and so i'll see what I can to lightweigth the images

Did you try to build the image ?
For me i get with thé actual version
user@machine: docker images
dipc-test.   92.6Mb

And with rust:slim
user@machine: docker images
dipc-slim.      86.1Mb

The major problem I get is when i use alpine or anything like this I get missing librairies for thé launchable docker
And if I compile with rust:alpine there is linking error.
Must i can save is using slim images.

Interesting. I will work on this and get back to you as soon as I can.

i'll finally found out to compile and have a workaround with alpine
now:
localhost/dipc            latest      bc8c90d9b0bc  5 minutes ago   16.4 MB
much better but there is maybe a better way to split the built to optimize cargo build (maybe in release mode) I don't use rust that much

Great work! We can focus on performance improvements later on, for now the PR just needs minor adjustments before being merged.

I still learning and I appreciate that i was not judge by my first PR
if you need anything else let me know

Don't worry, we're here to learn and your PR is a great contribution to dipc.
I will finish reviewing the PR this week. Great job and once again thank you!


[BUG] Circular import error with PyTorch nightly

Describe the bug
Circular import error with PyTorch nightly. If I uninstall deepspeed it works fine.
Traceback (most recent call last):
  File "/test/oss.py", line 322, in <module>
    mp.spawn(
  File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 283, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method="spawn")
  File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 239, in start_processes
    while not context.join():
  File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 190, in join
    raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/multiprocessing/spawn.py", line 77, in _wrap
    fn(i, *args)
  File "/test/oss.py", line 160, in train
    model = DDP(model, device_ids=device_ids, find_unused_parameters=False)  # type: ignore
  File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 900, in __init__
    optimize_ddp = torch._dynamo.config._get_optimize_ddp_mode()
  File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/__init__.py", line 2561, in __getattr__
    return importlib.import_module(f".{name}", __name__)
  File "/opt/conda/envs/ptca/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/_dynamo/__init__.py", line 3, in <module>
    from . import convert_frame, eval_frame, resume_execution
  File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 50, in <module>
    from . import config, exc, trace_rules
  File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/_dynamo/trace_rules.py", line 45, in <module>
    from .variables import (
  File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/_dynamo/variables/__init__.py", line 96, in <module>
    from .torch import TorchCtxManagerClassVariable, TorchInGraphFunctionVariable
  File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/_dynamo/variables/torch.py", line 137, in <module>
    torch._dynamo.external_utils.is_compiling: True,
AttributeError: partially initialized module 'torch._dynamo' has no attribute 'external_utils' (most likely due to a circular import)

To Reproduce
Steps to reproduce the behavior:

Install Pytorch nightly
Install Deepspeed
Clone fairscale repo: https://github.com/facebookresearch/fairscale
cd benchmarks/oss.py
Run python oss.py

Expected behavior
The script should run without any errors.
ds_report output
DeepSpeed C++/CUDA extension op report
NOTE: Ops not installed will be just-in-time (JIT) compiled at
runtime if needed. Op compatibility means that your system
meet the required dependencies to JIT install the op.
JIT compiled ops requires ninja
ninja .................. [OKAY]
op name ................ installed .. compatible
async_io ............... [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
cpu_adam ............... [NO] ....... [OKAY]
cpu_adagrad ............ [NO] ....... [OKAY]
cpu_lion ............... [NO] ....... [OKAY]
[WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
evoformer_attn ......... [NO] ....... [NO]
[WARNING]  NVIDIA Inference is only supported on Ampere and newer architectures
[WARNING]  FP Quantizer is using an untested triton version (3.0.0+dedb7bdf33), only 2.3.0 and 2.3.1 are known to be compatible with these kernels
fp_quantizer ........... [NO] ....... [NO]
fused_lamb ............. [NO] ....... [OKAY]
fused_lion ............. [NO] ....... [OKAY]
inference_core_ops ..... [NO] ....... [OKAY]
cutlass_ops ............ [NO] ....... [OKAY]
transformer_inference .. [NO] ....... [OKAY]
quantizer .............. [NO] ....... [OKAY]
ragged_device_ops ...... [NO] ....... [OKAY]
ragged_ops ............. [NO] ....... [OKAY]
random_ltd ............. [NO] ....... [OKAY]
[WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.5
[WARNING]  using untested triton version (3.0.0+dedb7bdf33), only 1.0.0 is known to be compatible
sparse_attn ............ [NO] ....... [NO]
spatial_inference ...... [NO] ....... [OKAY]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
DeepSpeed general environment info:
torch install path ............... ['/opt/conda/envs/ptca/lib/python3.10/site-packages/torch']
torch version .................... 2.5.0.dev20240815+cu118
deepspeed install path ........... ['/opt/conda/envs/ptca/lib/python3.10/site-packages/deepspeed']
deepspeed info ................... 0.14.5, unknown, unknown
torch cuda version ............... 11.8
torch hip version ................ None
nvcc version ..................... 11.8
deepspeed wheel compiled w. ...... torch 2.5, cuda 11.8
shared memory (/dev/shm) size .... 330.54 GB
Screenshots
If applicable, add screenshots to help explain your problem.
System info (please complete the following information):

OS: Ubuntu 20.04
GPU count and types: 1 Node, with 8 V100
Interconnects (if applicable): No
Python version: 3.10.14
Any other relevant info about your setup

Launcher context
Are you launching your experiment with the deepspeed launcher, MPI, or something else?
No
Docker context
Are you using a specific docker image that you can share?
Additional context
Add any other context about the problem here.

Hi @ajindal1 - I am trying to repro this but hitting an issue.
First I wanted to confirm that in a venv, if you following the following steps that you hit no issues?

Install Pytorch nightly
Clone fairscale repo and cd into it
pip install -r requirements.txt
pip install -r requirements-benchmark.txt
cd benchmarks
python oss.py

Doing this, I'm still hitting a ModuleNotFoundError: No module named 'benchmarks' issue I'll investigate, but I wanted to know if there was a reason that you thought to uninstall DeepSpeed?
Also sharing my pip list here:
Package                  Version
------------------------ ------------------------
attrs                    24.2.0
black                    22.3.0
certifi                  2024.7.4
cffi                     1.17.0
cfgv                     3.4.0
charset-normalizer       3.3.2
click                    8.1.7
coverage                 7.6.1
distlib                  0.3.8
docutils                 0.17
fairscale                0.4.13
filelock                 3.13.1
flake8                   4.0.1
flake8-annotations       2.7.0
fsspec                   2024.6.1
identify                 2.6.0
idna                     3.7
iniconfig                2.0.0
isort                    5.10.1
Jinja2                   3.1.4
joblib                   1.4.2
MarkupSafe               2.1.5
mccabe                   0.6.1
mpmath                   1.3.0
mypy                     0.910
mypy_extensions          0.4.4
networkx                 3.3
nodeenv                  1.9.1
numpy                    1.22.0
nvidia-cublas-cu12       <IP_ADDRESS>
nvidia-cuda-cupti-cu12   12.4.127
nvidia-cuda-nvrtc-cu12   12.4.127
nvidia-cuda-runtime-cu12 12.4.127
nvidia-cudnn-cu12        <IP_ADDRESS>
nvidia-cufft-cu12        <IP_ADDRESS>
nvidia-curand-cu12       <IP_ADDRESS>
nvidia-cusolver-cu12     <IP_ADDRESS>
nvidia-cusparse-cu12     <IP_ADDRESS>
nvidia-nccl-cu12         2.21.5
nvidia-nvjitlink-cu12    12.4.127
nvidia-nvtx-cu12         12.4.127
packaging                24.1
parameterized            0.9.0
pathspec                 0.12.1
pgzip                    0.3.1
Pillow                   9.3.0
pip                      24.0
platformdirs             4.2.2
pluggy                   1.5.0
pre-commit               3.8.0
py                       1.11.0
pycodestyle              2.8.0
pycparser                2.22
pyflakes                 2.4.0
pygit2                   1.11.1
pynvml                   8.0.4
pytest                   7.0.0
pytest-cov               3.0.0
pytest-timeout           2.1.0
pytorch-triton           3.0.0+dedb7bdf33
PyYAML                   6.0.2
remote-pdb               2.1.0
requests                 2.32.3
scikit-learn             1.1.3
scipy                    1.11.4
sentencepiece            0.2.0
setuptools               65.5.0
six                      1.16.0
sympy                    1.13.1
threadpoolctl            3.5.0
timm                     0.3.4
toml                     0.10.2
tomli                    2.0.1
torch                    2.5.0.dev20240822+cu124
torchaudio               2.4.0.dev20240822+cu124
torchtext                0.6.0
torchvision              0.20.0.dev20240822+cu124
tqdm                     4.66.5
typing_extensions        4.12.2
urllib3                  2.2.2
virtualenv               20.26.3

@loadams thanks for looking into it. So, the error only occurs when both deepspeed and onnxruntime-training are installed. The reason I suspected that removing deepspeed can help is because of my past experiences, for example this. It could be an issue with PyTorch as well as we found out in that issue.
To reproduce the error:
# Use docker image: nvcr.io/nvidia/pytorch:24.06-py3

# Uninstall current PyTorch and install nightly Pytorch
# Install deepspeed and onnxruntime-training
# Install fairscale
# Clone fairscale repo: https://github.com/facebookresearch/fairscale
cd fairscale
python benchmarks/oss.py

Just so you know, I got the same issue completely unrelated to DeepSpeed (I. didn't know the package until I came here).
I think the culprit might be pytest-timeout, since my AttributeError: partially initialized module 'torch._dynamo' has no attribute 'external_utils' (most likely due to a circular import) only arose after I started using it in my tests...!
Also, I have absolutely no idea how to fix it :)

Interesting, especially that you have this issue even if you don't have deepspeed installed.


locust close connections for each request

Locust user task uses requests library as underlying infra.
To ensure Locust loadbalances across new TGI pods with autoscaling, close the connection for each request so a new one is opened.

/gcbrun

/gcbrun

/gcbrun


Error MacOs M1: Could not create WasmInstance from valid module with context: unknown import: wasi_snapshot_preview1::fd_write has not been defined'

Hello, I started a new subgraph repo using the graph init cli from Thegraph documentation.
I had follow the matchstick install setup from Thegraph doc
When running graph test I'm getting the following error :

I'm using node version
╰─ node -v
v20.11.1

I'm also using homebrew that has install postgresql under /opt/homebrew/opt/postgresql@14/ folder, and I had copy the libpq.5.dylib where the doc suggest /usr/local/opt/postgresql/lib/libpq.5.dylib.
Here my repo dependencies:
    "dependencies": {
        "@graphprotocol/graph-cli": "0.74.1",
        "@graphprotocol/graph-ts": "0.32.0"
    },
    "devDependencies": {
        "@commitlint/cli": "^19.3.0",
        "@commitlint/config-conventional": "^19.2.2",
        "cross-env": "^7.0.3",
        "env-cmd": "^10.1.0",
        "husky": "^9.0.11",
        "matchstick-as": "^0.6.0",
        "mustache": "^4.2.0"
    },

Found the issue.
Error is trigger by a console.log


Add open source guidelines for this repository

Description
Adding open source guidelines will help the open source contributors know how to contribute and what needed to be followed to contribute to the project.
Suggested solution
Add the following guidelines:

contribution guidelines
code of conduct
commit convention
pull request guidelines

Alternative
No response
Additional context
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[ Triaged ] Good to start developing.


[rbuild][cget] OPTIONAL_AMDGPU_TARGETS is required

[Discussion]:
We need something similar to the following in either CK or MIOpen:
https://github.com/ROCmSoftwarePlatform/rocSOLVER/blob/f68a8839a514be6d908967db1fa60de9e23a17de/CMakeLists.txt#L134
Option 1: Let CK implement this so no changes are necessary in rbuild or cget
Option 2: let rbuild or cget implement this.
[Context]:
gfx941 and gfx942 should be optional GPU Targets, if the compiler being used cannot support it, we should be skipping it all together.

Do we also still have GPU_TARGETS option support? I presume, this check should also filter out unsupported targets from GPU_TARGETS.

So, if I understand the task correctly, we just need to add a check before we start building:

Check if all GPU_TARGETS are supported by the compiler being used;
Leave only those targets that are supported.
Did I miss anything?

@illsilin @dmikushin yes I think this check should be enabled in CK library :)

It looks like this target availability check is based on a function "rocm_check_target_ids", which is used not only by rocSOLVE, but by other libs, like rocThrust. But I don't see it in any of our dockers. So I'll need to first figure out how to build this function. May need some changes to our rocm installation.

It should be part of rocm-cmake.

You need to include the ROCMCheckTargetIds module.

@pfultz2 can you help to get it into rocm-cmake? Currently it becomes a blocking issue since the latest public ROCm release does not yet support these targets, if we add them to the requirements.txt line, there must be a way they are checked and be optional (with a warning message). Thanks!

I managed to run that function for the multiple targets:
GPU_TARGETS= gfx90a
checking if targets are supported
-- Performing Test COMPILER_HAS_TARGET_ID_gfx803
-- Performing Test COMPILER_HAS_TARGET_ID_gfx803 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx900_xnack_off
-- Performing Test COMPILER_HAS_TARGET_ID_gfx900_xnack_off - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx906_xnack_off
-- Performing Test COMPILER_HAS_TARGET_ID_gfx906_xnack_off - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx908_xnack_off
-- Performing Test COMPILER_HAS_TARGET_ID_gfx908_xnack_off - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx90a_xnack_off
-- Performing Test COMPILER_HAS_TARGET_ID_gfx90a_xnack_off - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx90a_xnack_on
-- Performing Test COMPILER_HAS_TARGET_ID_gfx90a_xnack_on - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx940
-- Performing Test COMPILER_HAS_TARGET_ID_gfx940 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx941
-- Performing Test COMPILER_HAS_TARGET_ID_gfx941 - Failed
-- Performing Test COMPILER_HAS_TARGET_ID_gfx942
-- Performing Test COMPILER_HAS_TARGET_ID_gfx942 - Failed
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1030
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1030 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1100
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1100 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1101
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1101 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1102
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1102 - Success
So I'd like to next add a loop over the list of targets given to cmake to filter out the unsupported ones, something like this:
foreach(target IN LISTS GPU_TARGETS)
message("for target = ${target} the compiler support is:")
set(check_status rocm_check_target_ids(DEFAULT_AMDGPU_TARGETS TARGETS "${target}"))
if(NOT "${check_status}"  MATCHES "Success")
list(REMOVE_ITEM GPU_TARGETS "${target}")
message("removing target ${target} from the list")
else()
message("target ${target} is legit")
endif()
endforeach()
But it doesn't look like the rocm_check_target_ids is returning any value and I'm not sure how to pipe its output into a variable that I could parse for "Success" or "Failed".
Any ideas?

@illsilin , CMake macros usually do not return much.
Here is the actual code of rocm_check_target_ids: https://github.com/umangyadav/rocm-cmake/blob/master/share/rocm/cmake/ROCMCheckTargetIds.cmake#L8
It basically does:
set(_result_var "COMPILER_HAS_TARGET_ID_${_result_var}")
check_cxx_compiler_flag("-xhip --offload-arch=${_target_id}" "${_result_var}")

So in order to see what succeeded, check the value environment variable defined separately for each target.

I have finally figured out how to use that function. Basically, you can feed any list of gpu targets to it and it will just eliminate all the unsupported ones.
But now we are facing a new challenge. Looks like when you set the DGPU_TARGETS flag on the cmake command line it overrides whatever you do in the cmake files afterwards. So if we set GPU_TARGETS="gfx908;gfx90a;gfx941" on the command line, then run that list through rocm_check_target_ids, eliminate gfx941 from the list, at the make step the compiler will still try to build the gfx941 targets.

This is because of misunderstanding the concepts of CMake in general :) CMake macro-language philosophy is to define new things, not to modify the existing ones (and heroically fight with scope/precedence rules). The COMPILER_HAS_TARGET_ID_... will be defined as a result of the checks, and you can simply recollect their values into a new list and use it as you like.

Looks like when you set the DGPU_TARGETS flag on the cmake command line it overrides whatever you do in the cmake files afterwards. So if we set GPU_TARGETS="gfx908;gfx90a;gfx941" on the command line, then run that list through rocm_check_target_ids, eliminate gfx941 from the list, at the make step the compiler will still try to build the gfx941 targets.

This is because GPU_TARGETS is a cached variable. You can overwrite the cached variable, but that can have issues. I am not sure how well this would work when changing the variable ccmake. I dont think its a good option.
It would be better for CK to define the default set of targets it supports:
# Define the full list of targets that ck support
set(CK_GPU_TARGETS ...)
# Filter the list using rocm_check_target_ids
...
# Set GPU_TARGETS
set(GPU_TARGETS ${CK_GPU_TARGETS} CACHE STRING "")
# Now find hip
find_package(hip)

Then MIOpen wont set targets in requirements.txt, as by default ck will build with all targets that both the compiler and ck supports.


fix: silent email address trim

Just apply a trim on the email input value in a reusable way.

Also, I think you do not have the pre-commit installed for the repo.


fix issues causing vitest to puke

might be good idea to try using swc to build the SDK. it's faster and I read in https://github.com/microsoft/TypeScript/issues/51783 strips out import type assertions


scipy.sparse.linalg.eigs() sometimes returns wrong EigenVectors

I discovered the following irregularity in the behavior of scipy.sparse.linalg.eigs():
When running mentioned function several times on the same matrix, the returned EigenVectors are wrong in a specific pattern. Using the minimal example below, scipy failed for samples 3, 12, 28 and 42. This sequence is reproducible, even on multiple laptops.
Note that it seems independent of the python version (2.7.9 and 3.4.2) and the scipy version (I started in 0.13.3 and it persisted in 0.15.1).
This bug might relate to #3613 and the therein mention scilab bug report.
The complete code, including matrix definition, is available here.
import numpy as np
import scipy.sparse as sparse
import scipy.sparse.linalg as sla

# matrix definition
# data    = np.array(...)
# indices = np.array(...)
# indptr  = np.array(...)

matrix = sparse.csr_matrix( (data, indices, indptr), shape=(400,400) )

samples = 50

for s in range(samples):
        w, pi = sla.eigs( matrix, k=1, which="LM" )
        pi = pi.reshape(pi.size,)
        pi /= sum(pi)
        print("EV[0:10] for sample", (s+1), ":")
        print(pi[0:10])

Note that the "exceptional" eigenvectors you get are correct, but
correspond to a different eigenvalue.
.
One issue is that iterative methods in general cannot give 100%
guarantee that if you specify which="LM", you get the largest
eigenvalue; the algorithm does not calculate all eigenvectors so it
cannot be fully sure that it found the largest.
.
Since the initial vector is randomly selected in case it is not
explicitly given, you can have bad luck and not converge where you
expect in some cases (cf. v0 parameter for initial guesses). You can
increase the probability to converge to the correct result by increasing
the ncv (the number of auxiliary eigenvectors considered).
.
Since you are using which="LM", the arpack-ng issue mentioned above is
probably not related.

@pv: Thanks for the quick response. It really looks like increasing the number of auxiliary eigenvectors solves the problem.


stabelize bt advertising in shell

fixing randomly bluetooth disconnection, when operating from shell.
https://github.com/zephyrproject-rtos/zephyr/issues/73131
to:
uart:~$ bt advertise on
Advertising started
Connected: E0:0A:F6:7A:3A:80 (public)
LE data len updated: TX (len: 251 time: 2120) RX (len: 251 time: 2120)
Remote LMP version 5.2 (0x0b) subversion 0x17da manufacturer 0x005d
LE Features: 0x0000000134665fbd
_uart:~$_

create an issue: https://github.com/zephyrproject-rtos/zephyr/issues/73131 to avoid overloading the description at this place.

close depending issue. this fix (issue) depends heavily on our SDK.
thanks for comments, pipeline build and checks!


mp4 option

Can we export as mp4?

Not directly, because MP4 is not a supported format in browsers.
But you have a couple of options:

export WebM/Tar and convert to mp4
use ffmpegserver as frame encoder (haven't actually tested since the last refactor)

Let me know if it works!

Hey the plotly graph i am capturing freezes when i try to capture and save using ffmpegserver
Codepen : http://codepen.io/rudi8292/pen/pbQpgA
I just copy pasted my code in test.html to try it out

As I said, it's not tested. I don't support the FFMpegServer back-end for the time being, but you're welcome to fix it and do a PR.

I was able to make a workaround using timeout around requestAnimationframe


feat: [M3-6736] - VPC detail summary

Description 📝
Add VPC Detail Summary section with Subnets placeholder
Preview 📷

How to test 🧪
yarn test VPCDetail

yarn test packages/manager/src/features/VPC/utils.test.ts

Create a VPC, then go to it's details page and confirm the summary
If there is no description for the VPC, the description field should not display
If there is a description, you should see the field
Try editing/deleting the VPC

In the event that a user updates a VPC with a shorter description, is it expected for the layout to shift and for the "Read More" link to remain visible?


Only count GPU pass-through devices

@friver-ms please fill out the description and the goal there. Also post the test result of the change base testing.

Test results -
   ID TestArea             TestCaseName                                                                TestResult TestDuration(in minutes)
-------------------------------------------------------------------------------------------------------------------------------------------
    1 GPU                  NVIDIA-CUDA-DRIVER-VALIDATION-MAX-GPU                                             PASS                 9.11
	Using nVidia driver : CUDA
	lsvmbus: Expected "PCI Express pass-through" count: 4, count inside the VM: 4 : PASS
	lspci: Expected "3D controller: NVIDIA Corporation" count: 4, found inside the VM: 4 : PASS
	lshw: Expected Display adapters: 4, total adapters found in VM: 4 : PASS
	nvidia-smi: Expected GPU count: 4, found inside the VM: 4 : PASS
    2 GPU                  NVIDIA-GRID-DRIVER-VALIDATION-MAX-GPU                                             PASS                 16.9
	Using nVidia driver : GRID
	lsvmbus: Expected "PCI Express pass-through" count: 4, count inside the VM: 4 : PASS
	lspci: Expected "3D controller: NVIDIA Corporation" count: 4, found inside the VM: 4 : PASS
	lshw: Expected Display adapters: 4, total adapters found in VM: 4 : PASS
	nvidia-smi: Expected GPU count: 4, found inside the VM: 4 : PASS

@friver-ms Could you please clarify the detailed failure without your fix? Like in which distro which case?

@friver-ms Could you please clarify the detailed failure without your fix? Like in which distro which case?

This is what I asked to @friver-ms but here is brief information. New cluster will come out soon, which enabled SR-IOV device. So lspci command result shows an extra pci device. So if the test is running in new cluster, then it fails. However, the current cluster (ENDURE) passes the test. So he would like to omit this test.

@friver-ms please fill out the description above. Also please post the change base test result.

Updated description. I think Lili posted the test result


Book review web Responsive UI

Describe the bug

Book review website looks great on desktop and on tablet devices too but layout failed in mobile screen size
Expected behavior

I can fixed the issue so that the layout will look cool and consistent irrespective of any screen size
Screenshots (Mandatory)

Additional context (Mandatory )
Are you contributing under any open-source program ?

Hey @apu52
Can you please assign me this issue under codepeak'23
Thank you


SharingManager AutoDiscovery not working anymore

If using SharingService with schema version 15 on client and server everything works fine (as in HoloToolkit-Sharing-240).
With schema version 17 it isn't working anymore !

Most likely related to the major sharing update (https://github.com/Microsoft/HoloToolkit-Unity/pull/459) that introduced multiple problems.
Thanks for submitting.

I confirm, my application crashes when I try to use:
SharingManager.Create(config);

If anyone has a fix, I've already submitted PR https://github.com/Microsoft/HoloToolkit-Unity/pull/484 for other misc fixes for the sharing service.
Please submit a PR on my branch to get those changes included to the existing PR.

@Lordinaire are you using the SharingStage.cs class or do you have your own implementation?
I have not been able to get my application to crash on SharinManager.Create(config);

RESOLUTION
In the main HTK project launching the SharingService.exe always launches with schema v17.
If you're using the HoloToolkit from the unity package, or exported the package yourself chances are you  got your SharingService.exe from another source. You just need to replace that old executable with the newest version.
For example, in my project, I use a symbolic link to reference the HTK specific libraries in a folder named ThirdParty, and my External\HoloToolkit\Sharing folder doesn't get updated when the HTK updates. (I could just sym link those too, but I'm lazy).

Great catch @HodgsonSDAS !!

@HodgsonSDAS I'm using the SharingStage.cs class and the latest version of HTK. I create a very simple project with just basic sharing scripts (see attached file) and it crashes after few seconds on my HoloLens. No exception are thrown :/
PS : I removed the HTK to be able to upload the project.
SharedHologram.zip

Sorry @Lordinaire I'm not going to download your zip file.  You can share on github if you'd like.

Of course ! I put it on GitHub : https://github.com/Lordinaire/HoloLens_Sharing_Test
If anyone can try to repro the crash it will help me a lot.

Have you tried pulling my PR addressing some of the sharing services?

Hello,
I am working on a project using the sharing components of the HoloToolKit for Unity. I am using the current version of the HoloToolKit in a Unity and still receiving issues when launching the sharingservice.exe, similar to @Lordinaire. The sharingservice.exe does say Schema Version: 15 when launched, but as soon as I attempt to connect to the server address I receive the Error code 5, exactly as pictured above in the post from @StephenHodgson. I made sure to check the resolution previously posted, but I have already completed those steps and still have issues. Additionally, I was able to get the sharing services to work for the HoloToolKit-240, but am having issues here. Any updated resolutions or advice would be greatly appreciated!
Thank you in advance.

Hi have a problem with the HoloToolKit for Unity. I am using the current version of the HoloToolKit in a Unity and still receiving issues when launching the sharingservice.exe. When I add the ImportExportAnchorManager the app dies as it loads on hololens with no debug. Tracing it through it appears to be after line 225 in ImportExportAnchorManager which in my build reads  SharingStage.Instance.SharingManagerConnected += Connected;
private void Start()
{
// SharingStage should be valid at this point, but we may not be connected.
if (SharingStage.Instance.IsConnected)
{
Connected();
}
else
{
SharingStage.Instance.SharingManagerConnected += Connected;
}
}
The only debug I get after this line is in SharingStage.cs in HandleLog which is an Assert with the log message logString = "WinRTScriptingObjectWrapper::InternalAddRef Reference Count cannot be zero"
Any ideas gratefully received

@Siman007, I suggest opening a new issue instead of posting off topic. Thanks

HI, I have a problem with sharing service. I developed few hololens apps using HTK V 1.5.6. For all those apps I used sharing service executable file provided in HTK 1.5.6 with schema 15  hosted on a public IP.
Recently I  developed 2 hololens apps with VOIP features using HTK V 1.5.8. In order to make this app work, I updated my sharing server with the new version available in 1.5.8 with schema 17. After that all my old apps using sharing service stop working.
With out migrating my previous apps, how can I make the sharing server to understand schema 17 and previous schema 14.
Thank you in advance

@RaoAM  Please make a new issue for this.


Update Closure Compiler to v20160517 and fix existing warnings

Update google-closure-compiler to 20160517.1.0
Fix Closure Compiler warnings:
matchNode in match() cannot be optional / undefined, otherwise a
TypeError would be thrown in getData().

Cool, thanks for cleaning this up.

@sparhami I guess incremental-dom is checked by Closure internally and this repository is synced back to the internal one? It probably makes sense to promote type warnings to errors internally, so they can get caught more easily.

Internally, I just use an build artifact from this repo, but only for releases. The build did run Closure Compiler for checking as part of the commit hook, but by default it seems checkTypes helpfully just reports a warning.


how could we stitch images into a bigger one?

Jimp, another NodeJS library https://github.com/oliver-moran/jimp, has a convenient method blit that lets me paste several images into a bigger one.
In my application, I don't need stretching, but simply stitch vertically several images that have the same width, without overlap.
Would you have suggestions on how to do it with sharp (much faster than Jimp).

Hello, you could try a variant of the reduce-based approach taken in https://github.com/lovell/sharp/issues/405#issuecomment-208033263, modified so that the initial image is already of the final required dimensions and the call to overlayWith provides an incrementing top coordinate.

Thanks a lot I wouldn't have found it by myself.

Closing due to inactivity but please re-open if questions remain.

The link in your response @lovell redirects to some weird sites.

The link in your response @lovell redirects to some weird sites.

@kawka-maciej Thanks, have removed.

@kawka-maciej Thanks, have removed.


Fix long issue

After https://github.com/mhallsmoore/qstrader/pull/135

Coverage decreased (-0.2%) to 72.593% when pulling b7a65a949139253f84ff75bcc5f3279b0d138120 on femtotrader:int_long_int64 into 8f8dab3fb3622576b8d4da8cfa1f02ed99224d01 on mhallsmoore:master.


fixed IONOS icon

added an IONOS icon with a better aspect ratio to fit layout

Why are the two versions of MyWebsite now separate technologies? They should be combined with version detection.

Both versions have nothing in common than the marketing name. Version 8 is self developed by IONOS and available for existing customers only. Version 9 is based on Duda and actively sold. That's the reason for the differentiation.
If I get you right you suggest to combine them in one ruleset?

If the website and icon can be the same you can just add the new JS pattern the existing section. You don't need to combine the regular expressions into one.