Getting app write error

Hi,
I have been trying to use the latest app version 2.x and keep encountering following issue.
I am using the token base and everything works fine (able to receive notifications etc) and ~50% of the time it would return the following error after calling the provider.send() method. Only way to recover is to restart my node app.
Error after send method with 2 device token (take a while for the following error to return)
[{"device":"..masked..","error":{"jse_shortmsg":"apn write failed","jse_cause":{"code":"ETIMEDOUT","errno":"ETIMEDOUT","syscall":"read"},"jse_info":{},"message":"apn write failed: read ETIMEDOUT"}},{"device":"..masked...","error":{"jse_shortmsg":"apn write failed","jse_cause":{"code":"ETIMEDOUT","errno":"ETIMEDOUT","syscall":"read"},"jse_info":{},"message":"apn write failed: read ETIMEDOUT"}}]

I got the same. Usually I get this error after an interval (0.5 - 1 hour) of passive connection (i.e. when I keep APNS connection opened but don't send no notifications).

I haven't come across this issue, yet. My current guess is that APNs closes the idle connection and it is not correctly reestablished.
@cmdkoh @vzayko If you have any more insight, please let us know.

@florianreinhart is there a way to get some internal logs when APN connection is lost, reconnecting, etc?

Since the 2.x rewrite for the new HTTP/2 provider interface there isn't a lot of internal logging in node-apn. Setting DEBUG=apn enables logging for node-apn, but you probably won't get anything useful from that.
We should definitely add more debug logging. However, I am currently short on time. So any help would be appreciated.

Folks, any progress about it? Please share with us :)

I have to pull back my issue confirmation. In my case most likely it was broken json in rawPayload. At least when I fixed this issue on my side, I no longer saw this problem. I'll come back if anything change. I'm sorry!

@vzayko could you please elaborate on what might have been wrong with your payload?
We are seeing the same issue described by @cmdkoh but also sometimes getting
"status":"500","response":{"reason":"InternalServerError"}}
I have tried to enable DEBUG=apn and are not seeing anything in the logs

I am testing the solution mentioned by @Discountrobot ( #454 and Code => HealthCheck) - So far seems promising...

@cmdkoh If it seem to work please post PR, I'm having same issues as above.

@cmdkoh So how about your testing ?
Does it seem to solve the issue ?

I have a lot of InternalServerError after update the code with Healthcheck :(
@cmdkoh Did work with you?

@sambegin , @casevictor I tried setting heart rate (PING interval) to about 5 minutes internal and all seems good until it reaches the 1 hour Token validity threshold (imposed by APNS) then all sub-sequence push will result in "500 InternalServerError". I would expect APNS to return "403 ExpiredProviderToken" which the client.js should already been taken care...

@cmdkoh As apple's doc says :

If your provider certificate is revoked, or if the key you are using to sign provider tokens is revoked, close all existing connections to APNs and then open new connections.

When I look into client.js code, in Cient.prototype.write function :
`stream.on("end", () => {
if (status === "200") {
resolve({device});
} else if (responseData !== "") {
const response = JSON.parse(responseData);
        if (status === "403" && response.reason === "ExpiredProviderToken" && retryCount < 2) {
          this.config.token.regenerate(tokenGeneration);
          resolve(this.write(notification, device, retryCount + 1));
          return;
        }

        resolve({
          device,
          status,
          response
        });
      } else {
        let error = new VError("stream ended unexpectedly");
        resolve({
          device,
          error
        });
      }
    });`

Am I the only one who thinks that this code doesn't close "all existing connections to APNs and then open new connections" ?
At this point, it only seems to re-write another stream without closing connections after the 1 hour provider token is revoked.
What if we force close / re-open new connections at this time ?

@cmdkoh
I tested it myself ( not with health check ) by adding some log on client.js where we should have the "403 error"  and after an hour, I did have the 403 ExpiredProviderToken and the node-apn package seems to handle it correctly by updating token...
I was also able to generate a token with a "iat" (issue at) timestamp that would not be within the last hour to force the 403 error and every time the package responded correctly.
So unfortunately, I don't think this is the problem.
I would appreciate having some help from anyone who understand clearly how stream / endpoint  are managed to help us understand those errors I've seen described in #461 ...

@sambegin To double check again, I went back to original code (without PING Frame) and Now I have the normal behavior as you described above and I did not get any 500 error at all.... this is kind of invalidated all my previous tests/assumption about Ping Frame requirements. :(

FYI, we reverted to the latest 1.x version and it's been working great for 2 days

@philberg We (me at least) never had any issues with 1.x (certificate based), issues only occurred in 2.x (HTTP/2 with Auth Token based)

I've made more test and it seems that Internal server error occurs when there's no notification that are send for a long period of time. In apple's doc, it says :

You should leave a connection open unless you know it will be idle for an extended period of time—for example, if you only send notifications to your users once a day, it is acceptable practice to use a new connection each day.

We are in a context that we sure have more traffic on certain period in a day it does not need "a day" before we get 500 status. Maybe an hour or two of inactivity seems to cause the issue.

I am going to run some test with the following code changes and see... (Checking for 500 and unexpected end of stream end then close the connection)
return new Promise ( resolve => {
        stream.on("end", () => {
          if (status === "200") {
            resolve({ device });
          } else if (responseData !== "") {
            const response = JSON.parse(responseData);
            if (status === "403" && response.reason === "ExpiredProviderToken" && retryCount < 2) {
              this.config.token.regenerate(tokenGeneration);
              resolve(this.write(notification, device, retryCount + 1));
              return;
            }
            else if (status === "500" && response.reason === "InternalServerError") {
              stream.connection.close();
              let error = new VError("Error 500, stream ended unexpectedly");
              resolve({ device, error });
              return
            }
            resolve({ device, status, response });
          } else {
            stream.connection.close();
            let error = new VError("stream ended unexpectedly");
            resolve({ device, error });
          }
        });
        stream.on("unprocessed", () => {
          resolve(this.write(notification, device));
        });
        stream.on("error", err => {
          let error;
          if (typeof err === "string") {
            error = new VError("apn write failed: %s", err);
          } else {
            error = new VError(err, "apn write failed");
          }
          resolve({ device, error });
        });
        stream.end();
      });```

The node-http2 module, which we use for HTTP/2, uses bunyan for logging. I have added an option to pass a bunyan logger to the http2 module. This is currently on the branch logging, see #470.
To enable logging for the http2 module please do the following:

Update your package.json to use node-apn from the logging branch and include bunyan:

{
  "dependencies": {
    "apn": "https://github.com/node-apn/node-apn/archive/logging.tar.gz",
    "bunyan": "^1.8.4"
  }
}

Create a bunyan logger and pass it in the options when creating your provider:

const http2Logger = require("bunyan").createLogger({
    name: "http2",
    stream: process.stdout,
    level: "trace"
});

const provider = new apn.Provider({
    token: apnsToken,
    production: true,
    http2Logger: http2Logger
});

@sambegin, @philberg,

@sambegin, @philberg is your server hosted in Azrue when you guys encounter this issue?

No. We are on Galaxy ( Amazon meteor app platform)

@cmdkoh no, ours is self-hosted, vm on vmware

Can confirm I am seeing this issue with the latest release too. Any workaround available other than downgrading?

@cmdkoh my server is hosted on Azure but I get ECONNRESET after short periods of inactivity.

@jkoepcke We've implemented a "workaround" to let us time to find the real problem.
We've reset the provider (shutdown and creating new one) every 50 minutes... It's been three days and we did not have any error yet again. Note that this is only a "dirty fix" before finding what's the real problem.

@sambegin also Apple has some docs that state that we should not close and re-open connections too often (their example mentioned 1 day period as normal). The docs are destroyed now though... looks like Apple is pushing some new docs out.

@quixoticaxis I know, I've read that doc again and again ... :) But for now we are in a situation where our users did not received notifications at all... And I told myself, "what if you deploy in prod every 1 hour ? (That can happen in our case) Your connections between Apple would also restart each hour... " So clearly it needs to support that.
For now this is just a work around as I said, this is not the way we should handle that but I've put hours and hours in this package to find what's the real cause of the problem and did not find anything... Need more time on that ;)

I'm also killing and respawning a connection every hour.

Can i use temporarily the following option to fix this problem:
var apnsProvider = new apns.Provider({
  production: true,
  token: tokenObject
});

apnsProvider.client.shutdownPending = true;   // this option adds auto disconnect every time when queue of pending notifications is fully drained

P.S. I'm really looking forward to a stable solution to this problem. But i need a temporary solution.

@sambegin Similar to your "dirty fix", when I get a failed response I reset the provider variable and call the send function once again, like this:
    provider.send(note, deviceToken).then( (result) => {
        if (response.failed) {
            provider = new apn.Provider({
                        // Token, etc.
                }
            });

            // Try again :( ...
            provider.send(note, deviceToken).then( (result) => {
            });
        }
    });

@AndySoHotRightNow I do the same but I also assign a number to a provider and bind the same number to the callbacks, so in situation when I get errors I can check if I'm currently going to shutdown the same provider I worked with.

@AndySoHotRightNow I think you should instead look for Internal server error or other error (like stream write failed... ) before closing and re opening provider.
Lots of failure can be associated to device tokens and are not "errors". Look at https://developer.apple.com/library/content/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/CommunicatingwithAPNs.html#//apple_ref/doc/uid/TP40008194-CH11-SW1
export const BAD_DEVICE_TOKEN = 'BadDeviceToken';
export const DEVICE_TOKEN_NOT_FOR_TOPIC = 'DeviceTokenNotForTopic';
export const INVALID_PROVIDER_TOKEN = 'InvalidProviderToken';
export const UNREGISTERED = 'Unregistered';
export const TOPIC_DISALLOWED = 'TopicDisallowed';

On those response from apple, you should remove device tokens you've registered (if you did) and you don't have to create a new provider because clearly the service is working properly.

NOTE AGAIN, that this is a temporary fix, clearly we have to find the real cause of the problem...

Hi Guys,
I've been testing my changes and is been a week and it seems like it solve my issue. Two things I changed.

Adding Ping Frame (interval 60 seconds): This take care of my issue with hosting in Azure which disconnect TCP connection after 4 minutes idling - was mentioned here

Add Token expiration check prior to sending Push: This take care of issue from APN when no 403 is returned (when token is expired over certain amount of time, e.g. x mins). Currently hardcoded to 55 minutes threshold - if token is issued over 55 minutes, a new token will be regenerated

If you guys are interested, you can test it using below GitHub repository (i submitted a pull request, hope the main contributor can verify the code and suggest better approach)

Update your package.json to use node-apn from the my branch:

{
  "dependencies": {
    "apn": "https://github.com/cmdkoh/node-apn/archive/fix_Getting_app_write_error_%23449.tar.gz"
  }
}

You can change the default heartbeat of 60 seconds in config (in milliseconds)

var options = {
  token: {
    key: "path/to/key.p8",
    keyId: "...",
    teamId: "...",
  },
  production: false,
  heartBeat: 30000
};

var apnProvider = new apn.Provider(options);

@cmdkoh Probably I will test tomorrow! Thanks :)

Anything I can do to help out? - Eager to see this resolved

I met this error too
I created 10 provider, 10 connection each provider, and lots of apn write VError logged. Not as you guys said happens when connection is idle.
so what's the right gesture, recreate a provider instance ? Or just downgrade to 1.x ?

so will the TimeoutError be rejected ? I only saw the console logged

@florianreinhart - I think we should make a call. This is essentially making the library unusable with Token based authentication. I think we should get PR by @cmdkoh reviewed ASAP, check-in and release a patch such that more ppl can test it.
If there is no clear evidence that that PR resolves all the problems, we should make it clear in the docs that Token authentication has issues.

@casevictor Did you get a chance to test it? It has been in good shape at our production env for almost a month now. Only thing is the ..11 wakeup listeners added.. warning which I still do not have a chance to dig inside.

I have tested several times and I got two types of errors.

Establish a connection to APNs by sending at least one push notification and idle it for a long time(almost one day). I find that this connection would not be closed, since 'GOAWAY' handler would not be called.
DID NOT send any push notification after initializing a provider and idle it for a long time (almost one day). Then I tried to send push notification, and I firstly received a 403 error then endless 500 error.

My points are as follows (I am not quite familiar with HTTP2, I'm learning about it).

'GOAWAY' frame received from APNs isn't handled correctly.
The token in stream header may be timeout after idling for a long time.

It would be very appreciated if you guys give me some suggestions.

@zaumreit Can you try again using the latest commit? (I updated the comment here https://github.com/node-apn/node-apn/issues/449#issuecomment-262670555)

@cmdkoh Sure, I have applied your approach to refresh JWT token and seems that there is few 403 error anymore which finally causes 500 error before. Thank you very much :)

@cmdkoh
I'm using "apn": "https://github.com/cmdkoh/node-apn/archive/fix_Getting_app_write_error_%23449_part4.tar.gz"
Solved my  Internal server error.

@xuduo
i have status 400 with your code.

I have this issue too on self hosted server. Added logging of errors, will see what type of error I'm getting when it fails.

Ok, error I got when sending push:

"apn write failed: connection error: Error: stream.push() after EOF"

This solution didn't help:

apnsProvider.client.shutdownPending  = true;

@xuduo i am wrong, this fix worked exactly!

@Discountrobot I left my app working for about 20 hours and then made action which triggers push notification sending, push was not sent and I got that error

@Discountrobot node.js 7.2.0, node-apn 2.1.2, token based auth

I am experiencing the this issue with certificate based authentication (pfx).
I am running a self hosted server with node.js 6.9.4 and node-apn 2.1.2.

I'm experiencing the same issue with node 7.2.0, node-apn 2.1.2 running on heroku using token based auth.

Hey, friends. I'm the primary author of Pushy, and we're receiving reports of similar issues. Since we're seeing similar behavior (HTTP/500 after a period of inactivity) with an entirely separate codebase on an entirely separate platform (Java), I'm pretty convinced that this is an upstream problem.
To be clear, I think I'm hearing two different issues here, and want to make sure we're on the same page about which one we're also seeing:

If left completely idle for a long time (i.e. no PING frames), the connection will become completely unresponsive.
If a connection is left idle except for PING frames for a long time, messages sent with an (expired?) authentication token on that connection will result in an HTTP/500 error instead of a "token is expired" error.

If that's accurate, let me be clear that we're seeing the second problem, but not the first. My strong recommendation in that case is to file a bug report with Apple about the issue if you have been affected and can reproduce the issue.

I've reported this to Apple as rdar://30168506.

@jchambers any news by apple ?
For your information, we've made a dirty fix 3 months ago (by resetting the provider every 1 hour
--> https://github.com/node-apn/node-apn/issues/449#issuecomment-260354323)
and everything was working fine (no error from apple, all notification received) until today...
We are getting several "apn write failed: write ECONNRESET" on almost every notification sent. Weird thing is that maybe 80 % of notifications are still delivered correctly on devices, so we don't know why we are getting so many ECONNRESET errors from the provider.
Hope this information could help. Hope someone at apple could also help us on that.

any news by apple ?

Negative.

Hope someone at apple could also help us on that.

Seems worthy of its own bug report!

PR #480 is now available in 2.1.3.

Guys, somebody already tried v2.1.3 in their projects? Does it solve the issue?

Upgraded last weekend. Service was idle (meaning no pushes were sent) for ~24h before a push was successfully delivered on Monday. Previously, pushes were not delivered once the service was idle for more than a few hours.
So for me it was definitively an improvement.

@ffried Sounds good, will try soon to move to v2 too

@airs0urce updated few days ago to 2.1.3, today tested on an instance which was idle ~24 hours and it worked

Ok, now I can confirm it works for me too

I'm still seeing this:
[VError: apn write failed: read ETIMEDOUT]
APNS   jse_shortmsg: 'apn write failed',
APNS   jse_cause: { [Error: read ETIMEDOUT] code: 'ETIMEDOUT', errno: 'ETIMEDOUT', syscall: 'read' },
cert based auth apn 2.1.3 :(

I can confirm, this error still occurs, although is rare, but it still happens. :(

@jeacott1  @mihai-iorga I am closing this issue. If you are still encountering a similar issue, please open a new issue and provide as much information es possible.

Hello, friends!
I'm the maintainer of Pushy, an APNs provider library for Java. We've also seen this problem (please see https://github.com/relayrides/pushy/issues/408). As part of the debugging effort, I opened a bug report with Apple back in January. They've just responded wondering if this is still an issue and seeking more information.
If any of you are still getting a cascade of HTTP/500 responses, could you please email me directly at<EMAIL_ADDRESS>Thanks kindly!


Delete index.mdx

Preview is available here:http://developers-pr58.previews.kube001.services.stellar-ops.com/docs


[ADD] Check format of parameter groups in EighComputation

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

21 of 21 (100.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.1%) to 81.585%

Totals

Change from base Build<PHONE_NUMBER>:
0.1%

Covered Lines:
1143

Relevant Lines:
1401

💛  - Coveralls


[Feature] Simple untrusted software arithmetic syscalls: SysUntrustedModInv, SysUntrustedModSqrt, SysUntrustedModPow

Feature
I propose to add three software syscalls (i.e., https://github.com/risc0/risc0/blob/main/risc0/zkvm/src/host/server/exec/syscall.rs#L107) that are useful for modular operations and do not require any change in the circuit.

SysUntrustedModInv: obtain an untrusted modular inverse of REG_A3 under the modulus REG_A4, write the result back
SysUntrustedModSqrt: if REG_A3 has a square root, return an untrusted square root of REG_A3 under the modulus of REG_A4, and if REG_A3 doesn't have a square root, return an untrusted square root of REG_A3 * REG_A5, where REG_A5 is a user-provided quadratic nonresidue (with an additional output on whether the square root exists). Note that REG_A5 is needed even if REG_A3 is a quadratic residue because of the Tonelli–Shanks algorithm.
SysUntrustedModPow: compute pow(REG_A3, REG_A4, REG_A5), which is for hash functions that leverages x^{1/a} where computation is difficult but verification is easy.

The insight is that:

they can be easily verified through the existing bigint syscall
they are generic and leverage nondet
they are common in a large number of applications

We likely do not need more than these three because there are not a lot of nondet that we know in "fields". What is missing from this list includes "cubic root", but I found it less interesting.
Motivation
This simplifies the nondet and provides functionality necessary to accelerate many EC-related and finite-field-related libraries.
Implementation
The implementation can be simple: adding three syscalls to the codebase, using some general-purpose implementation to realize them.
We need to place a requirement that SysUntrustedSqrt only works for a prime, as the non-prime case is very tricky.

Related to this: https://github.com/l2iterative/ark-bn254-r0/blob/main/src/fields/fr.rs
which aims to provide a drop-in replacement of ark-bn254 that uses RISC Zero

This has been implemented in the following fork of RISC Zero:
https://github.com/l2iterative/risc0-bigint/commit/4a0558cdd8fdc55162d8c3165bd04c5a64e6fe96


(PUP-6099) add additional autorequires for file/mount interactions

With this improvement, file resources will execute before their mount point is mounted and file resources will execute after their parent mount is mounted.
I need some help writing the spec tests because I am unable to fully parse how the Puppet spec API is working.  Please assist.
Also I probably could use some code criticism.  The coding style between the mount and file types and their spec tests were rather different and I tried to stick to the style within each.

I did a good bit more work on the spec tests.  I noticed there are no spec tests on the user and group autorequires for the file resource, so I understand I am probably foraging into unknown territory here by trying to implement cross-resource autorequire spec tests here.  If I succeed, then you guys could potentially use this pattern for the other unimplemented spec tests.
Also this PR would prompt an update to the file and mount resource type documentation.
Still would appreciate some code criticism and test assistance though.

Thanks for this contribution! I can't think of any reasons at the moment why this would cause problems (/cc @branan). I'll take a look into helping with specs a bit later.

Thank so much for the feedback.  I will take a look at these comments and update accordingly (I have been resetting and force pushing to keep everything in one commit).
I notice a lot of random unrelated spec tests have been failing for this changeset.  I am wondering if I have done something really bad or if I had an unlucky fork.

Github is currently ignoring all of my pushes to the Puppet fork, so I reforked it and Github is still ignoring my pushes for some reason.
While I run down Github's technical difficulties, this is the diff of the newest changes:
diff --git a/lib/puppet/type/mount.rb b/lib/puppet/type/mount.rb
index 9e798a1..3fa7427 100644
--- a/lib/puppet/type/mount.rb
+++ b/lib/puppet/type/mount.rb
@@ -14,7 +14,11 @@ module Puppet

       **Autorequires:** If Puppet is managing any parents of a mount resource ---
       that is, other mount points higher up in the filesystem --- the child
-      mount will autorequire them."
+      mount will autorequire them. If Puppet is managing the file path of a
+      mount point, the mount resource will autorequire it.
+
+      **Autobefores:**  If Puppet is managing any child file paths of a mount
+      point, the mount resource will autobefore them."

     feature :refreshable, "The provider can remount the filesystem.",
       :methods => [:remount]
@@ -293,5 +297,16 @@ module Puppet
       dependencies[0..-2]
     end

+    # Autorequire the mount point's file resource
+    autorequire(:file) { Pathname.new(@parameters[:name].value) }
+
+    # Autobefore the mount point's child file paths
+    autobefore(:file) do
+      dependencies = []
+      Pathname.new(@parameters[:path].value).descend do |child|
+        dependencies.push child.to_s
+      end
+      dependencies[0..-2]
+    end
   end
 end
diff --git a/spec/unit/type/mount_spec.rb b/spec/unit/type/mount_spec.rb
index 4cd94a4..0976e13 100644
--- a/spec/unit/type/mount_spec.rb
+++ b/spec/unit/type/mount_spec.rb
@@ -545,15 +545,22 @@ describe Puppet::Type.type(:mount), :unless => Puppet.features.microsoft_windows
     end
   end

-  describe "establishing autorequires" do
+  describe "establishing autorequires and autobefores" do

-    def create_resource(path)
+    def create_mount_resource(path)
       described_class.new(
         :name => path,
         :provider => providerclass.new(path)
       )
     end

+    def create_file_resource(path)
+      Puppet::Type.type(:file).stubs(:defaultprovider).new(
+        :path => path,
+        :provider => Puppet::Type.type(:file).stubs(:defaultprovider).new(:path => path).provider
+      )
+    end
+
     def create_catalog(*resources)
       catalog = Puppet::Resource::Catalog.new
       resources.each do |resource|
@@ -563,38 +570,71 @@ describe Puppet::Type.type(:mount), :unless => Puppet.features.microsoft_windows
       catalog
     end

-    let(:root_mount) { create_resource("/") }
-    let(:var_mount)  { create_resource("/var") }
-    let(:log_mount)  { create_resource("/var/log") }
+    let(:root_mount) { create_mount_resource("/") }
+    let(:var_mount)  { create_mount_resource("/var") }
+    let(:log_mount)  { create_mount_resource("/var/log") }
+    let(:var_file) { create_file_resource('/var') }
+    let(:log_file) { create_file_resource('/var/log') }
+    let(:puppet_file) { create_file_resource('/var/log/puppet') }

     before do
-      create_catalog(root_mount, var_mount, log_mount)
+      create_catalog(root_mount, var_mount, log_mount, var_file, log_file)
     end

     it "adds no autorequires for the root mount" do
       expect(root_mount.autorequire).to be_empty
     end

-    it "adds the parent autorequire for a mount with one parent" do
+    it "adds the parent autorequire and the file autorequire for a mount with one parent" do
       parent_relationship = var_mount.autorequire[0]
+      file_relationship = var_mount.autorequire[1]

-      expect(var_mount.autorequire).to have_exactly(1).item
+      expect(var_mount.autorequire).to have_exactly(2).item

       expect(parent_relationship.source).to eq root_mount
       expect(parent_relationship.target).to eq var_mount
+
+      expect(file_relationship.source).to eq var_file
+      expect(file_relationship.target).to eq var_mount
     end

-    it "adds both parent autorequires for a mount with two parents" do
+    it "adds both parent autorequires and the file autorequire for a mount with two parents" do
       grandparent_relationship = log_mount.autorequire[0]
       parent_relationship = log_mount.autorequire[1]
+      file_relationship = log_mount.autorequire[2]

-      expect(log_mount.autorequire).to have_exactly(2).items
+      expect(log_mount.autorequire).to have_exactly(3).items

       expect(grandparent_relationship.source).to eq root_mount
       expect(grandparent_relationship.target).to eq log_mount

       expect(parent_relationship.source).to eq var_mount
       expect(parent_relationship.target).to eq log_mount
+
+      expect(file_relationship.source).to eq log_file
+      expect(file_relationship.target).to eq log_mount
+    end
+
+    it "adds the child autobefore for a mount with one file child" do
+      child_relationship = log_mount.autobefore[0]
+
+      expect(log_mount.autobefore).to have_exactly(1).item
+
+      expect(child_relationship.source).to eq log_mount
+      expect(child_relationship.target).to eq puppet_file
+    end
+
+    it "adds both child autbefores for a mount with two file childs" do
+      child_relationship = var_mount.autobefore[0]
+      grandchild_relationship = var_mount.autobefore[1]
+
+      expect(var_mount.autobefore).to have_exactly(2).items
+
+      expect(child_relationship.source).to eq var_mount
+      expect(child_relationship.target).to eq log_file
+
+      expect(grandchild_relationship.source).to eq var_mount
+      expect(grandchild_relationship.target).to eq puppet_file
     end
   end
 end

I am unsure how to get Puppet to traverse unknown child paths of a mount point, unless it is working from the file resources themselves (which would be back to the autorequire). I am sure the mimicked code I put in will not actually work correctly.
Changed up the spec test for everything being in mount, but still expecting the file resources to not be generated correctly.

Github broke my fork (I think they were having some technical/networking issues in the last hour) so I have to open a new PR.


 This leaves consuming code in an incorrect state where it still thinks the button is pressed when it's not.
I think we should either call onReleased or add an onCancel callback to ButtonComponent

Sounds like onCancel should be called when that happens.

I don't believe that member exists, I wasn't able to find it.

Yeah, I mean that it should be added. :)

Fixed by #2193


[Scenario] Treaty of Champions

Greetings,
I have adapted a new Descent 2E scenario:
TC2 - Treaty of Champions
Could you please add it to the mission listing?
Thanks
What kind of mission do you want to add (mark with an X, e.g. [X] Descent)?

[ ] Mansions of Madness
[X] Descent

What expansions does the scenario require?
None
What expansions can the scenario optionally use?
All
Please enter the GitHub Url to your scenario here
https://github.com/afellowspelunker/TreatyofChampions

Quest added to the manifest. It should be available within an hour. Thanks a lot for your work, @afellowspelunker

Looks good, thanks!


Failed to execute goal org.scala-tools:maven-scala-plugin:2.15.2:compile (default) on project gatling-core

Hi !
One of my teachers told me about Gatling I'm trying to use it.
But when I'm doing "mvn clean install", the first time it gives me an exception (see this issue title).
Then i'm running it a second time so this time there's no exception, but when I'm using Gatling with the default scenario (number 1 or number 2 ) , I'm getting an exception : "NoClassDefFoundError: com/excilys/ebi/gatling/charts/component/impl/ComponentLibraryImpl"
So is this a real issue or am I doing something wrong ?

Hi,
I don't know what happened on your first run, there should have been a
cause indicated somewhere in your logs. I'll try to reproduce it with an
empty repository, but without your whole logs, I'm practically bling.
Regarding your second issue, please read the first question and my answer
on the following thread :
https://groups.google.com/group/gatling/browse_thread/thread/1a81b947481b65a
Just one question? Are you sure that you want to manually build the master
sources?
You can find ready to use binaries in the download section :
https://github.com/excilys/gatling/downloads
Beware that the links on top ("Download as zip" and "Download as tar") are
standard links from Github for downloading the sources.
Those are NOT the link for downloading the binaries.
Those links are quite confusing and I asked Github if it was possible to
rename them.
Sincerely,
Steph
2012/2/3 taharBakir <
<EMAIL_ADDRESS>

Hi !
One of my teachers told me about Gatling I'm trying to use it.
But when I'm doing "mvn clean install", the first time it gives me an
exception (see this issue title).
Then i'm running it a second time so this time there's no exception, but
when I'm using Gatling with the default scenario (number 1 or number 2 ) ,
I'm getting an exception : "NoClassDefFoundError:
com/excilys/ebi/gatling/charts/component/impl/ComponentLibraryImpl"
So is this a real issue or am I doing something wrong ?

Reply to this email directly or view it on GitHub:
https://github.com/excilys/gatling/issues/394

Ok thanks after reading the https://groups.google.com/group/gatling/browse_thread/thread/1a81b947481b65a now it makes sense.
I used the link you gave me for the binaries, works fine.
Peace

For the record, I tried to compile the sources with an empty maven local repository and everything ran fine.


Support for virtual fields in AR

Has the Yii team ever considered implementing support for virtual fields in AR?
CakePHP has it here: http://book.cakephp.org/2.0/en/models/virtual-fields.html
We have implemented it for Yii2 here: https://github.com/MarketFlow/yii2-virtual-fields
(Documentation is there, the actual files will follow in a few mins / hours).
Advantage built-in of virtual field support over implementing stuff like "getPostCount()" in each model is that virtual fields can be greedily loaded, greatly reducing the number of queries required.

How this is different from:
http://www.yiiframework.com/doc-2.0/guide-db-active-record.html#data-transformation

@klimov-paul  maybe he's talking about a field defined by SQLs not by php code... something similar to statistical queries in Yii1... although in Yii2 i just do this by adding a custom property + Query::select()

As far as I understand @SamMousa propose creation of model virtual property, which may be populated either by eager loading in case they are mentioned at SELECT or using lazy loading, when record has been fetched without it.
It seems this could be done in following way:
class Customer extends ActiveRecord
{
    private $_orderCount;

    public function setOrderCount($value)
    {
        $this->_orderCount = $value;
    }

    public function getOrderCount()
    {
        if ($this->_orderCount === null) {
            $this->_orderCount = Order::find()->where(['user_id' => $this->id])->count();
        }
        return $this->_orderCount;
    }
}

@klimov-paul, you are right about my goal, however I do not see how your example uses greedy loading.
The problem with lazy loading is that if I want to get a list of Authors with their postCount, it will do n+1 queries. My proposal reduces this to 1 query.

None prevents you to write eager loading for virtual property:
$customers = Customer::find()->select(['*', 'COUNT(orders.id) AS orderCount'])->joinWith('orders')->all();
foreach ($customers as $customer) {
    echo $customers->orderCount; // no extra query
}

$customer = Customer::find()->one();
echo $customer->orderCount; // + 1 query.

Yes I'm doing stuff that is "impossible" however, you must agree your example, is not DRY and will not support lazy loading.
The point of this proposal is that you can implement both if you want and use them everywhere like $customer->orderCount.
Also I think your example might actually not work since populateRecord will not set the orderCount property unless I add it to the model as well. So that means I have to edit the code in a lot of places.

your example, is not DRY and will not support lazy loading.

Why? getOrderCount() will do lazy loading in case $_orderCount is not set ealier.

The point of this proposal is that you can implement both if you want and use them everywhere like

You can. If you don't like writing select() and joinWith() at several places you can compose this code into a ActiveQuery scope and then use it if needed.

your example might actually not work since populateRecord will not set the orderCount property unless I add it to the model as well.

It should set it, as far as I see:
public static function populateRecord($record, $row)
    {
        $columns = array_flip($record->attributes());
        foreach ($row as $name => $value) {
            if (isset($columns[$name])) {
                $record->_attributes[$name] = $value;
            } elseif ($record->canSetProperty($name)) { // we have setter, so we can set `orderCount`
                $record->$name = $value; // here `orderCount` will be set.
            }
        }
        $record->_oldAttributes = $record->_attributes;
    }

Ah, your 2nd example is added to the first. =)
So then I have to create a scope for each ActiveRecord to add custom fields, forcing me to define the same "relation"  in 2 places, lazily in the model, greedily in the scope.
I do not see how that is a better approach than an approach which allows you to simply define it in one place where it will always work.

@klimov-paul I tend to agree to @SamMousa. It could be very convenient to list SQL expressions for those virtual attributes. Using scopes is pretty cumbersome (and not sure if it works easily if you have more than one of those attributes anyway).
So having something like
public function virtuals()
{
    return [
        'cssClass' => 'IF(x.value IS NULL, '',x.value)',
        'priceCut' => 'IF(x.old_price>0, FLOOR(100*(1-x.price/x.old_price)), 0)',
    ];
}

in ActiveRecord could be very convenient. This would move some calculations to the DB (making these fields orderable) and avoid that you have to write too many getters / scopes.

Btw, here is the syntax i'm proposing:
public static function virtualFields() {
  return [
    'postCount' => [
      'lazy' => function($model) { return $this->getPosts()->count; }
      'greedy' => Post::model()->limit(1)->select('count(*)')->condition('author_id = author.id')
    ]
  ];
}

I think it's better if it's static since during greedy loading you don't have a model instance, (it will also be less error prone).
Lazy loading can be ommitted and it will throw an exception if you use it lazily.

forcing me to define the same "relation" in 4 places

??? - I can't see any relation definition involved here.

query object would have to be defined just to accomodate virtual fields.

I don't understand. If you with to use a scopes, which is common practice, you have to define a special Query class for your ActiveRecord. I can see no harm, if you need to do so for the virtual properties as well.

I do not see how that is a better approach than an approach which allows you to simply define it in one place where it will always work.

In return: I do not see why we should add an extra mechanism for class properties definition, while this can be easy done using existing one: setter/getter.
Creating custom setter/getter grands more flexibility: you can decide if you wish or not throw an exception on attempt to access not filled virtual property (no support for lazy load) and so on.

So having something like ... in ActiveRecord could be very convenient.

Yes, it will - but I am afraid this is not that simple. If you use an aggragation to query orderCount, you will need to add a GROUP BY statement, but your virtual property may be composed not only from aggragation and may not require a GROUP BY.  There is no way some 'pretty' syntax be composed, just look at @SamMousa implementation: it is a getter composed into anunymous function and scope definition.

There are quotes over "relation" for a reason. I'm not talking about an AR relation but about the fact that the number of posts for an author is related to both Author and Post.
The whole point of these virtual fields is NOT to use joins, removing the need for group by.
Adding joins and group by can definitely break stuff.
Suppose we extend the example to say that Authors not only have Posts, but they also have Followers.
Now I need one overview to get all Authors with the number of posts and the number of followers. Clearly this cannot (easily) be done using joins and group by.
Instead using subqueries in the SELECT is intuitive and in most cost results in an extremely efficient execution plan.
I'm not sure on your last argument, definitely a syntax that resembles stuff you guys use in Yii for things like:

relations,
validation rules,
attribute labels
etc..

Is better than requiring changes in 4 places to implement 1 virtual field. I'll reiterate the places:

Scope class
Private / protected model propert $_postCount;
Getter in model,
Setter in model.

And yes, my implementation for lazy loading is a getter composed into an anonymous function. And yes, the greedy loading can be defined in a scope. But isn't the whole purpose of the AR abstraction to make things DRY?

virtual attributes it group scopes for query?  - create scope.
problem with static relation (eager) it not solve

Any of the other team members have some feedback on this?

A way to eagerly load various "scalar" data such as what's generated by statistical queries would be very welcome.
It would be ideal though if we could load them in with() like regular relations, for example:
$baskets = Basket::find()->with(['customer', 'customer.orderCount']);
foreach ($baskets as $basket) {
    echo $basket->customer->orderCount; // without additional query
}

I think this isn't possible with either of these two proposals?

It actually is, i chose to implement it via the select; but doing it via
with would be just as easy and not change the syntax for defining it.
On May 10, 2015 22:05, "Laszlo"<EMAIL_ADDRESS>wrote:

A way to eagerly load various "scalar" data such as what's generated by
statistical queries would be very welcome.
It would be ideal though if we could load them in with() like regular
relations, for example:
$baskets = Basket::find()->with(['customer', 'customer.orderCount']);foreach ($baskets as $basket) {    echo $basket->customer->orderCount; // without additional query}
I think this isn't possible with either of these two proposals?
—
Reply to this email directly or view it on GitHub
https://github.com/yiisoft/yii2/issues/8316#issuecomment-100690325.

@SamMousa I didn't check out your extension in detail yet, but if it is indeed possible, I would say it's a welcome change: it would make the orderCount "property" available from any query, instead of only when directly querying its parent class.

Well, it is not a normal relation and since this is about adding subqueries in the select clause I'm not sure it will work if you query chained relations. I explicitly don't want to reintroduce stat relations; I'm assuming the yii team decided to omit it for q reason

Thinking some more about it, @laszlovl.
It is definitely useful to be able to specify virtual fields even when the virtual field is not in the primary model. I'm unsure if this should be combined with relations though.
Instead we could add syntax for it; that way people don't get confused AND we have no backward compatiblity issues.
Currently my syntax using select would allow for this:
    Post::find()->with(['Author' => function($q) {
        $q->addSelect('postCount');
    });

Obviously this is a bit verbose and we could add some syntactic sugar.
    Author::find()->addVirtualField('postCount');
    Post::find()->with('author')->withField('author.postCount');

This clearly specifies that we are getting a virtual field; it is also easy to parse for active query (since it needs to add the virtual field definition to the select clause).
Regarding syntax I'm just making suggestions; I think this functionality should be added to the core even if the dev team decides on different syntax or integration with existing syntax (select or with for example).
Currently the only argument against is that "it can be done without", however that argument is obviously void, by definition a framework does not do anything PHP cannot do by itself, the goal is to add structure and avoid repetition which this certainly does!

I've been playing with eagerly loading a virtual count attribute in the primary query like in the examples listed above. That didn't work out very well though, as the GROUP BY necessary for the count calculation would mess up the results in the primary query. Perhaps it would be safer and more consistent if the virtual attribute were loaded in a separate query, just like relations are.
That led me to this proof of concept:
class User {
    public $count;

    public function getPostCount()
    {
        return $this->hasOne(self::class, ['id' => 'id'])
            ->select('User.id, COUNT(Post.id) as count')->joinWith('posts', false)->groupBy('User.id');
    }

    public function getPosts()
    {
        return $this->hasMany(Post::class, ['user_id' => 'id']);
    }
}

It's not beautiful, but it is fast and DRY: the same definition can be used for both lazy and eager loading:
$baskets = Basket::find()->with(['user.postCount'])->all();
// SELECT * FROM `Basket`;
// SELECT * FROM `User` WHERE `User`.`id` IN (1, 2);
// SELECT `User`.`id`, COUNT(Post.id) as count FROM `User` LEFT JOIN `Post` ON `User`.`id` = `Post`.`user_id` WHERE `User`.`id` IN (1, 2) GROUP BY `User`.`id`

foreach ($baskets as $basket) {
    echo $basket->user->postCount->count;
}

$user = User::findOne(1);
echo $user->postCount->count;
// SELECT `User`.`id`, COUNT(Post.id) as count FROM `User` LEFT JOIN `Post` ON `User`.`id` = `Post`.`user_id` WHERE `User`.`id`=1 GROUP BY `User`.`id`

Perhaps if ActiveQuery would decouple the class used to load relations from the class that's instanciated, we could clean it up a bit. Instead of $user->postCount returning a User instance, one could then make it return a VirtualPropertyContainer or something, and even override __get to directly return that container's value instead of the container itself.

@laszlovl If you had read my implementation / documentation linked in the first post you'd see that i'm not doing any joins on purpose. So don't worry about that.
The whole idea behind virtual fields is to add subqueries in the SELECT clause. So no joins no group by, nothing. This will always work and assuming you have the correct indexes it will be performant as well.
Especially when using pagination, since then the subquery will only be run for every record in the result set, not for every record in the join set.

@SamMousa Right, I should have clarified I ment @klimov-paul's examples. I checked your approach and the concept works well.

@SamMousa the idea sounds interesting. Need to dig deeper into the details to say more but I would keep this open as I think it could be a useful feature.

@cebe I'm available on IRC (sammousa) in case you want a more in-depth / live discussion.
Also I'd be up for creating an implementation and pull request.
We mainly need to think of a syntax for retrieval if we want to support this through relations as well. (Parsing the select is error prone and not very user friendly.)
A suggestion that was made to use this the with() function; alternatively we could add a new function which is probably the cleanest approach:
   Posts::find()->with('Author')->virtualFields(['likeCount',['Author.postCount']);

Cheers,

Hmm, was thinking on syntax:
    Posts::find()->with(['Author' => function($query) { $query->withFields('postCount'); }])->withFields(['likeCount', 'followerCount']);

Basically the same as the with syntax; easy to parse and no need to parse complex structures. Each withFields call applies directly and only to the query it is being called upon.
@cebe If you are happy with that syntax I can rework the implementation and create a PR.

@SamMousa this is how I do it in my models http://pastie.org/10270677
The findWithCantidad() method is what you are missing and that you can reuse anytime you want.
Alternatively you can make an ActiveQuery class to do this automatically.
Grouping fields like count(distinct id) are not doable just adding values to the select (which btw you can do using the ActiveRecord::addSelect()) you need joins and group by too

@faryshta there is no need for joins. Joins will actually give you a lot if head aches and functionality is limited.
Your implementation will not work for multiple virtual fields for example (since you can't have multiple independent group by)

@SamMousa then how do you get counts and sums without grouping?
how do you get stuff like Product::category_name without joins?

subqueries in the select clause, if you scroll up you can find the implementation.

@SamMousa honestly i think thats complicating the ActiveRecord to a point where only a few people will use it and be able to debug that code.
Anyway, how do you set those subqueries without remaking the relations? In my example it is possible to add conditions to the counters (for example only likes from people in my country) and i don't see how do it in yours.

Relations and virtual fields are different functionalities.
They solve a different problem.
Anyway, read the CakePHP docs linked in my OP, they definitely have a use.
They do not complicate AR at all and any functions with joins are by definition more complicated. The Yii team even chose not to use joins by default for relations.
Try getting the count of 2 different relations using 1 query with joins, that's not gonna work. Using virtual fields it always works.

I think the main question is: how do you set those subqueries without remaking the relations?

Faryshta, that is not a strict requirement. Currently there is no efficient solution to get this to work at all.
My solution might require a little code duplication, but it is restricted to the model only and even there it's not much that is getting duplicated.
Also note that virtual fields are not the same as relations, they have more use cases than just aggregating relations. For example consider a table containing "squares" on a 2d plane. Typical properties would be:
x1, y1, x2, y2
Virtual fields that might come in handy here are width, height and area. The advantage of defining these is that you can:

Query on these fields
Use them as sortable and filterable attributes in grid views and other components.
Cache the results easily by utilizing database caching.

Query on these fields

But you can do this also with the current form of doing it.

Use them as sortable and filterable attributes in grid views and other components.

You can do that too with the current way of doing it.

Cache the results easily by utilizing database caching.

You can do that too with the current way of doing it.

Faryshta, just read up and read the implementation...
The discussion about usability has already been had and there is use even if you do not see it.

I've implemented the following syntax, which is easy to implement and supports nesting properly:
    $query = Portfolio::find()
        ->limit(1)
        ->with(['projects' => function($query) {
            $query->withField('cityName');
        }])

The ActiveQuery class will recalculate the select clause each time a field is added through addSelect(), select() or withField().
Currently ActiveQuery uses a public property $select, which is not ideal since the implementation of virtual fields requires me to keep it in sync all the time.
Ideally it would be better to make $select private / protected and instead provide a getSelect() to get the array. Note that the property is only written to in Yii core, by the query class itself (so protected makes more sense than public in any case).
So the changes required to Yii core, would be:

Make Query::$select protected / private and rename to $_select.
Add getter getSelect().
Add function withField() to ActiveQuery or even Query (since this approach works for regular models as well not just active record).
Add static Model::virtualFields() function with default implemention (return [];)
Update attributes() function to include array_keys(self::virtualFields());
Update __get, __set to include virtualFields.

Bump?

I am interested in this proposal.
Or is there already another easier approach to add subqueries into the primary SELECT query?

@solicomo this is what we created / use: https://github.com/MarketFlow/yii2-virtual-fields
We use it as is (I think) and it hasn't needed updating in a long time.
Feel free to create issues / PRs there if you like it but require some changes!

@SamMousa Okay, then.
I think this is the best way to use this feature for now since the official team doesn't want to accept it.
Sometimes, we need to hack by ourselves.
Thank you. ^_^


Add support for giving salaries and distributing profits and losses

As from PR #4 this feature was automatically added. Therefore closing this issue


Fix wrong migration paths in example

What changed?
Just fixing wrong migration paths in the command example.

Why?

How did you test it?

Potential risks

Release notes

Documentation Changes

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: Shaddoll:x: kotcrabYou have signed the CLA already but the status is still pending? Let us recheck it.


Update service-fabric-linux-windows-differences.md

Added Service Fabric Events Channel difference as mentioned in article https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-diagnostics-overview#platform-cluster-monitoring

@tomvcassidy
Can you review the proposed changes? Note that the validation warning must be resolved before merging.
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


Inconsistency with struct type check

Name
Value

Dart source
https://pastebin.com/exVF9ka7

Dart version
Dart SDK version: 2.16.1 (stable) (Tue Feb 8 12:02:33 2022 +0100) on "windows_x64"

hetu_script version
0.3.12

hetu_script_dev_tools version
0.0.9

Hetu source:

type StructType = {
  a: int,
  b: str,
}

struct Named {
  var a: int;
  var b: str;
  construct(a, b) {
    this.a = a;
    this.b = b;
  }
}

var literalStruct = {a: 1, b: "2"};
var namedStruct = Named(1, "2");

print("Struct type: ${StructType}");
print("\nLiteral struct: ${literalStruct}\nDuck typing result: ${literalStruct is StructType}");
print("\nNamed struct: ${namedStruct}\nDuck typing result: ${namedStruct is StructType}");
print("\nStruct types equal: ${typeof literalStruct == typeof namedStruct}");

Docs say

Structural type are a kind of duck typing system

So I expected struct type to behave like duck typing in haxe or other languages, but results are weird:

It turns out, struct type can only match literal struct, but can't match named struct of the same structure, despite the fact, that both of those structs have types, that are considered equal when comparing via ==, so this looks pretty mych like a bug
Additionally, it can't match structs that have additional elements, which means that it's closer to structural typing than duck typing
type StructType = {
  a: int,
  b: str,
}
var literalStruct = {a: 1, b: "2", c: 3};
literalStruct is StructType;

I don't know what the design choice is about this one, so I want to ask, is it a bug or intended behavior?

After some more testing, I realised that first issue is directly caused by the second one.
struct Named {
  var a: int;
  var b: str;
  construct(a, b) {
    this.a = a;
    this.b = b;
  }
}

var a = Named(1, "2");
var aType = typeof a;

var b = {a:1, b:"2"};
var bType = typeof b;

print(aType == bType);
print(bType);
print(aType);

So apparently, while types are considered equal, named struct type have an additional field $construct, and it causes "duck typing" to fail


Top-level this is the same as global, not exports

The CommonJS plugin is incorrectly treating this.x = 1; as equivalent to exports.x = 1;, which isn't true. We should consider changing top-level this to global.

You seems not right there:
$ echo 'console.log(exports === this)' >> test.js
$ node test.js
true

Your are correct. Thanks!


add isCustomElement flag

@justinfagnani Was a little bit unsure whether this should be on CustomElement, or as a required property on ClassLike, what do you think?

@thepassle I think it should probably be non-optional on CustomElement and set to true. That should let it function as a type discriminator:
interface CustomElement extends ClassLike {
  customElement: true;
}

As far as naming, we haven't use the isX format yet - we have static and optional, not isStatic or isOptional - so I'd go with just customElement.

@justinfagnani PTAL

@justinfagnani friendly ping in case you missed this 🙂  This is the last thing we said we'd get in before V1 so it'd be nice to get this merged

I'd kind of hate to delay things even further, but thinking more about this, maybe we should use the already existing kind field for this instead?
So instead of "kind": "class", it would be "kind": "custom-element". Considering we already have "kind": "mixin" this might make more sense than adding a dedicated boolean field (e.g. customElement) for it.
Example:
        {
+        "kind": "custom-element",
          "description": "",
          "name": "MyEl",
          "members": [],
          "attributes": [],
          "superclass": {
            "name": "HTMLElement"
          },
          "tagName": "my-el"
        }

@thepassle with custom element declarations being a subclass of of class declarations, I think it'd be nice if code could list all the classes by filtering for kind === 'class'. Fundamentally a custom element definition is still a class, with additional fields.


Convert AmphoraControllers to DaemonSet

Looks like kuttl failed with "resource Pod:octavia-kuttl-tests/: .metadata.annotations.openshift.io/scc: value mismatch, expected: anyuid != actual: restricted-v2"

/lgtm

/retest-required

/test octavia-operator-build-deploy-kuttl

Needs added to the amphoracontroller_controller I think
//+kubebuilder:rbac:groups=apps,resources=daemonsets,verbs=create;delete;get;list;patch;update;watch


Ability to add title

Is there way to add title to each action?

My mistake. It's already implemented


[HacktoberFest] Chispend-Airtime-Payout-With-Flutter

Issue #7
Created a mobile (Flutter) Application that leverages the Pay with Wallet functionality on the ChiSpend widget to make an airtime payout.

LGTM


Bluetooth: shell: Add controller DTM commands

Add controller Direct Test Mode commands to bt shell module.
Signed-off-by: Vinayak Kariappa Chettimada<EMAIL_ADDRESS>

So I tried this from my application as I don't have a shell yet due to a USB issue.
It compiled but not working.  What state should the controller be in (bt_enable()) before this call did not work.
int hci_le_set_tx_dtm_test(u8_t chan /* 0->39 /, u8_t len / 0->37 /, uint8_t payload / 0->2 */)
{
struct bt_hci_cp_le_tx_test *cp;
struct net_buf *buf;

buf = bt_hci_cmd_create(BT_HCI_OP_LE_TX_TEST, sizeof(*cp));
if (!buf) {
        return -ENOBUFS;
}

cp = net_buf_add(buf, sizeof(*cp));
cp->tx_ch = chan;
cp->test_data_len = len;
cp->pkt_payload = payload;
int r_val = bt_hci_cmd_send(BT_HCI_OP_LE_TX_TEST, buf);

return r_val;

}


Allow a course to open/close their queues

as title says.
here are some pictures:

fixes #62

There's two high-level design bits that I notice from the screenshots:

There are two different grays (one in the "there aren't any open queues" and a different one for the closed queue).  It feels like once should be chosen and used for both cases.
There seems to be a different spacing under "Open queues" as "Closed queues".

Otherwise, this looks amazing!!  I'm going to leave this unreviewed for @nwalters512 to dive into the code.


Correction for raw affiliation AMAP, INRA, IRD, CIRAD, CNRS, and University of Montpellier, Bd de la Lironde Montpellier France; Now at: Max Planck Institute for Biogeochemistry Jena Germany

Correction needed for raw affiliation AMAP, INRA, IRD, CIRAD, CNRS, and University of Montpellier, Bd de la Lironde Montpellier France; Now at: Max Planck Institute for Biogeochemistry Jena Germany
raw_affiliation_name: AMAP, INRA, IRD, CIRAD, CNRS, and University of Montpellier, Bd de la Lironde Montpellier France; Now at: Max Planck Institute for Biogeochemistry Jena Germany
new_rors: 051escj72;051yxp643;020nks034
previous_rors: 051escj72;051yxp643
works_examples: W2970137199
contact: b63cff40b6d00b25b0cf9db84d8af30e:c306fa402e3e01ae9225cc7d3f57a75e2117 @ inrae.fr

This issue was accepted and ingested by the OpenAlex team on 2024-11-18. The new affiliations should be visible within the next 7 days.


V-REP player is not supported

https://github.com/EricssonResearch/scott-eu/blob/simulation-ros/simulation-ros/src/vrep_ros_interface/cmake/modules/FindVREP.cmake#L19 requires the programming/ folder to exist, which is missing on the V-REP installations.

This is odd. I downloaded the linux version and it has the programming folder. If you're using the V-REP Player instead, don't do this. Stick to the V-REP PRO EDU.

I included instruction in the readme to use the PRO or PRO edu versions. We didn't try to use the Player version yet.

I was just thinking to use Player in a VM if it is lighter. And because instructions suggested so, I think. But thanks @albertohata for the update.


vencord: 1.10.1 -> 1.10.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for vencord is: Vencord web extension
meta.homepage for vencord is: https://github.com/Vendicated/Vencord
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.10.2 with grep in /nix/store/m187vxzxjd9irkvri693jyb1h9fv823n-vencord-1.10.2
found 1.10.2 in filename of file in /nix/store/m187vxzxjd9irkvri693jyb1h9fv823n-vencord-1.10.2

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

4 package rebuild(s)

First fifty rebuilds by attrpath

vencord
vencord-web-extension
webcord-vencord

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/m187vxzxjd9irkvri693jyb1h9fv823n-vencord-1.10.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A vencord https://github.com/r-ryantm/nixpkgs/archive/7be9dd1366370a0939aad6bb3a7d882ddad86a0f.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/7be9dd1366370a0939aad6bb3a7d882ddad86a0f#vencord

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/m187vxzxjd9irkvri693jyb1h9fv823n-vencord-1.10.2
ls -la /nix/store/m187vxzxjd9irkvri693jyb1h9fv823n-vencord-1.10.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

3 packages built:

vencord
vencord-web-extension
webcord-vencord

Maintainer pings
cc @FlafyDev @NotAShelf @Scrumplex for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.

@NixOS/nixpkgs-merge-bot merge


PD: Drop Tips at End of Protocol

As a protocol designer, I would like all of my protocols to automatically include a 'drop tips' command at the end of my protocol, so that I don't accidentally finish a protocol with dirty tips still on the pipette.
Acceptance Criteria

Generated protocol file includes a 'drop tips' command at the end for each pipette if that pipette has tips.

Implementation Notes

Is there a possible conflict with user-generated drop-tips command
Assuming the user doesn't need to delete this step, is there any reason to show it in the step list (can we just do this invisibly)

Design
None needed

This seems like it's not working anymore


We need a gentle introduction to MASS

Original report by Joseph Fowler (Bitbucket: joe_fowler, ).

I've long understood that we need a gentle getting-started guide to MASS. Some elements this would include are:

A correct list of installation dependencies (e.g., Cython, scikit-learn, etc etc).
Installation instructions.
One or more examples of simple, everyday usage steps, from data summarizing to a fitted Mn Ka spectrum.

@oneilg says that he has a public GitHub repository containing a few example "reference" microcal files. I'm assuming it's this one: ReferenceMicrocalFiles.jl (Don't worry about the fact that it's nominally a Julia package.) These could be used as a common ground for the tutorials.

Original comment by Joseph Fowler (Bitbucket: joe_fowler, ).

Completed in aed2a8d

Original comment by Joseph Fowler (Bitbucket: joe_fowler, ).

This will include running and--as needed--updating the demonstrations in mass/demo.


Missing file for Entity Disambiguation

Hi.
I want to run PURE with GrailQA for entity retrieval, as you mentioned in README.
When I execute $ sh retriever/scripts/entity_retrieval.sh for entity disambiguation, I found that there is no 'retriever/results/disamb/grail_dev/predictions.json'
in https://github.com/microsoft/KC/blob/main/papers/TIARA/src/utils/disamb_to_entity.py#L18 .
Do you mind if you help me to get that file?

Hello. This file is obtained from the entity retrieval part of RnG-KBQA(https://github.com/salesforce/rng-kbqa).
I don't know if you have checked that first. Please reply if you meet any further problems.


Jaeger to support badger

align with https://github.com/istio/istio/pull/16010
Signed-off-by: clyang82<EMAIL_ADDRESS>

I don't know who 'badger' is, but from the PR description looks good..

https://www.jaegertracing.io/docs/1.12/deployment/#badger-local-storage


Rename rdopkg -i to rdopkg status

rdopkg interface tries to mimick git and other already existing tools that users are used to. In this spirit, rdopkg -i should be renamed to rdopkg status as it does very similar thing to git status.

Done in 191b53e2d3ac85c84a0.


[Improvement] Create kyuubi-util and kyuubi-util-scala modules

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Search before asking

[X] I have searched in the issues and found no similar issues.

What would you like to be improved?
Some modules do not depend on kyuubi-common since they are designed to be independent w/ Kyuubi server/engine, this cause some code duplication in whole codebase.
How should we improve?
Create the following  modules

kyuubi-util only depends on JDK and slf4j-api
kyuubi-util-scala only depends on kyuubi-util and scala-library

Are you willing to submit PR?

[ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
[ ] No. I cannot submit a PR at this time.

+1


Better messaging when paket.dependencies cannot be found

I had a hard time figuring out why paket was not updating references for a project. It was because I had put paket.dependencies in the .paket directory.  I am proposing the following message update. The italicized is new.
"Could not find 'paket.dependencies'. To use Paket with this solution, please run 'paket init' first.
If you have already run 'packet.init' then ensure that 'paket.dependencies' is located in the source directory.
Like this:
MySolutionDir
.paket
MySolution.sln
paket.dependencies"
This change would be in PublicAPI.fs static member Locate to the failwithf call.  Let me know if you like the message and I can send the PR.

Yes that sounds good. But please don't talk about solutions here. It should
be on top of the repo, but paket actually doesn't care about sln so that
might be misleading
On Mar 15, 2016 4:58 PM, "Phil Cleveland"<EMAIL_ADDRESS>wrote:

I had a hard time figuring out why paket was not updating references for a
project. It was because I had put paket.dependencies in the .paket
directory. I am proposing the following message update. The italicized is
new.
"Could not find 'paket.dependencies'. To use Paket with this solution,
please run 'paket init' first.
If you have already run 'packet.init' then ensure that
'paket.dependencies' is located in the source directory. Like this:
MySolutionDir .paket MySolution.sln paket.dependencies"
This change would be in PublicAPI.fs static member Locate to the failwithf
call. Let me know if you like the message and I can send the PR.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/fsprojects/Paket/issues/1518

I don't understand "on top of the repo".  Can you link where this discussion should move? Thanks.

We usually assume that paket.dependencies is in the top level folder of
your git repository (which is usually where you find a sln)
On Mar 15, 2016 5:01 PM, "Phil Cleveland"<EMAIL_ADDRESS>wrote:

I don't understand "on top of the repo". Can you link where this
discussion should move? Thanks.
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/fsprojects/Paket/issues/1518#issuecomment-196896517

I see that the sln doesn't really matter. Maybe it would be better to just show paket.dependencies at the same level as .paket.
e.g.
MySourceDir
.paket
paket.dependencies


Rework nt-rs websocket protocol

WPILib may be taking a different path than WS BIN/NT when integrating it into ntcore. There may also be a separate, easier to parse protocol to make it easier to integrate with less built up libraries. Make changes as necessary as the documentation for ntcore websocket comes out

It seems there aren't plans to support NTv3 over WS BIN for websocket servers


Build metadata notes

There are more things to consider as "metadata" but those require a scope analyzer which I cba to write and test :) maybe someday I will.

Meow

The big data

Thanks, this is quite the effort. I appreciate it.
Unfortunately, progress on this project (and all of my projects in general) is stagnating because I'm preoccupied with work, brain static, and other real life minutiae. Once/if I get back to this, though, I'll definitely reference the notes you've added.


chunk

https://lodash.com/docs#chunk

proposal on Go repository:
proposal: slices: add Chunk function to divide []T into [][]T chunks https://github.com/golang/go/issues/53987


IN WSL2 WITH KALI-LINUX TOOL NOT WORKING

Hello There,
Tool is not working after install.
1)for install
pip install git+https://github.com/sanjai-AK47/ShodanX
2)shodanx --help
then i got these errors
[INFO]: Import Error occured in Module imports due to: No module named 'alive_progress'
[INFO]: If you are encountering this issue more than a time please report the issues in ShodanX Github page..
you can check screenshot with this message

Hey @d4715 ,
Please use this command to install the missing package and run the shodanx
without any errors.
pip install alive_progress

Kind regards,
@RevoltSecurities

Hello There,
Thank you so much . now tool works fine .
if any doubt occurs so i will ping you back.
Thank you so much
Thank,
D4715


556 all adopter applications visible to staff

🔗 Issue

#556
✍ Description

Changes the self.org_pets_with_apps method to include pets that have been adopted so they show up on the adoption applications review index page and adds tests for the updated method.
Right now the order of the pets changes after the adoption has been made. In the recording below you can see that Jasmine moved the bottom of the page after the adoption was made, do we want to specify an ordering for the pets?
📷 Screenshots/Demos

https://github.com/rubyforgood/pet-rescue/assets/82609260/279e7a22-c2c4-4762-8380-4cf6867a4642

@kasugaijin I fixed the tests.
As for differentiating adopted and not yet adopted, I can think of at least a couple ways to do it. We could do it in the db query, something like: Pet.org_pets_with_apps(old_id).where.missing(:match) for not yet adopted and where.associated(:match) for  adopted. Or we could stick with the db query for all pets with applications and then add a default filtering to  the results in the controller action if there's no specified application status. Just some preliminary thoughts.

@MooseCowBear thank you! Yes those two options would work. I think it depends on how big the collections are, and we might consider caching also once things start to get large. TBD.


Allow to use existing IAM resources instead of creating new ones

PR o'clock
Description
This PR adds the ability to conditionally use existing IAM roles and instance profiles instead of creating new ones.
The feature is crucial in organizations where you have very strict iam:* access and can't freely create/attach/pass roles.
Checklist

[X] terraform fmt and terraform validate both work from the root and examples/eks_test_fixture directories (look in CI for an example)
[ ] Tests for the changes have been added and passing (for bug fixes/features)
[ ] Test results are pasted in this PR (in lieu of CI)
[X] Docs have been updated using terraform-docs per README.md instructions
[X] I've added my change to CHANGELOG.md
[X] Any breaking changes are highlighted above

I'm not sure how you want to do with the test fixture? Maybe add a second one which assumes existing service roles, instance roles and instance profiles?

Hey @mandrean,
Thanks for the PR!
There's quite a lot of complexity here for something that probably 95% of users won't use. I'm not against merging it, just want to question a few things....
In regards to the workers, put simply, all you want to do is change the iam_instance_profile attribute of the worker LC for 1 or all worker groups. Am I wrong?
Wouldn't this work?
resource "aws_launch_configuration" "workers" {
  iam_instance_profile = "${lookup(var.worker_groups[count.index], "iam_instance_profile", aws_iam_instance_profile.workers.id}"

I know this will still create aws_iam_role and aws_iam_instance_profile but they won't be used.

@sc250024

I would disagree with @max-rocket-internet that it's something "95% of users won't use"

Fair enough, maybe you are right, it might be more than I think 🙂

It seems to me that #137 is excessive since with Kube2IAM / Kiam

I don't think we want to merge changes with the assumption of people using a specific tool unless it's REALLY common. I don't think Kube2IAM / Kiam fits that criteria right now. The cluster-autoscaler would fit that criteria I think.
I still don't quite understand your point though. If you want to use Kube2IAM and assign specific roles to pods then neither of these PRs make any difference to that ability. Unless I'm mistaken?
Anyway, none of this really matters. The author of this PR hasn't replied to my question from 2 weeks ago and @laverya has been very responsive. I also prefer https://github.com/terraform-aws-modules/terraform-aws-eks/pull/137 since it's quite a lot simpler.
If either @sc250024 or @mandrean want to make a new PR then all good 👍

There's quite a lot of complexity here for something that probably 95% of users won't use. I'm not against merging it, just want to question a few things....

Maybe these stats are, maybe not.
I'm sure a majority of enterprise users like banks, insurance companies, large corporations etc. would need these features though. And with EKS being out in Europe, more and more enterprises (just like my current client) will look at moving their existing kube-aws or kops clusters to EKS instead for manageability and cost-saving reasons.
It would be a shame if they couldn't use this great module and would have to resort to CloudFormation instead, just because this module assumes you have full iam:* permissions which is rare in large companies.

@max-rocket-internet To respond to your points:

I still don't quite understand your point though. If you want to use Kube2IAM and assign specific roles to pods then neither of these PRs make any difference to that ability. Unless I'm mistaken?

You assign specific roles to pods, but those pod roles must be assumed by the worker node role that's attached to the worker nodes. As I understand it, when using this Terraform module in its current form, a new worker node role is generated for every cluster. It would be nice to have one worker node role for all potential EKS clusters; that way, someone doesn't have to re-grant any new worker node role privileges to assume all the pod roles required. At my startup, we have around ~15 pod roles for 15 apps, each with their own set of permissions; all 15 of these need to trust the worker node role to allow that role to assume the pod role.
I think Kube2IAM / Kiam are used quite frequently actually. A quick Google search reveals this:

Securing AWS EKS with Calico and Kube2iam
Securing AWS IAM access in Kubernetes
Moving from Kube2Iam to Kiam
Installing kube2iam in AWS Kubernetes Kops Cluster
CloudPosse Kube2IAM Docs

What I would say to people running any Kubernetes cluster in AWS is, if you're not using Kube2IAM / Kiam, you absolutely should; otherwise you're allowing any pod to have credentials to resources that it shouldn't. Google Cloud works in the same way, although they have a direct link between service accounts created in the GCloud IAM area, and the service accounts that exist in Kubernetes. AWS doesn't have this as such, but the closest thing is Kube2IAM or Kiam.
@mandrean I think first to resolve the merge conflicts.

@sc250024

As I understand it, when using this Terraform module in its current form, a new worker node role is generated for every cluster.

By default, yes.

It would be nice to have one worker node role for all potential EKS clusters

But now that 137 is merged, you can just pass the same role ARN for each of your work groups if you want. So that's all sorted then?

if you're not using Kube2IAM / Kiam, you absolutely should; otherwise you're allowing any pod to have credentials and access to resources that it shouldn't.

Only if you are giving access to your applications via the worker node role, which you should not be doing. So currently by default any pod would have this access and no more.

At the end of the day, whichever PR solves the base problem (re-using worker IAM roles) is fine with me.

This is already done as I understand it, you just need to set iam_role_id in your worker group config to whatever you want. Or you can even use the workers_group_defaults variable to add a default that applies for all your worker groups.
@mandrean
I understand all your points. But I think PR137 has solved this, with less complexity in this module. Am I wrong?

It would be a shame if they couldn't use this great module and would have to resort to CloudFormation instead (😱)

BTW, they current CFN templates from AWS don't support this feature either 😉
Always happy to accept more PRs guys ♥

Thanks @max-rocket-internet for the merge on https://github.com/terraform-aws-modules/terraform-aws-eks/pull/137, great points too. Cheers!

I'm gonna close this PR as we merged the other one in the end.
Feel free to create a new PR if you feel that there is still functionality missing though 🙂


Added URLSchemes

This pull requests adds the feature from #58. The way you can launch a window using a url scheme would be by specifying the video id in {video-id} on nativeyoutube://video-id/{video-id}.
An example to try: nativeyoutube://video-id/dQw4w9WgXcQ
:)

dQw4w9WgXcQ

That looks very familiar...

Also awesome it works pretty well! Great work 🙌🏼


Places Types

Hi!
Please, how to search with multiple types as per google documentation

Please, can anyone help me?

Its worked with
queryTypes={'geocode|establishment'}

@zapcriativo sorry, was in moving process and didn't have the net until yesterdays. Sorry for that, happy you did find the answer. Do you want to add this to the readme and do a pr for this ? :) So sorry again


Editor window - close header

Is it possible to close also the header in the workspace?
By Ronald van Loenen

Duplicate of #110


Run "on_stage_end" on all processes and save on only a single process

Alternative proposal to #2053 .
In this, we just run on_stage_end on all processes all the time, and take care of running things on a single process at the most granular level.
There's likely still areas that need to be fixed here, like logging or something like this.
Does this seem better or worse to you @Gastron @Adel-Moumen or any others?

This seems better to me. But in the current implementation any registered save hooks have expected to be called on the main process. I don't know if Checkpointer should wrap saving in run_on_main or the individual objects. Maybe Checkpointer should do it?
Can saving still only run on the main process under FSDP, or do the other processes need to explicitly do something?
Also, the intra-epoch checkpoint saving as currently implemented (based on time) is not compatible with this: https://github.com/speechbrain/speechbrain/blob/5d479fc6713f3aa0e218802019b7f7bceb46146c/speechbrain/core.py#L1136
This makes me realise that if we put run_on_main somewhere, then all processes must hit that code, so if the run_on_main goes into classes, we must ensure that all processes run that (and in the same order, i.e. if one process runs into save barrier and another into gradient sync barrier then it ends up in deadlock). Maybe we don't need the barrier inside run_on_main in many cases? There is already the if_main_process() (which really should be called is_main_process()), that might be enough in most cases. The other processes will wait at the next barrier, while main process saves or does whatever errands we need.
Note: Checkpointer should also wrap the checkpoint deletion in run_on_main.

This seems better to me. But in the current implementation any registered save hooks have expected to be called on the main process. I don't know if Checkpointer should wrap saving in run_on_main or the individual objects. Maybe Checkpointer should do it?
Can saving still only run on the main process under FSDP, or do the other processes need to explicitly do something?

You've correctly deduced the reason why I put this in individual classes -- some (including FSDP) may want to run code on all processes. Perhaps a more backwards-compatible way of doing this would be to add some sort of argument (to mark_as_saver?) that would by default run only on main but would allow running on all if passed.... Thinking aloud on this one.

Also, the intra-epoch checkpoint saving as currently implemented (based on time) is not compatible with this:
https://github.com/speechbrain/speechbrain/blob/5d479fc6713f3aa0e218802019b7f7bceb46146c/speechbrain/core.py#L1130-L1144
This makes me realise that if we put run_on_main somewhere, then all processes must hit that code, so if the run_on_main goes into classes, we must ensure that all processes run that (and in the same order, i.e. if one process runs into save barrier and another into gradient sync barrier then it ends up in deadlock). Maybe we don't need the barrier inside run_on_main in many cases? There is already the if_main_process() (which really should be called is_main_process()), that might be enough in most cases. The other processes will wait at the next barrier, while main process saves or does whatever errands we need.
Note: Checkpointer should also wrap the checkpoint deletion in run_on_main.

Perhaps by default we should call if_main_process() unless the barrier is really needed. Wondering if there would be a reason for the barrier that I haven't thought of -- a timeout in the gradient sync or something like this.

Continuing on the checkpoints with time intervals: basically, time cannot be relied on for multi-process synchronisation (almost by definition). If FSDP needs Checkpointer code to run on all nodes then I think that this checkpoint with time interval is not possible with FSDP.
With the coming changes in SpeechBrain, I think we can afford to break some backwards compatibility and move to checkpoints every x steps. Some people also requested that anyway.

I think the mark_as_saver decorator could indeed have an optional parameter (something like @mark_as_saver(main_process_only=False)), and checkpointer could then keep track of this.

With the coming changes in SpeechBrain, I think we can afford to break some backwards compatibility and move to checkpoints every x steps. Some people also requested that anyway.

Should that be done as part of this PR or could it be a separate one?

I think the mark_as_saver decorator could indeed have an optional parameter (something like @mark_as_saver(main_process_only=False)), and checkpointer could then keep track of this.

Implementation attempted here -- does this look reasonable?

I think the two separated sets of default save hooks (main proc only vs. all procs) will lead to many errors. When searching for a default hook, the code looks through the class hierarchy, where it will often find e.g. torch.Module, and use that default hook. While we could change the order so that all_procs hooks are prioritized (as they are less common), I fear that this will lead to similar weird and hard to find bugs.
I think we'd want to keep one set of save hooks. Perhaps the checkpoint registering could wrap the saver hook callable in a wrapper that only runs on the main process:
from functools import wraps
def main_process_only(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        if sb.utils.distributed.if_main_process():
            func(*args, **kwargs)
    return wrapper

Or we could keep more information about those hooks, e.g. the hooks dict could map types to some Hook dataclass with entries like callable, run_on_all_procs, etc. but then that might require changing all existing default hooks.

With the coming changes in SpeechBrain, I think we can afford to break some backwards compatibility and move to checkpoints every x steps. Some people also requested that anyway.
Should that be done as part of this PR or could it be a separate one?
It's something that breaks from this, so it could be done as part of this change or if we want to be tidy it could be a separate change implemented before this.

I think we'd want to keep one set of save hooks. Perhaps the checkpoint registering could wrap the saver hook callable in a wrapper that only runs on the main process

This is a neat solution, I wish I'd thought of it first!

With the coming changes in SpeechBrain, I think we can afford to break some backwards compatibility and move to checkpoints every x steps. Some people also requested that anyway.

Okay, I've made the change in core.py. Pretty much every recipe uses it, so it'll take some time to change over, but fortunately it's only a one-line change. I looked and the number of minutes varies from recipe to recipe. I'm not sure how to select a sane value for the checkpoint steps... any ideas?
I was thinking of assuming there's 5 iterations per second (a tremendously inaccurate assumption) and converting the minutes to steps (* 60 * 5), but there's probably a better way to do this.

Okay, I've made the change in core.py. Pretty much every recipe uses it, so it'll take some time to change over, but fortunately it's only a one-line change. I looked and the number of minutes varies from recipe to recipe. I'm not sure how to select a sane value for the checkpoint steps... any ideas?

Oof. Well, one idea is to keep the time based checkpointing, it is kind of useful and works in most cases. We could also try to detect FSDP and raise an error, though I guess at the moment we cannot easily detect FSDP. Maybe a hack for now, something like checking if the name "FSDP" is in globals, I don't know.

Otherwise, I think this could be a good implementation. You need to remember to wrap torch_recovery with main_process_only, and same for checkpoint deletion. Also, I wonder if many recipes do things like write some WER file in an on_stage_end, expecting this to happen only on main?

You need to remember to wrap torch_recovery with main_process_only, and same for checkpoint deletion.

Ah yes, I've added this for checkpoint deletion. However, for torch_recovery I'm pretty confident we don't want it to happen only on main, but rather on all processes, since we use this to load models on all devices (and there's no contested resource).

Also, I wonder if many recipes do things like write some WER file in an on_stage_end, expecting this to happen only on main?

These should be fixed:
$ grep -r 'with open(self.hparams.wer_file' recipes/ | wc -l
      33

But there are some cases of saving that don't necessarily need to be fixed:
grep -r 'torchaudio.save(' recipes/ | wc -l
      63

Looking at these, they mostly happen in places other than on_stage_end so it seems like the recipes are not expected to work on multiple processes.

You need to remember to wrap torch_recovery with main_process_only, and same for checkpoint deletion.

Ah yes, I've added this for checkpoint deletion. However, for torch_recovery I'm pretty confident we don't want it to happen only on main, but rather on all processes, since we use this to load models on all devices (and there's no contested resource).

Oops yea sorry, I meant torch_save

Side note: I was going to nitpick about using sys.exit to crash Brain in case of wrong arguments, but I noticed that this is done for some other options as well and you were using the same convention. I propose a separate PR to change all those to raising an
error like a proper respectful Python class should.

There is an issue with this PR, as reported by @Adel-Moumen:
"I trained a model with 3 GPUs and basically, I got [three] save folders"
and
"they all have:
# yamllint disable
WER: 5.784713797286864
end-of-epoch: true
unixtime:<PHONE_NUMBER>.4516888

except one with the right .ckpt.  however, the problem is that when loading the new ckpt we are getting an error saying that we are failing to load the .ckpt as there is no ckpt, and indeed, the issue is that the last ckpt (the most recent one) is lacking from this .ckpt"


Compile fails with tauri-1.0.0-rc.5.

Cargo.toml
[build-dependencies]
tauri-build = { version = "1.0.0-rc.5", features = [] }

[dependencies]
serde_json = "1.0.79"
serde = { version = "1.0.136", features = [ "derive" ] }
tauri = { version = "1.0.0-rc.5", features = [ "api-all" ] }

[dependencies.tauri-plugin-window-state]
git = "https://github.com/tauri-apps/tauri-plugin-window-state"

I get the following error message:
   Compiling tauri-plugin-window-state v0.1.0 (https://github.com/tauri-apps/tauri-plugin-window-state#9e57d4bd)
error[E0308]: mismatched types
  --> /Users/Foo/.cargo/git/checkouts/tauri-plugin-window-state-26e55ae178072d0f/9e57d4b/src/lib.rs:88:7
   |
87 |     window.on_window_event(move |e| match e {
   |                                           - this expression has type `&tauri_runtime::window::WindowEvent`
88 |       WindowEvent::Moved(position) => {
   |       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected enum `tauri_runtime::window::WindowEvent`, found enum `tauri::WindowEvent`

error[E0308]: mismatched types
   --> /Users/Foo/.cargo/git/checkouts/tauri-plugin-window-state-26e55ae178072d0f/9e57d4b/src/lib.rs:108:7
    |
87  |     window.on_window_event(move |e| match e {
    |                                           - this expression has type `&tauri_runtime::window::WindowEvent`
...
108 |       WindowEvent::Resized(size) => {
    |       ^^^^^^^^^^^^^^^^^^^^^^^^^^ expected enum `tauri_runtime::window::WindowEvent`, found enum `tauri::WindowEvent`

For more information about this error, try `rustc --explain E0308`.
error: could not compile `tauri-plugin-window-state` due to 2 previous errors

Gonna fix that on the Tauri side.

I could see the fixes merged. Thank you so much.


variables are not getting set if the SET_VARIABLE task is followed by a workflow task

The conductor is not setting any variables when a SET_VARIABLE task is followed by a workflow.
This is easy to reproduce with a small workflow having a set variable followed by another workflow.

As shown in the attached workflow sample snippet. the set_output_variables_ref_1 is trying to set two variables user and param.
even after the successful execution of the task the varaibles are not set and they dont even appear in the varaibles pane under"Workflow Input/output"
Hence when the last task divide in the snippet tries to access the variable param it gets null

I observe the same condition is true when the set variable task is followed by a switch task as well. the variables are not getting set globally. After the set variable task you can access using workflow.variables for the first access but post that the variable is getting set to null.

@missioncoding I believe this is a bug.  Will send a PR to fix.

Thanks @v1r3n. it would be great to get a fix asap.


TSC --watch should survive file renames

I use tsc --watch a lot but whenever I rename a file tsc --watch goes stale and doesn't recover from it hence I end up with stale JS without noticing. What you get in the console:
[1:13:49 PM] File change detected. Starting incremental compilation...

error TS6053: File '/..../npm/importLinker.ts' not found.

[1:13:50 PM] Found 1 error. Watching for file changes.

We should either fix this in TSC or at least make the task detect this and automatically restart.

This is not an issue with VS Code but with TypeScript. Please close this issue and create the same in TypeScript's official repository.
https://github.com/microsoft/TypeScript


TT1 Typo on the "Instructions" page

https://sagebionetworks.jira.com/browse/DCA-61

Closing this because it appears to be part of an outdated app. The instructions page is no longer part of the app.