QA / UDM Band and/or role?

In my Planet data descriptions I have an 'unusable data mask' asset, that's a single band with 7 values. Or I think maybe there's a second band that also has confidence?
It seems like there's no practices to describe it. But it strikes me that our UDM is really similar to the QA band in others. And that it'd be valuable to have in 'bands', and as a 'role'.
Thoughts? @matthewhanson

I was perusing the Earth Engine STAC catalog the other day and noticed they wrote a custom extension to convey bit-masked QA bands, for example Landsat 8:
{
  "name": "BQA",
  "description": "Landsat Collection 1 QA Bitmask ([See Landsat QA page](https://www.usgs.gov/land-resources/nli/landsat/landsat-collection-1-level-1-quality-assessment-band))",
  "gee:bitmask": {
    "bitmask_parts": [
      {
        "description": "Designated Fill",
        "bit_count": 1,
        "values": [
          {
            "description": "No",
            "value": 0
          },
          {
            "value": 1,
            "description": "Yes"
          }
        ],
        "first_bit": 0
      },
      {
        "description": "Terrain Occlusion",
        "first_bit": 1,
        "bit_count": 1,
        "values": [
          {
            "description": "No",
            "value": 0
          },
          {
            "value": 1,
            "description": "Yes"
          }
        ]
      },
      {
        "description": "Radiometric Saturation",
        "first_bit": 2,
        "bit_count": 2,
        "values": [
          {
            "description": "No bands contain saturation",
            "value": 0
          },
          {
            "value": 1,
            "description": "1-2 bands contain saturation"
          },
          {
            "value": 2,
            "description": "3-4 bands contain saturation"
          },
          {
            "value": 3,
            "description": "5 or more bands contain saturation"
          }
        ]
      },
      {
        "description": "Cloud",
        "first_bit": 4,
        "bit_count": 1,
        "values": [
          {
            "description": "No",
            "value": 0
          },
          {
            "value": 1,
            "description": "Yes"
          }
        ]
      },
      {
        "description": "Cloud Confidence",
        "first_bit": 5,
        "bit_count": 2,
        "values": [
          {
            "description": "Not Determined / Condition does not exist.",
            "value": 0
          },
          {
            "value": 1,
            "description": "Low, (0-33 percent confidence)"
          },
          {
            "value": 2,
            "description": "Medium, (34-66 percent confidence)"
          },
          {
            "value": 3,
            "description": "High, (67-100 percent confidence)"
          }
        ]
      },
      {
        "description": "Cloud Shadow Confidence",
        "first_bit": 7,
        "bit_count": 2,
        "values": [
          {
            "description": "Not Determined / Condition does not exist.",
            "value": 0
          },
          {
            "value": 1,
            "description": "Low, (0-33 percent confidence)"
          },
          {
            "value": 2,
            "description": "Medium, (34-66 percent confidence)"
          },
          {
            "value": 3,
            "description": "High, (67-100 percent confidence)"
          }
        ]
      },
      {
        "description": "Snow / Ice Confidence",
        "first_bit": 9,
        "bit_count": 2,
        "values": [
          {
            "description": "Not Determined / Condition does not exist.",
            "value": 0
          },
          {
            "value": 1,
            "description": "Low, (0-33 percent confidence)"
          },
          {
            "value": 2,
            "description": "Medium, (34-66 percent confidence)"
          },
          {
            "value": 3,
            "description": "High, (67-100 percent confidence)"
          }
        ]
      },
      {
        "description": "Cirrus Confidence",
        "first_bit": 11,
        "bit_count": 2,
        "values": [
          {
            "description": "Not Determined / Condition does not exist.",
            "value": 0
          },
          {
            "value": 1,
            "description": "Low, (0-33 percent confidence)"
          },
          {
            "value": 2,
            "description": "Medium, (34-66 percent confidence)"
          },
          {
            "value": 3,
            "description": "High, (67-100 percent confidence)"
          }
        ]
      }
    ],
    "total_bit_count": 13,
    "description": "Bit 0: Designated Fill\n * 0: No\n * 1: Yes\nBit 1: Terrain Occlusion\n * 0: No\n * 1: Yes\nBits 2-3: Radiometric Saturation\n * 0: No bands contain saturation\n * 1: 1-2 bands contain saturation\n * 2: 3-4 bands contain saturation\n * 3: 5 or more bands contain saturation\nBit 4: Cloud\n * 0: No\n * 1: Yes\nBits 5-6: Cloud Confidence\n * 0: Not Determined / Condition does not exist.\n * 1: Low, (0-33 percent confidence)\n * 2: Medium, (34-66 percent confidence)\n * 3: High, (67-100 percent confidence)\nBits 7-8: Cloud Shadow Confidence\n * 0: Not Determined / Condition does not exist.\n * 1: Low, (0-33 percent confidence)\n * 2: Medium, (34-66 percent confidence)\n * 3: High, (67-100 percent confidence)\nBits 9-10: Snow / Ice Confidence\n * 0: Not Determined / Condition does not exist.\n * 1: Low, (0-33 percent confidence)\n * 2: Medium, (34-66 percent confidence)\n * 3: High, (67-100 percent confidence)\nBits 11-12: Cirrus Confidence\n * 0: Not Determined / Condition does not exist.\n * 1: Low, (0-33 percent confidence)\n * 2: Medium, (34-66 percent confidence)\n * 3: High, (67-100 percent confidence)"
  }
}

@kylebarron - that's cool, I hadn't seen that.
Took a note https://github.com/stac-utils/stactools/issues/56 to do similar in Planet, but then realized that we don't use a single band for our UDM. So we'd need many band names, but even just using a 'role' might be useful?

I think the card4l 'roles' that are coming into best practices with #989 that are more specific. So I think I'll just use cloud, cloud_shadow, snow/ice roles all for the UDM.


fix: rollback auro-library to fix generateDoc scripting

Alaska Airlines Pull Request
auro-loader has v2.10 of auro-library installed which breaks the generateDocs scripting, and any component that has a dependency on them.
The component needs to rollback auro-library to v2.8.
Resolves: #46
Summary:
Rollback auro-library
Type of change:
Please delete options that are not relevant.

[X] Revision of an existing capability

Checklist:

[X] My update follows the CONTRIBUTING guidelines of this project
[X] I have performed a self-review of my own update

By submitting this Pull Request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Pull Requests will be evaluated by their quality of update and whether it is consistent with the goals and values of this project. Any submission is to be considered a conversation between the submitter and the maintainers of this project and may require changes to your submission.
Thank you for your submission!
-- Auro Design System Team

:tada: This PR is included in version 3.1.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Standardize See also sections

@NextTurn : Thanks for your contribution! The author(s) have been notified to review your proposed change. @Joe-Sewell-PreEmptive


unipept.log file

I got this message in the unipept.log file in my home folder when processing 12 samples of an EBI shotgun metagenomics project at once.

This website is under heavy load

We're sorry, too many people are accessing this website at the same time. We're working on this problem. Please try again later.

This is indeed a nice feature, but some minor remarks:

the command line tools each time overwrite the same file; is this what we want, or do we want log messages with a given timestamp
shouldn't the file be a hidden file, so .unipept.log ?

Original issue by @pdawyndt on Wed May 14 2014 at 20:44.
Closed by an unknown user on Mon May 19 2014 at 10:46.

(That message isn't something we added, but the webserver (passenger) complaining there are too many (100) open connections.)
Original comment by @bmesuere on Wed May 14 2014 at 20:59.


Support multi plugins for one target class

This pr is for fixing #192.
Adjust PluginFinder.pluginDefineMap. With the <string, list> data construction, multi plugins can target same class, of source only one will enhance the target class actually.
@bai-yang I think you are interested in this.
cc @sky-walking

Coverage decreased (-0.05%) to 71.146% when pulling 8b074a0a97e9fbefe8f01aaed9b573da74c174eb on feature/192 into ecf7a2bf2a5c589525105b7727c0ef630e784978 on master.


Can it work on GPU?

@lmolhw5252 yes but it's not optimized at all
Ive updated the section
https://github.com/philipperemy/tensorflow-multi-dimensional-lstm#limitations


Rebase (220820)

Rebase all changes from mainline.

Caution: Requires Updated ACPI for UFS CCA. See TG


Faster gather and scatter.

Reduce unnecessary integer ops, especially since
there kernels are integer bound.
Increase number of iterations for benchmarks for
better smoothing.
Github Issue #506
Proposed changes
Please include a description of the problem or feature this PR is addressing. If there is a corresponding issue, include the issue #.
Checklist
Put an x in the boxes that apply.

[X] I have read the CONTRIBUTING document
[X] I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
[X] I have added tests that prove my fix is effective or that my feature works
[X] I have updated the necessary documentation (if needed)

Perf on M1 Max MBP (32-core GPU, 64GB RAM)
gather_bench (Before)
====================
X (100, 64), Indices (1000000,)
MLX: 3.128ms
PyTorch: 13.422ms
X (100, 1024), Indices (100000,)
MLX: 4.917ms
PyTorch: 21.406ms
X (4, 1000000), Indices ()
MLX: 0.270ms
PyTorch: 0.173ms
gather_bench (After)
(mlx2) vjkrish@Vijays-MacBook-Pro-3 mlx % python benchmarks/python/gather_bench.py
X (100, 64), Indices (1000000,)
MLX: 2.232ms
PyTorch: 13.336ms
X (100, 1024), Indices (100000,)
MLX: 3.434ms
PyTorch: 21.071ms
X (4, 1000000), Indices ()
MLX: 0.277ms
PyTorch: 0.194ms

Scatter Perf on M1 Max MBP (32-core GPU, 64GB RAM)
Before
X (1000000, 64), Indices (1000000,)
MLX: 6.627ms
PyTorch: 13.380ms
X (1000000, 64), Indices (1000000,)
MLX: 6.778ms
PyTorch: 14.002ms
X (100000, 64), Indices (100000,)
MLX: 2.381ms
PyTorch: 1.571ms
After
X (1000000, 64), Indices (1000000,)
MLX: 5.925ms
PyTorch: 13.205ms
X (1000000, 64), Indices (1000000,)
MLX: 6.045ms
PyTorch: 14.023ms
X (100000, 64), Indices (100000,)
MLX: 2.280ms
PyTorch: 1.522ms


No warning on Android 12 for wrong time

Avoid duplicates

[X] Bug is not mentioned in the FAQ
[X] Bug is specific for Android only, for general issues / questions that apply to iOS and Android please raise them in the documentation repository
Related to https://github.com/corona-warn-app/cwa-app-android/issues/5183

Technical details

Device name: Samsung Galaxy A52 5G (SM-A526B/DS)
Android version: 12
App version: 2.25.0

Describe the bug
App does not stop exposure check taking place when date incorrectly set to future before installation on Android 12.
On Android 8 an error message is output and no exposure check takes place.
Steps to reproduce the issue
On Android 12:

Set device date to two days into the future
Uninstall CWA (if already installed)
Install CWA
Open CWA
Tap "Let's Get Started"
Tap "Next"
Tap "Activate Exposure Logging"
Tap "Turn on"
Tap "Next", tap "Next"
Tap "Do Not Share"
Tap "OK"
Tap "Allow", disable optimized battery usage for Corona-Warn (device specific)
Dismiss QR Code Scanner pop-up by tapping on "X"
Note "Exposure check failed, with timestamp Today"

Repeat the steps 1 - 11 on Android 8:
At step 11, Android 8 already shows an error message:

Tap OK for message "Exposure Check Not Possible"

Tap "Allow", disable optimized battery usage for Corona-Warn (device specific)
Dismiss QR Code Scanner pop-up by tapping on "X"
Note "Encounters have not yet been checked."

Expected behavior

On Android 12 the app should prevent an exposure check from taking place if the date does not match the server date. It should output the same error message "Exposure Check Not Possible" as displayed on Android 8 under the same conditions.

Internal Tracking ID: EXPOSUREAPP-9077

@MikeMcC399 Thank you for reporting the issue which is fixed with #5183

@sapcoder123

the issue ... is fixed with https://github.com/corona-warn-app/cwa-app-android/issues/5183

#5183 is another related issue. The resolution is supposed to be in PR #5465.

I successfully tested this issue with Corona-Warn-App-2.27.0-RC0-deviceForTesters-release.apk so I am now closing it.


v0.10 Release Tasks

[x] Port to ReScript syntax
[x] Align naming / terminologies
[x] Remove legacy code (see Changes.md for details)
[x] Upgrade to<EMAIL_ADDRESS>[x] Align upcoming docs for<EMAIL_ADDRESS>(available here)
[ ] Write migration-guide for<EMAIL_ADDRESS>-><EMAIL_ADDRESS>[x] Try rescript-react in existing codebases before official release
[x] Release<EMAIL_ADDRESS>[x] Add a note on reason-react for new users

@ryyppy can this be closed? Probably a bit late now anyway for the migration guide as 0.10.3 was published in May 2021...


Call from script?

how can you call the module from a script? I noticed once installed I can run a powershell and import-module to get the connect-exopssession command. I have several scripts that I user this to generate my connection:
$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.outlook.com/powershell -Credential $O365Cred -Authentication Basic -AllowRedirection
Import-PSSession $Session
Set-Location 'S:\Network Administration\Office365 Powershell'
.\office365.ps1

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 65e527db-a4c2-782e-5a5f-ad8b3159a977
Version Independent ID: f2de1272-5e37-c59f-fcb9-34fa66b0d268
Content: Connect to Exchange Online PowerShell using multi-factor authentication
Content Source: exchange/docs-conceptual/exchange-online/connect-to-exchange-online-powershell/mfa-connect-to-exchange-online-powershell.md
Service: exchange-online
Product: unspecified
GitHub Login: @chrisda
Microsoft Alias: chrisda

Sadly you can't connect with  $cred inside script, when you have enabled MFA.
You must type user/pass everytime, witch is a major drawback for administrators.
I'd like MS to make this more easy.
regarding your quiestion you can import the exo modul in your script with this:
Import-Module $((Get-ChildItem -Path $($env:LOCALAPPDATA + "\Apps\2.0") -Filter Microsoft.Exchange.Management.ExoPowershellModule.dll -Recurse).FullName | ?{ $_ -notmatch "none" } | select -First 1)
$EXOSession = New-ExoPSSession -UserPrincipalName $UPN
Import-PSSession $EXOSession -AllowClobber


Fixing cedar-java-hello-world for cedar-examples release/3.1.x

https://github.com/cedar-policy/cedar-examples/issues/168
Backporting changes from https://github.com/cedar-policy/cedar-examples/pull/170 and https://github.com/cedar-policy/cedar-examples/pull/173 to 3.1.x. See those PRs for more info.

Thanks! Can you add these changes to the main branch too?

https://github.com/cedar-policy/cedar-examples/pull/175


Delete pool cleanup test

Closes: #794
Description

For contributor use:

[ ] Targeted PR against correct branch (see CONTRIBUTING.md)
[ ] Updated relevant documentation (docs/) or specification (x/<module>/spec/)
[ ] Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
[ ] Re-reviewed Files changed in the Github PR explorer

Codecov Report

:exclamation: No coverage uploaded for pull request base (v6.x@f2a8bb0). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             v6.x     #799   +/-   ##
=======================================
  Coverage        ?   19.05%
=======================================
  Files           ?      167
  Lines           ?    23653
  Branches        ?        0
=======================================
  Hits            ?     4506
  Misses          ?    18368
  Partials        ?      779

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f2a8bb0...430143f. Read the comment docs.


manage_app_pool - add recycle logging control

manage_app_pool - add recycle logging control
new param default undef for backward compatibility
logic in class handles undef
caller specify one or more values from
['Time','Requests','Schedule','IsapiUnhealthy','OnDemand','ConfigChange','PrivateMemory']
empty array [] will clear current logging on app pool (set to no logging)

@lcoulson this will need a docs update and i'd really like to see some spec tests for this!

You'll need to drop the mixed case Puppet variables. Underscore variable names would be gooderific as well.

i'll fix this on Monday, I haven't any more strength today to battle rspec

@igalic @juniorsysadmin  - should be fixed now.
Added tests, standardized variable naming, added example to doc.
note - inconsistent rules for Robocop between these test build plan thingys.  most recent required crazy ruby 'word array' constructor slang %w(foo bar) - others do not.
formatting in puppet class of split string on unless only way I've found to satisfy all test types for rspec, so yes that spit is flush against the hard left of the line.


Simplified json schemas by breaking out common definitions

Created a common definitions schema in schemas/Definitions.
Referenced some common definitions from the other schemas to that one.
I needed to instantiate a local RefResolver to be able to use a custom uri handler which was needed to be able to use relative file paths in the schema refs.

While I sympathize with the intent, I'm not convinced.
As I see it, there are pros for breaking out common snippets:

Less boilerplate; more concise schemas.
Easier to spot any inconsistencies between events.

But there are also cons:

Creates coupling between events, which are in fact independent in definition and in versioning.
Will require management of dependencies, and additional version control of the common snippets.

All in all, let's say I'm tentatively against it :)

(That being said, we have a problem in how meta and links are documented today. As they are described in documents shared by all event types, there is an implicit coupling there. A more conducive way of documenting these would be good, without reiterating the same information over and over. It's essentially the exact same problem as for schemas.)

Should this be closed?


Update dalamud and net8

First time I meddle with dalamud, hope I got everything right.
Updated dalamud files (with a few changes) and net7 to net8, seems to be working as expected.
Edit: If anyone wants to use it while waiting for the update, can use my fork:
https://raw.githubusercontent.com/SubZero0/Dalamud.SkipCutscene/dist/repo.json
Fixes #18

Fixed by 21c1c0b


Access of static class method by static generic method

Dart SDK Version: from https://dartpad.dartlang.org
Windows (x64)
Google Chrome 68

I try call static method of class by generic method and it's not compiling with error:
Error: The method 'foo' isn't defined for the class 'dart.core::Type'
https://dartpad.dartlang.org/a94baf3ef84a6b22827a39aa4c5133b8
class A {
  static void foo() {
    print('I <3 ');
  }
}

class B {
  static void bar<T extends A>() {
    T.foo();
    print('Dart');
  }
}

void main() {
   B.bar();
}

PS: I doesn't know, it's an issue, bug or only question, but I would be glad if it was possible

And it's normally work in Java!
public class A {
  public static void foo() {
    System.out.print("I <3 ");
  }
}

public class B {
  public static <T extends A> void bar() {
    T.foo();
    System.out.print("Java");
  }
}

public class Main {
  public static void main(String[] args) {
     B.bar();
  }
}

It is not possible in Dart to call static methods this way.

Hi @artemirq, @matanlurey already mentioned that Dart static methods cannot be called as shown. However, I'm tempted to add a few more words on that.
First, it doesn't actually work in Java:
// Cf. SDK issue #34131

class A {
  public static void foo() {
    System.out.print("I <3 ");
  }
}

class A2 extends A {
  public static void foo() {
    System.out.print("Let's see whether B also <3 ");
  }
}

class B {
  public static <T extends A> void bar() {
    T.foo();
    System.out.print("Java");
  }
}

public class n000 {
  public static void main(String[] args) {
     B.bar(); // Prints 'I <3 Java'
     B.<A2>bar(); // Also prints 'I <3 Java'
  }
}

With javap you can see that it's generating code for an invocation of A.foo, and it doesn't matter that T can have other values than A for some invocations of bar:
class B {
  B();
    Code:
       0: aload_0
       1: invokespecial #1                  // Method java/lang/Object."<init>":()V
       4: return

  public static <T extends A> void bar();
    Code:
       0: invokestatic  #2                  // Method A.foo:()V
       3: getstatic     #3                  // Field java/lang/System.out:Ljava/io/PrintStream;
       6: ldc           #4                  // String Java
       8: invokevirtual #5                  // Method java/io/PrintStream.print:(Ljava/lang/String;)V
      11: return
}

But that's not surprising—Java must ignore the actual value of T, because Java performs type argument erasure during compilation, that is, there is nothing around at run time which could be used to select A.foo for the first invocation and A2.foo for the second one.
Dart has reified type arguments, so T is actually around at run time, and it wouldn't be particularly hard to support that T.foo() must call A.foo() when T has the value A, and A2.foo() when T is A2.
The main reason why we did not do it is that there is no typing relation between the static methods of any two classes, and it makes no difference whether those two classes are subclasses or subtypes of each other—each class has its own set of static methods, and there is simply no relationship between those sets of methods.
So even if we did give you the ability to do T.foo() where T is a type variable where we only know that T extends A, we would have no idea whether that T actually has a static method named foo, and, if so, whether it would accept the empty argument list.
So what you are asking for does not match up with the more strict typing that Dart uses today (especially Dart 2 which was released recently, but we've been moving in that direction for a couple of years before that).

Interesting! I had to find a bit of time to look into D in order to understand what on earth that piece of code is doing. ;-)
However, it looks like foobar is a template and the two invocations will actually be resolved at compile-time, such that the calls to T.foo() will work just like two separate pieces of code where T is replaced by A in the first case and by A2 in the second: We have the limitation 'Templates cannot be used to add non-static fields or virtual functions to classes or interfaces' here, which seems to fit that assumption.


Add minima 3 installation to readme

Closes #661

Wrong branch, sorry. Will create another one with the right branch.


Rewording needed?

Super small issue - I believe the description of the runImage parameter needs to be reworded here: https://github.com/pivotal/kpack/blob/4e5647354c3a55fee655199646059efd93f9b0c4/cmd/build-init/main.go#L30.

Good catch! @BRONSOLO could you make a real simple PR for that?

Sure thing @matthewmcnew! I just proposed a change.


Admin command to join or kick from lobby

Feature
Admin command to let others (or just the admins and moderators from the discord server) to force join the spectators into the lobby, and have a command to kick a person from the lobby if he's not in the voice channel (and not in the match) anymore.
Reasons for Implementing
For the force join command, the idea is to let someone with higher permissions to force others join the lobby, most to prevent someone from not joining, and also helping those who are playing using the mobile discord and mobile among us.
As for the kick command, I've noticed that if the person who is leaving the match doesn't use the command to leave the lobby, it will stay in there. Also, the same reason for the mobile players.
I guess this will help to manage the lobby a lot more easier.
Thanks in advance.
Ideas & Details
Alternatives

I've considered a force join. I think that's a good idea.
Fortunately, there's already the !sau eject @mention to kick people out. :)
(I haven't done permissions at all yet. It's on the todo list.)

I did saw the "eject" in the commands folder, but it was after I posted :v hahaha
I guess it would be nice to mention the command in the help menu, haven't found it there.
Also, regarding the commands, it would also be nice to have a list of what settings we can change using the config command, and an exemple of usage.
Many thanks mate! It's a preeeeetty nice job you're doing here

Huh... eject isn't in the help text. Interesting. I'll fix that.
For the config command, currently there's only a prefix option. (Kinda, mostly, almost, documented in the readme.) But you're right, that command needs better docs.
And thanks! I'm glad the bot is working out for you. :) Thanks so much for all your feedback. It's super helpful.

I've been thinking about this a bit more, and I don't think I can add a "force-join" command, since it would require allowing one player to alter the database record of another player. I might be able to solve that later, but it won't be near-term.
I've gone ahead and added an example for the config command, though, and added the missing eject command to the help. Those will both be in the next version. (It's tagged, so you'll see an update here.)

This issue is resolved in version 2.3.0


[Curiosity] Looking for a tutorial

Hello! First of all, sorry for my bad English... I'm not sure if I'm in the right place to ask this question.
I've been a Flutter developer for a long time, and I'm always following Gskinner's projects, I consider you my inspiration and the reference of the level I want to reach.
But I have difficulties in writing codes of this level, is there any tutorial like "How to program like the Gskinner team"?

No sorry, we don't have anything like that at the moment. The closest thing is probably our blog where we post various articles on different approaches etc: https://blog.gskinner.com/?s=flutter


Bump phpUnit version

Description
Bump phpUnit version
Documentation
Does this require changes to the WorkOS Docs? E.g. the API Reference or code snippets need updates.
[ ] Yes

If yes, link a related docs PR and add a docs maintainer as a reviewer. Their approval is required.

I have just opened a PR covering tests for all the supported versions of this package: cf. https://github.com/workos/workos-php/pull/179
 Some tests in the strategy matrix will probably fail either because phpunit < 10.4, or because the new minimum PHP requirement is now php >= 8.1 (8.0 beling still widely used)
cf. https://github.com/sebastianbergmann/phpunit/blob/10.4/composer.json#L25
I strongly believe that this version constraint should be reverted as this also forces any PHP project to have the very same phpunit constraint

@lucasmichot Thanks for pointing this out! We'll revert.


Cannot find package 'typescript'

Transcript:
❯ npx knip -h
node:internal/errors:484
    ErrorCaptureStackTrace(err);
    ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'typescript' imported from /home/tobi/.nvm/versions/node/v18.12.1/lib/node_modules/knip/dist/index.js
    at new NodeError (node:internal/errors:393:5)
    at packageResolve (node:internal/modules/esm/resolve:860:9)
    at moduleResolve (node:internal/modules/esm/resolve:909:20)
    at defaultResolve (node:internal/modules/esm/resolve:1124:11)
    at nextResolve (node:internal/modules/esm/loader:163:28)
    at ESMLoader.resolve (node:internal/modules/esm/loader:841:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:18)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:76:40)
    at link (node:internal/modules/esm/module_job:75:36) {
  code: 'ERR_MODULE_NOT_FOUND'
}

Node.js v18.12.1

I’m using nvm, maybe this is the reason?
❯ nvm -v
0.39.1

Guess this was some kind of auto-completion in my IDE, ts should be imported from a dependency (not a devDependency).
Thanks for reporting this, should be fixed in v0.13.1


New command to update dist-git from source-git

I'm personally not too excited about the fact that everything is written from scratch in a different way than rest of the codebase, some overall points:

Which part is more problematic? The "from scratch" or the "in a different way than the rest" one?

functions instead of classes

Why is this an issue? Here is my explanation, why this code was written using functions:
One of the reasons why I couldn't figure out a way to refactor the current propose-downstream/sync-release implementation is that it's written with classes. Even if one would only want to use one method of a class, an object of that class would need to be instantiated, and all the dependencies of that object need to be taken care of. Put in other words: while classes are a handy way to abstract things and make working with these abstractions easier, they also form an artificial boundary around their methods, making it impossible to use these methods outside of the context of the class.
This rigidity makes refactoring difficult. This is problematic, b/c until the workflow and the tooling for source-git is proven and mature, we might find the need to refactor or reshuffle code multiple times. Keeping implementation in functions could help with this.

rebase-helper is not used

Why is this a problem?
Tbh, I see using rebase-helper a little bit problematic: Packit uses only a small subset of functionality from it (mostly things related to spec-files), but Packit's goals are often seem to diverge from rebase-helper objectives. This is why we probably subclass rebasehelper.SpecFile. So I feel that rebase-helper is a problematic dependency. There are some ways this could be addressed: one is by not using it; or maybe by factoring our the spec-file related code in a separate library. :thinking:

sh instead of subprocess (what's actually the advantage of the library?)

sh is for subprocess what requests is for urllib3. It's already pretty wide-spread (it's also packaged for epel8), so I would say it should be pretty safe to use it, instead of writing our own code to work with subprocess.

I really like that source-git update logic is now completely factored out from the regular updates which will simplify that code immensely.
We should also cover the new code with the same set of test cases as the former codebase.

I wanted to test this, picked qemu-kvm in CentOS Stream 9 and sadly the spec couldn't be parsed:
Traceback (most recent call last):
  File "/home/tt/.local/bin//packit", line 33, in <module>
    sys.exit(load_entry_point('packitos', 'console_scripts', 'packit')())
  File "/usr/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/tt/g/user-cont/packit/packit/cli/update_dist_git.py", line 66, in update_dist_git_cmd
    update_dist_git(
  File "/home/tt/g/user-cont/packit/packit/update_dist_git.py", line 228, in update_dist_git
    source_archives = download_source_archives(
  File "/home/tt/g/user-cont/packit/packit/update_dist_git.py", line 151, in download_source_archives
    spec = rpm.spec(str(specfile))
ValueError: can't parse specfile

I presume the problem is that rpm isn't able to parse the spec since archive or patches are not present: https://github.com/rebase-helper/rebase-helper/blob/e20eece8881a59a48d1a44b8278cdd6596e0b0f1/rebasehelper/helpers/rpm_helper.py#L201

I'm personally not too excited about the fact that everything is written from scratch in a different way than rest of the codebase, some overall points:

Which part is more problematic? The "from scratch" or the "in a different way than the rest" one?

The first one: I don't want to end up in the state where we fix the same problems we had 2 years ago since this PR does not reuse anything from the existing codebase.

functions instead of classes

Why is this an issue? Here is my explanation, why this code was written using functions:
One of the reasons why I couldn't figure out a way to refactor the current propose-downstream/sync-release implementation is that it's written with classes. Even if one would only want to use one method of a class, an object of that class would need to be instantiated, and all the dependencies of that object need to be taken care of. Put in other words: while classes are a handy way to abstract things and make working with these abstractions easier, they also form an artificial boundary around their methods, making it impossible to use these methods outside of the context of the class.
This rigidity makes refactoring difficult. This is problematic, b/c until the workflow and the tooling for source-git is proven and mature, we might find the need to refactor or reshuffle code multiple times. Keeping implementation in functions could help with this.

That is a good point. On the other hand, if we start refactoring the class abstraction we have right now (which Mato already started #1157), we could still ended up with a solid, reusable design.
My problem with functions is that you need to explicitly pass every argument all the time, instead of passing it once to a class and then work with it as an attribute. As an example from this PR: patch_id_changed, copy_git_repo_dir, purge_repo - all three use git.Repo as the first argument, so it would be natural to me to group them in a class.
Overall, having a program structured in functions instead of classes is downgrade in the developer experience to me. On the flip side, well-scoped functions without nested dependencies are easier to test.

rebase-helper is not used

Why is this a problem?
Tbh, I see using rebase-helper a little bit problematic: Packit uses only a small subset of functionality from it (mostly things related to spec-files), but Packit's goals are often seem to diverge from rebase-helper objectives. This is why we probably subclass rebasehelper.SpecFile. So I feel that rebase-helper is a problematic dependency. There are some ways this could be addressed: one is by not using it; or maybe by factoring our the spec-file related code in a separate library. thinking

I still don't understand where the problem is. Yes, we are only using the SpecFile class in a different use case - at the same time, rebase-helper is already a proven and battle-tested project which processed hundreds of spec files (and it has a ton of code to ease specfile parsing and fixes some questionable design decisions done in the rpm specfile parser).
So, if we don't need rebase-helper's functionality, let's use rpm parser directly, but once we start reimplementing that code base, we should get back to the drawing board.
+1 on separating the specfile parser in a dedicated library, that would be the best

sh instead of subprocess (what's actually the advantage of the library?)

sh is for subprocess what requests is for urllib3. It's already pretty wide-spread (it's also packaged for epel8), so I would say it should be pretty safe to use it, instead of writing our own code to work with subprocess.

what I absolutely hate about sh is that it doesn't print the whole output and trims the actual command stdout/stderr which complicated debugging in dist-git-to-source-git
just checked the docs and have to say, the async execution is pretty good, we should then be able to drop our code to print live logs

I wanted to test this, picked qemu-kvm in CentOS Stream 9 and sadly the spec couldn't be parsed:

Nice. Can you please share the source-git and dist-git repo and branches on which you've run the command and also the command you've run. Thanks!

I wanted to test this, picked qemu-kvm in CentOS Stream 9 and sadly the spec couldn't be parsed:

Nice. Can you please share the source-git and dist-git repo and branches on which you've run the command and also the command you've run. Thanks!

Oh, should have done that originally, sorry about it; here it is:
$ git remote -v
origin  git@gitlab.com:redhat/centos-stream/src/qemu-kvm (fetch)
origin  git@gitlab.com:redhat/centos-stream/src/qemu-kvm (push)

$ git rev-parse HEAD
08fec6178906e86920b6110f51a8a0cdbbadce74

$ packit srpm
2021-04-21 13:48:48.524 srpm.py           INFO   SRPM: /home/tt/p/c/s/qemu-kvm/qemu-kvm-5.2.0-15.g08fec617.fc33.src.rpm

$ git clean -df
$ git checkout .
$ echo 'making sure the working tree is clean'

$ packit update-dist-git . /home/tt/p/c/r/qemu-kvm
error: patch 1 defined multiple times
Traceback (most recent call last):
  File "/home/tt/.local/bin//packit", line 33, in <module>
    sys.exit(load_entry_point('packitos', 'console_scripts', 'packit')())
  File "/usr/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/tt/g/user-cont/packit/packit/cli/update_dist_git.py", line 66, in update_dist_git_cmd
    update_dist_git(
  File "/home/tt/g/user-cont/packit/packit/update_dist_git.py", line 228, in update_dist_git
    source_archives = download_source_archives(
  File "/home/tt/g/user-cont/packit/packit/update_dist_git.py", line 151, in download_source_archives
    spec = rpm.spec(str(specfile))
ValueError: can't parse specfile

$ git -C /home/tt/p/c/r/qemu-kvm remote -v
origin git@gitlab.com:redhat/centos-stream/rpms/qemu-kvm (fetch)
origin git@gitlab.com:redhat/centos-stream/rpms/qemu-kvm (push)

is this helpful?

Closing this, in favour of #1211 and future PRs.


[RFC] Merging RABIT into XGBoost.

Background
This is a RFC for merging RABIT into XGBoost.  For a long time, RABIT enables support of distributed training for XGBoost and is integrated as a git submodule.  Most of tests are run on XGBoost and the code base of XGBoost is tightly connected to RABIT.  For example, serialization is built on rabit serializable interface, nccl unique ID is obtained from RABIT, quantile merging between workers is based on RABIT's serializable allreduce handler.  Because there are more matured MPI solutions like OpenMPI and UCX for CPU, RABIT did not get too much attention beyond XGBoost.  Eventually maintaining RABIT in a separated repository creates more overhead on developers than actual benefits, which is one of the reasons that RABIT is rarely updated.  Also we plan to sunset the allreduce implementation in RABIT in the future and utilize other widely adopted MPI solutions listed previously.  Merging RABIT into XGBoost first will allow us achieve that incrementally with sufficient tests.
Plan

[ ] Merge rabit as a git subtree under XGBoost as a standalone directory, name of which is to be decided.
[ ] Merge tests into XGBoost.
[ ] Change CMake scripts for better integration.
[ ] Add win socket support in order to reduce the 3 rabit libraries into 1, while supporting distributed training on Windows. (rabit_robust, rabit_base, rabit_empty).
[ ] Enable clang-tidy on RABIT, other than code style, clang-tidy found some real issues on RABIT's code base so we should enable it as soon as possible.
[ ] Rework on the OpenMPI backend to have better support for new backend (to be decided), probably adding nccl as a new backend too.  This way we can have both old and new backends enabled for a smooth transition.
[ ] Decide our plan around single point model recovery.  This should be part of another issue so I won't delve into it.
[ ] Drop old backends.

Concerns
As if we replace RABIT with other MPI solution and drop single point model recovery, would it be better if we don't merge it at all?  This seems much cleaner, but as mentioned previously XGBoost is tightly connected to RABIT, also every change on RABIT must be tested on XGBoost first before merging.  The replacement won't be trivial and I would like to do it incrementally and carefully.

Rabit would still be important for integration with existing opensource ecosystem, e.g. Dask, spark and flink. The fault tolerance part is important for scaling in real world settings where a node can fail.
Single worker failure and restart is quite common in real world production env, due to pre-emption or spot instance on the cloud. Normal MPI-based solutions usually requires all node to be alive and no failure at all, which brings more cost for real world production settings.
So I think we should continue to support the rabit as the first class-citizen. While NCCL or OpenMPI can be supported, they are only useful in an HPC based setting, rabit should still be the primary protocol we use for distributed training

@tqchen Thanks for the comment.  Thus far single point recovery doesn't work on XGBoost as bootstrapping is still an unresolved issue.   We haven't decided what to do with it so we might continue extending the previous work on it or drop it completely in XGBoost.  The best scenario from my perspective is we can have it enabled on top of other MPI solutions.  If we want more development on it, incubating it inside XGBoost seems better than developing it as a stand alone project at this stage as most of the tests are running on XGBoost.  From our perspective @CodingCat @hcho3 merging RABIT into XGBoost can ease the development a lot.

I did see single point recovery work in previous cases, as long as the distributed protocol strictly follows the rabit protocol.
Hopefully the motivation was clear, that in production systems single point failure can happen more often and being able to handle them is critical to make the system scalable in a prod setting. I did recall the code worked for single point failure before.
If it does not work for the current version m, we should spend some effort to look into it and resolve  the issue.
I am not against moving rabit into the codebase.
However I do not agree that the path forward is to build the solution on top of existing MPI interfaces. Most MPI API brings requirements that goes beyond allreduce, and it is impossible to resolve single point of failure in common MPI based setting — this is the motivation of the original the rabit protocol.
One goal of distributed XGBoost is to be able to embed the job into common distributed runtimes with the minimum requirement of fail-restart for fault tolerance.
We should continue to make the rabit based protocol as the primary way to drive such needs and to only make MPI as something that is optional

Got it.  The RABIT protocol is difficult to follow for most of the existing pipelines in XGBoost as we need synchronization during DMatrix construction.  So the real issue is how to recover DMatrix in a multi threaded (async) environment.  I'm not sure about the case you have seen as who did the test did not made the test part of XGBoost and so far I haven't seen any working example.  I believe it's possible to make it working in general XGBoost pipelines but with good amount of effort.  Let me sort out a more detailed roadmap on how do we proceed with this.

@tqchen I just thought about the spot instance use case, I'm not sure how single point model recovery can help.  Since once a worker got unplugged all the other workers can not proceed until there's a replacement worker available.   So wouldn't it be better if we just check point the model and stop the training altogether?  XGBoost does work and being tested for training continuation.  For a concrete example, if we are training using 4 workers and 1 of them got unplugged during the 3th iteration, we can simply checkpoint the 2 constructed forests/trees, and start training from 3 iteration again with re-partitioned data on remaining workers or a new set of workers according to some scheduling algorithms?
What I'm confused is, to me the only use case for single point model recovery in spot instance is we let all the other workers idle and wait for a new worker from scheduler.

In most of the distributed execution env(e.g. spark, hadoop, dask). The data partitioning is done as part of the pipeline, in most such cases, the basic primitive are
mapGroup(data, some-func)

Within the map function, the data is partitioned(by the execution env) into several group, and each of the group contains one partition of the dataset. Most of these frameworks also have fail-restart mechanism built in, which means if one of the worker, say worker0 get unplugged due to pre-emption or failure. The specific part of the job will restart on another instance somewhere. The rabit protocol fits into such setups naturally.
While we could do things like repartitioning the data, or using other tricks, the data shuffling mechanisms need to be implemented in xgboost itself and breaks the principle of natural integration with these frameworks.
More importantly, in cases where hundreds of workers are involved in the training. The cost of restarting a single worker is much smaller than restarting all the workers all together. Imagine each worker have a probability p of being preempted for each iteration, and n is the number of workers. The success probability of restart all together approach(which means all workers need to be alive for a single iteration) goes down as we increase n, which is a quiet undesirable problem. That means the bigger the dataset we want to train, there is a larger probability of failure.
In the case of single point failure recovery, while it is true that other workers need to stay idle while waiting for a single worker to restart, the cost of restarting the a single, or small number of workers is small, and is done by most of the distributed exec env.
The problem of initial statistics sync in DMatrix construction can be resolved by dividing the protocol into initial and iteration phase.  This is exactly the problem we aim to tackle in a real prod setting. For example, i know there are some uber's internal use-cases that relies on this feature, so that the distributed learning process can scale to the internal cluster where pre-emption can easily happen.

Thanks for the clarification, the failure probability bit helped a lot, also the possible cost savings.  I will consider putting more effort into supporting single point model recovery in the future, would also like to hear opinions from others.
But just to to clear, according to @CodingCat the feature is not used, also the bootstrapping work done by @chenqin laid down a solid foundation but it still has its unaddressed limitations so the feature can not be used yet.  Hence I believe enabling the feature is not a process of simple debugging but requires more thoughts and works.
I have some initial ideas on how to extend @chenqin s work on this, but before anything can happen I need to get opinions from other maintainers whether they want to share the responsibility of maintaining it, as I personally don't want to maintain a comm primitive library.

Right now spark package kills the spark context in the face of failure, which I believe is to avoid hanging caused by model recovery.

Again back to the point of being able to embed into distributed execution env. Note-ably there are two problems in relying on an existing MPI solution:

A MPI library may not be readily available, nor compatible with the existing distributed execution env. As a matter of fact, most MPI implementations present themselves as exclusive execution environment without fault tolerance baked in.
MPI contains a bigger collection of primitives that goes beyond allreduce, as a result, and it is much harder to design fault tolerance protocol for the MPI based solution.

Thanks for the detailed explanation.  Let me dig into the issues you listed.  :-)

As a matter of fact, most MPI implementations present themselves as exclusive execution environment without fault tolerance baked in.

Indeed ..

my 2 cents here
for us to use in a production environment,  fail all and recover from checkpoint is good enough. As many other frameworks like TF seems also go in this path. Based on this, to maintain Rabit the complexity of which are mostly comes from recovery logic is really an overhead
However, there are cases that the single point recovery is useful, e.g. I do see sometimes the Spark executors are preempted but returned back within 5 mins , for this case, re-starting the whole Spark applications is less efficient than just hanging all other executors there and wait for things to be stable...
I discussed with @trivialfis regarding a similar story happened in Spark community. Originally we had Akka as the RPC call framework, which just looks like rabit here, most of charming features are about fault-tolerance, etc. but Spark didn't use it at all but pay the maintenance cost for dependency, etc. Later, Spark implements its own RPC framework with the same interface with Akka. There were 1-2 versions where Akka/Spark-RPC co-exists for the convenience of falling back until Spark-RPC fully replaced Akka.
For us, can we have a simple allreduce framework based on the same interface with Rabit which facilities the co-existence of multiple frameworks. Even when someone wants to implement a new AllReduce framework like Tiger, as long as she/he implements these interface, it can be integrated with XGB easily  (it's easy to do in JVM world, not sure C++....).

Some thoughts from me:
After @tqchen's explanation I see the benefit of single worker recovery. My concern is that we don't have enough contributors with expertise here to implement and make the feature robust. I think it needs a lot of work, and even then may not be super robust due to so many edge cases in xgboost, such as trying to recover with process_type=update or with GPU training.
If we remove single worker recovery and windows support (both broken), as well as apply clang-tidy analysis, we have a chance to get the code base into something we can maintain.

I think the bottom line is that we need to make sure that the interface are compatible, and can be embedded into the distributed exec env like Dask/Spark. There is a lot of values in ability to embed an allreduce into the distributed execution env. And I see value of that in rabit beyond the single point recovery ability.
It is fine to drop in and replace with other allreduce frameworks as long as the interface is standardized around the rabit style API for cases like GPU(where NCCL makes more sense)

It's fairly easy on C++, we just need to add an backend.  MPI is a no-go as mentioned by @tqchen .  But something more primitive like UCX might still be a viable option, I will need to look into it in with more details.  Single point recovery is a nice idea, especially in the face of growing size of data.  So if the RABIT protocol can be enabled as a thin layer on top of another comm library I'm fine with it.  But actually developing a comm library seems to be a bit too much for this feature.

https://github.com/dmlc/xgboost/pull/6112 Removes the single point recovery in XGBoost.

Merging RABIT is considered complete.  Future development will continue as part of XGBoost as discussed.  Thanks for everyone who participated, suggestions and guidance are invaluable.  @tqchen @CodingCat @RAMitchell


Number of products on wishlist page

Questions
Answers

Description?
When we have a restriction by productshown on wishlist view page the number of products that shows is not corect

Type?
bug fix

BC breaks?
no

Deprecations?
no

Ticket?
Fixes https://github.com/PrestaShop/PrestaShop/issues/28164

How to test?
Go to the backoffice, configure the number of products to show in list, add more products in wishlist then that number, in list view yyou will see that the number of products in paginated in wrong way. See here https://drive.google.com/file/d/1lv2MVH6wTiH3reIKj-SHHHAMWMMvKViQ/view

Possible impacts?
View wishlist page.

Hello @denys202
Thank for the "How to test," but could you report this bug by creating a new Issue? Thanks to this QA team can make sure that everything is reproducible and mark this problem as a bug.

@kpodemski Ok, thank you for your comment, I created an issue 28164

Thanks @denys202 & @florine2623


Current Pendot Designer.glyphsFilter crashes

With the latest changes the filter crashes on Glyphs startup and no longer activates, and generates the following error the next time Glyphs starts up:

I believe this is because, as dotting is now done using fontTools, you need the fontTools library installed in Glyphs (using the Plugin Manager). When we have something ready for "public" release, we'll put Dotter in the Glyphs Plugin Manager as well to make it easy to install, and that can declare a dependency on fontTools so that fontTools will be automatically installed with it.
For the time being I've made the plugin raise an alert message to tell the user to install fontTools if it is not found.

Unfortunately I can't reproduce it, it's working for me on the same Glyphs version. So if that doesn't work, the next debugging steps are:

Remove the plugin entirely and clone the repo again inside the Plugins directory.
Run Glyphs from the command line and look for the error message:

% /Applications/Glyphs\ 3.app/Contents/MacOS/Glyphs\ 3
2023-12-07 11:37:42.575 Glyphs 3[30660:757947] principalClass time: 0.853249  DotterController.glyphsReporter
2023-12-07 11:37:42.579 Glyphs 3[30660:757947] PendotDesigner has encountered a fatal error, and will now terminate.
2023-12-07 11:37:42.579 Glyphs 3[30660:757947] Traceback (most recent call last):

  File "/Users/simon/Library/Application Support/Glyphs 3/Plugins/Dotter/Pendot Designer.glyphsFilter/Contents/Resources/plugin.py", line 7, in <module>
...

Deleting the existing repo clone and recloning got it working again - thanks!


Dog list

@akrawchyk , spent some time styling the dashboard sidebar and header styles, not done and could use some guidance with styling and creating interactions with the checked in box and radio buttons. Does this look good to merge for today?
I'll create a new branch after and keep working tomorrow.

Looks great, thanks for splitting the files out! Just a few things:
Why did you change the default value for being checked in to true? Dogs should initially be considered checked out. https://github.com/istrategylabs/paw/pull/31/files#diff-c0d44b6ea516df6dabbed954868e1acaR7
And follow-up, why did you change when to show dog status updates? We should only show the different states a dog can be in while in the office if they are checked in. https://github.com/istrategylabs/paw/pull/31/files#diff-fc9c7da91ffd59f56bfa284a3016a7fbR19

@akrawchyk whoops, I was just changing the values to change what I was seeing on the front end. I just switched it back now


HTTP2: Send trailers with data

Kestrel is sending HEADERS and DATA together in a TCP packet, followed by HEADERS (trailers) in the next TCP packet.

The C-Core gRPC server sends HEADERS, DATA and HEADERS (trailers) in a single TCP packet.

So..

Do more TCP packets equal less performance?
Should Kestrel combine packets?

// @Tratcher @steveharter @davidfowl

Relevant code?
https://github.com/aspnet/AspNetCore/blob/b528fd0a46422d2625ef690e89bf04557bbd7286/src/Servers/Kestrel/Core/src/Internal/Http2/Http2OutputProducer.cs#L318-L326

Theory: It's likely this is the result of two passes through the loop. The first pass picks up the data before the pipe is complete here. The second pass picks up the complete status and sends the trailers.
In most cases there's going to be a non-trivial time gap between the data write and the call to Complete that sends the trailers. What pattern would you use to prevent the data from being available before calling Complete? GetMemory, Advance, but no call to FlushAsync? Even in that case we'd have to tweak Http2FrameWriter.WriteDataAsync not to flush.

GetMemory, Advance, but no call to FlushAsync?

Yes, gRPC does that for calls with a single message response. I expect that FlushAsync would always result in separate packets.

Even in that case we'd have to tweak Http2FrameWriter.WriteDataAsync not to flush.

That's what I thought. There would be a new version of WriteDataAsync that didn't flush which would be called in the code above (line 322).

What pattern would you use to prevent the data from being available to the read loop before calling Complete? GetMemory, Advance, but no call to FlushAsync?

Yes.

Even in that case we'd have to tweak Http2FrameWriter.WriteDataAsync not to flush.

That's what I thought. There would be a new version of WriteDataAsync that didn't flush which would be called in the code above (line 322).

I don't think line 322 where Http2OutputProducer.ProcessDataWrites() calls Http2FrameWriter.WriteDataAsync() needs to change.
We want to ensure the Http2OutputProducer.ProcessDataWrites() doesn't wake up at all meaning the call to PipeReader.ReadAsync() shouldn't complete until the entire response body is available.
This should already work because Kestrel's internal HttpResponsePipeWriter.Advance() calls HttpProtocol.Advance() which in turn calls Http2OutputProducer.Advance() which shouldn't flush.
We just need to ensure that we can signal the end of the response body and flush to observe backpressure at the same time. CompleteAsync() should work for that.
@jkotalik

@halter73 I don't follow. CompleteAsync isn't doing anything new, it's just doing it earlier than normal. The traces above show that the current model is splitting frames, presumably due to a flush somewhere, so how do we prevent that?

so how do we prevent that?

You said it yourself. "GetMemory, Advance, but no call to FlushAsync." The backpressure problem I bring up is unavoidable if we want to buffer indefinitely in order to avoid early flushes. We should have logic to always call FlushAsync after a certain amount of data is written.
Kestrel will do the final flush by calling WriteStreamSuffixAsync after the middleware exits. This no-ops if the middleware already called CompleteAsync. If middleware for whatever reason wanted to observe if there was backpressure due to flow control or something, the middleware would have to call CompleteAsync itself to observe Http2OutputProducer's _dataWriteProcessingTask.

@JamesNK want to retry this scenario with CompleteAsync?

I'm not advocating that Kestrel never flush data out before the response completes. I think for small responses (which will cover most API responses) it is a good optimization.

CompleteAsync also outputs two packets.

I'm not advocating that Kestrel never flush data out before the response completes.

I didn't think you were advocating that. The point I'm trying to make is that the "GetMemory, Advance, but no call to FlushAsync" pattern leaves when to flush completely up to the application not Kestrel. If the app doesn't call FlushAsync or WriteAsync, Kestrel cannot by contract flush on your behalf.
Kestrel can and does coalesce flushes if they are two frequent, but you can still end up with multi-frame responses as you observed, so having a way for the application to easily buffer and prevent flushing is great, but the application needs to have the logic "to always call FlushAsync after a certain amount of data is written" not Kestrel.

CompleteAsync also outputs two packets.

We'll need to investigate that. I don't think theres any reason in principle this shouldn't be combined into a single flush/packet given the current design.


Update asterisk syntax

Add switch statement
https://wiki.asterisk.org/wiki/display/AST/Switch+Statements
More codecs
https://wiki.asterisk.org/wiki/display/AST/Asterisk+Audio+and+Video+Capabilities
Spaces are allowed between equal

Have you tried contacting the maintainer?

I've just sent him a letter.

Hello,
All right for the new codecs and the switch keyword, but I have doubts about the other modification.
You allow spaces before the "=" sign when setting variables; I believe it is not valid: https://wiki.asterisk.org/wiki/display/AST/Asterisk+16+Application_Set
And it has the side-effect of breaking dialplan extension highlighting, although I don't understand why.

https://wiki.asterisk.org/wiki/display/AST/Asterisk+16+Application_Set

Indeed, Set(QWE = qwe) produces warning Please avoid unnecessary spaces on variables as it may lead to unexpected results ('QWE ' set to ' qwe').
But I added spaces for context options, for example:
[tcp]
        type = transport

spaces are allowed here
https://wiki.asterisk.org/wiki/display/AST/Asterisk+Main+Configuration+File

In this case the syntax file should be re-written to properly differentiate context options from variables (which is not so easy, admittedly).
Since the current modification introduces a regression in extensions highlighting, I think it should not be included in this pull request. I suggest you republish the pull request with only the 1st and 3rd line included. Re-writing the file for context options handling may go in a separate pull-request.

Agree, done.

https://github.com/vim/vim/issues/7070

OK for me but I don't have the merging rights.

Yep, I know.

I see that the maintainer OK'ed this change, thus I'll include it.  Please don't forget to update your own copy.


UNOMI-616: Update GraphQL Playground to the latest version or switch …

UNOMI-616 - Update GraphQL Playground to the latest version or switch to GraphiQL
Switched from GraphQL Playground to GraphiQL Playground v3.3.1
Module unomi-graphql-playground was renamed to unomi-graphql-ui

@sergehuber Did you have the time to look at my PR?

Could you fix the conflict @anatol-sialitski and then I'll test it ?

@sergehuber done, please have a look at


Maya wrapper for K8s Service

Is this a BUG REPORT or FEATURE REQUEST?
FEATURE REQUEST
Why is this required?

This will help maya in building a K8s Service object from various options that are DevOps friendly
This is part of #976

Merged as per pull openebs/maya/pull/218


Dataset access

Hi!
Is it possible to access the SKM-TEA dataset? I cannot find the link in the paper or in the repository.
Thanks!

Thanks for your interest! You can download the dataset at the link below:
https://stanfordaimi.azurewebsites.net/datasets/4aaeafb9-c6e6-4e3c-9188-3aaaf0e0a9e7
Please note that the data sharing and paper are undergoing some final processing and will be ready by the Neurips camera ready deadline (early November). There will also be guidelines for how to use the dataset and how to compute/report metrics for standardized comparison at that time.


Couple of typos

@gewarren : Thanks for your contribution! The author, @asb3993, has been notified to review your proposed change.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mac/index.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Thanks for this @gewarren !!


Switch to google-auth-library

With some of the recent additions to google-auth-library. I think we can switch this lib off of google-auto-auth.  It may be nice to eliminate the middle-lib here.
@stephenplusplus what are your thoughts?

Sure, that would be fine. If google-auto-auth is invalidated, it should be chopped!


meshio-convert vtk to msh version 2

Dear all,
I would like to know if it is possible to export a mesh in vtk format (generated with tetgen) to gmsh format (ascii version 2). The meshio-convert option is not clear for me, my first attemps are the following:
meshio-convert -i vtk -o gmsh22 test_mesh.vtk test.msh
WARNING:root:Binary Gmsh needs 32-bit integers (got int64). Converting.
meshio-convert -i vtk -o gmsh test_mesh.vtk test.msh
WARNING:root:Binary Gmsh needs c_size_t (got int64). Converting.
The idea is to preserve the material tags of the vtk mesh ($ PhysicalNames). A test vtk mesh is attached.
Furthermore, the pytest has not passed, there are many errors with the installation. My numpy version is 1.18.2 under ubuntu 18.04. The pytest output is also attached.
output_pytest.txt
test_mesh.zip

Could you send a smaller example?  I'm working under straitened circumstances at the moment; meshio does O. K. but the Gmsh GUI is sluggish with so many cells.
I'm trying:
import meshio

mesh = meshio.read('test_mesh.vtk')
mesh.cell_data['gmsh:physical'] = mesh.cell_data.pop('cell_scalars')
meshio.write('test_mesh.msh', mesh, file_format='gmsh22')

Sure, this a smaller example (I hope)
test_mesh_small.zip

This isn't smaller:
<meshio mesh object>
  Number of points: 43023
  Number of cells:
    tetra: 265604
  Cell data: cell_scalars

If you have access to a larger machine, you could try the script suggested above.

Sorry, I will check your script. In addition, I can also convert tetgen node/ele to gmsh? If it is possible, which are the instructions? In this way, the physical tags are preserved?

Hi, after using your script, the issue persists on a machine with 128gb of RAM:
WARNING:root:Binary Gmsh needs 32-bit integers (got int64). Converting.
the output doesn't make sense.
I have prepared the smallest mesh I can generate to reproduce the error. I hope it is suitable for you to do the tests.
small.zip

Thanks. I'm away from.the desk for a.while now but this warning

WARNING:root:Binary Gmsh needs 32-bit integers (got int64). Converting.

can safely be ignored, I'm pretty sure. (I mean it should probably be fixed, but I haven't seen it do any harm.)
How do you mean

the output doesn't make sense.

Do you mean the output file or that warning? I thought the file looked O. K., it was just too big to easily inspect.

My output.msh file is attached. The gmsh format is not consistent and the output is the same on both machine (local and another cluster)
output_test_mesh.zip

At 144 437 tetrahedra, this mesh is still a little big for debugging.  Do you mean that small (less than a hundred elements, say) meshes generated in the same way don't evince the bug?  There is something funny going on here, but I can't quite pick out what it is as there are too many points and cells to read through.
Aside from a that, a correction: Gmsh's physical identifiers start from 1 not 0 so please add 1 when converting from VTK.  Sorry.
mesh.cell_data['gmsh:physical'] = [k + 1 for k in mesh.cell_data.pop('cell_scalars')]

Don't worry!
But no, the issue persists... please see the attached test case:
Mesh points: 186
Mesh tetrahedra: 584
The output doesn't have a
small.zip
sense...

I've just fixed an issue related to gmsh22 reads that might or might not be related https://github.com/nschloe/meshio/pull/769.
@ocastilloreyes Writing that "the output doesn't have a sense..." is not very helpful. Try to answer the following questions: What doesn't work exactly? What do have to do to reproduce the issue? What do you expect instead?
Apart from that, let me add that gmsh22 is a terrible format. Also, you never want to work with ASCII data. VTK binary is fine.

@nschloe thank you for your feedback. Right now I have done the update but the problem has not been solved.
On the clarity of the issues, I comment on the following:

The output doesn't have a sense because the content of the msh file is something like:

This is binary gmsh22. Seems okay.
Do you need ASCII format? Again, this is not something that anyone would recommend, unless for debugging. Use binary format whenever possible. If you really want ascii, you could use
meshio-convert out.vtk out.msh --ascii

or
meshio-ascii out.msh

on the command line.

Now is correct, misinterpretation of mine. I am sorry...
problem solved...
Thank you for your support

Cool.


[fix] fix dereference to nullptr in Tablet::calc_delete_bitmap

Proposed changes
Issue Number: close #xxx
Problem summary
Describe your changes.
Checklist(Required)

Does it affect the original behavior:

[ ] Yes
[ ] No
[ ] I don't know

Has unit tests been added:

[ ] Yes
[ ] No
[ ] No Need

Has document been added or modified:

[ ] Yes
[ ] No
[ ] No Need

Does it need to update dependencies:

[ ] Yes
[ ] No

Are there any changes that cannot be rolled back:

[ ] Yes (If Yes, please explain WHY)
[ ] No

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

*** Query id: 0-0 ***
*** Aborted at<PHONE_NUMBER> (unix time) try "date -d @1671959691" if you are using GNU date ***
*** Current BE git commitID: 2b61c39 ***
*** SIGSEGV address not mapped to object (@0x18) received by PID 1682989 (TID 0x7f82b1bfb700) from PID 24; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at ../src/common/signal_handler.h:420
1# 0x00007F846C128060 in /lib/x86_64-linux-gnu/libc.so.6
2# doris::Tablet::calc_delete_bitmap(doris::RowsetId, std::vector<std::shared_ptrdoris::segment_v2::Segment, std::allocator<std::shared_ptrdoris::segment_v2::Segment > > const&, std::unordered_set<doris::RowsetId, doris::HashOfRowsetId, std::equal_todoris::RowsetId, std::allocatordoris::RowsetId > const*, std::shared_ptrdoris::DeleteBitmap, long, bool) at ../src/olap/tablet.cpp:2049
3# doris::Tablet::update_delete_bitmap_without_lock(std::shared_ptrdoris::Rowset const&) at /usr/local/include/c++/11.3.0/bits/shared_ptr.h:122
4# doris::Tablet::revise_tablet_meta(std::vector<std::shared_ptrdoris::RowsetMeta, std::allocator<std::shared_ptrdoris::RowsetMeta > > const&, std::vector<doris::Version, std::allocatordoris::Version > const&) at ../src/olap/tablet.cpp:212
5# doris::EngineCloneTask::_finish_full_clone(doris::Tablet*, doris::TabletMeta*) at ../src/olap/task/engine_clone_task.cpp:654
6# doris::EngineCloneTask::_finish_clone(doris::Tablet*, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, long, bool) at ../src/olap/task/engine_clone_task.cpp:521
7# doris::EngineCloneTask::_do_clone() at ../src/olap/task/engine_clone_task.cpp:117
8# doris::EngineCloneTask::execute() at ../src/olap/task/engine_clone_task.cpp:69
9# doris::StorageEngine::execute_task(doris::EngineTask*) at ../src/olap/storage_engine.cpp:1025
10# doris::TaskWorkerPool::_clone_worker_thread_callback() at ../src/agent/task_worker_pool.cpp:917

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.81 seconds
load time: 641 seconds
storage size:<PHONE_NUMBER>5 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221226142026_clickbench_pr_69001.html


Users can't specify additional modules

It came up in an INC recently (INC0347305) that a user needed to load a particular module, and while they could do so in a terminal session, they couldn't for an rstudio session.
Specifically the user needed nc-config out of the netcdf module becuase they where trying to build and install the R package ncdf4 (which required nc-config in PATH).
This ticket is to allow users to load arbitrary modules as they please.

This was fixed in #56

This was fixed in #56


Navigator.BackAsync exception when getting back from modal to another modal

Hi! I have some modals and for one case I have to open one modal from another. It work fine, but when I use Navigator.BackAsync, I'm getting exception:
The control Pixed.Application.Windows.PaletteWindow (Content = DockPanel) already has a visual parent AvaloniaInside.Shell.StackContentView (Name = PART_Modal) while trying to add it as a child of AvaloniaInside.Shell.StackContentView (Name = PART_ContentView).'
PaletteWindow is a parent modal. Exception is thrown in StackContentView::PushViewAsync#L64. I locally added catch to try/finally for ignoring this exception and it works, but its temporary workaround.

Hello @Mateusz-Nejman
Thanks for reporting the error. I can see line above there is a check to remove and add again the control if already has a parent.
It can help, if you can provide an real example or solution that I can run and debug.
Thanks.

Repro repository: https://github.com/Mateusz-Nejman/AvaloniaInside.Shell-Issue-64


specifies to drop packets in case egress_port is invalid

Fixes #317 (actually only the remaining clarification).

Chiming in late, but I think what you were both moving toward was to have PSA define a constant that is the initial value of the egress spec and does not correspond to any port, physical or virtual. Although the semantics of "undefined values" is somewhat subtle in P4_16 (as we all know) at least we would have that if that field is never assigned to, then it still has that value.
I agree with @jafingerhut that it's problematic to talk about undefined values.
I also like this suggestion:

I am perfectly fine using a phrase like this in the PSA pseudocode: "if (ostd.egress_port is a value that is not associated with any output port) { drop the packet }". That can be implemented at run time.


Octal numbers in vim9script.

Steps to reproduce
Running:
:legacy echo 0700
" Outputs: 448

:vim9cmd echo 0700
" Outputs: 700

Expected behaviour
When enter octal numbers in vim9script with format 0<number> outputs the correct correspondent decimal value.
With formats 0o<number> or 0O<number> everything works well, that make me think if this the expected behavior.
Version of Vim
9.0.1677
Environment
OS: Void Linux
Terminal: st
$TERM: st-256color
Shell: zsh 5.9
Logs and stack traces
No response

Right, it is working as documented at :help octal:

Oh, before I find that octal numbers with 0 only prefix are no more valid, I was read the doc but don't have see that for some reason. Thanks.


Everything depends on common module

Hey @Morfly !
I loved your presentation on Android Worldwide. I love how every dependency is so well scoped and the modules are loosely coupled.
The only problem I can see right now is that literally every module depends on common. I feel like when we have a new feature request we will introduce new Repositories and new domain models, but the repository and the models will only be really used in one or two feature modules.
Because the repositories and the domain models all live under the common module, every change to them will recompile the entire app, right? If so do you think it would be a good idea to separate the common modules based on the domain models, eg:
banking app

transactions module with Transaction domain model and TransactionRepository
user module with User domain model and UserRepository.

These modules could work like common, but only those modules would depend on them that really need the models and repositories.
Does this make sense, or would this be unnecessary?

Hi @AradiPatrik
Thank you for watching my talk!
You're raising a very reasonable point! Because the sample project is small I added only 1 common module and put everything there. However, for larger projects, it's definitely a good idea to group the domain components of the app into corresponding separate modules and avoid storing all of them in a single common module.
This will help to ensure the change in one of the common modules won't drop the local cache of the entire project but rather only of those components that use transactions or user respectively.
Thank you for calling this out!

@ber4444 For my project I ended up having use case modules, like: ImageManipulations, FaceDetection, Segmentation
ImageManipulations contains use-cases like converting white pixels to transparent ones etc...
FaceDetection and Segmentation modules use the on device ML and provide use cases to detect and segment faces
StickerGeneration contains domain use-cases for generating stickers and depends on the modules above
I do have api and impl modules for them and in the api I have the following:
interface ImageManipulationsProvider {
    val loadImageBitmapFromUri: LoadImageBitmapFromUri
    val cropToRect: CropToRect
    val cropToContour: CropToContour
    val getImageExifRotation: GetImageExifRotation
    val createMask: CreateMask
    val scaleBitmap: ScaleBitmap
    val padBitmapToSize: PadBitmapToSize
    val addTextToImage: AddTextToImage
    // ...
}

In the impl module I have the @Module and the @Component
In the Application class I have lot's of code like this:
val stickerGenerationProvider = DaggerStickerGenerationComponent.builder()
            .platformProvider(platformProvider)
            .faceDetectionProvider(faceDetectionProvider)
            .imageManipulationsProvider(imageManipulationsProvider)
            .segmenterProvider(segmenterProvider)
            .dataProvider(dataProvider)
            .build()

It's a lot of boiler plate in the Application class
In the MainActivity I have this monstrosity now:
CompositionLocalProvider(
      CompositionLocals.ofType<NavigationProvider>() provides application.appProvider,
      CompositionLocals.ofType<DataProvider>() provides application.appProvider,
      CompositionLocals.ofType<PlatformProvider>() provides application.appProvider,
      CompositionLocals.ofType<FeatureEntriesProvider>() provides application.appProvider,
      CompositionLocals.ofType<FaceDetectionProvider>() provides application.appProvider,
      CompositionLocals.ofType<ImageManipulationsProvider>() provides application.appProvider,
      CompositionLocals.ofType<SegmenterProvider>() provides application.appProvider,
      CompositionLocals.ofType<StickerGenerationProvider>() provides application.appProvider,
      CompositionLocals.ofType<LandingProvider>() provides application.appProvider
) {
      App(navController, appViewModel)
}

And then again in the features I have similar code to this:
val currentDataProvider = CompositionLocals.current<DataProvider>()
val currentPlatformProvider = CompositionLocals.current<PlatformProvider>()
val navigationProvider = CompositionLocals.current<NavigationProvider>()
val stickerGenerationProvider = CompositionLocals.current<StickerGenerationProvider>()
val faceDetectionProvider = CompositionLocals.current<FaceDetectionProvider>()
val imageManipulationsProvider = CompositionLocals.current<ImageManipulationsProvider>()
val landingProvider = CompositionLocals.current<LandingProvider>()

return rememberScoped(rootEntry) {
   DaggerPromptToStickerRootComponent.builder()
                .dataProvider(currentDataProvider)
                .platformProvider(currentPlatformProvider)
                .faceDetectionProvider(faceDetectionProvider)
                .stickerGenerationProvider(stickerGenerationProvider)
                .imageManipulationsProvider(imageManipulationsProvider)
                .navigationProvider(navigationProvider)
                .landingProvider(landingProvider)
                .build()
}

So @ber4444 I really like the idea of yours where I would just not do this api/impl dance for use-cases because it leads to lots and lots of boilerplate.
But do you also suggest to ditch the common data module? Or not to have api/impl separation for the data module either? Because if we want to have one single db for our app (which is quite convenient) then using one single module for the db is inevitable, and it feels wrong to depend on the implementation details of this module from almost every use-case.
If we don't have api-impl separation between the use-cases and the data module then if the data module implementation details change then all the use-cases have to be recompiled and so all the feature modules have to be recompiled so in the end the whole world has to be recompiled.

api and impl is optional here considering Kotlin has compilation avoidance and incremental compilation

But the api and impl separation still has the benefit that if only the implementation detail of a use-cases changes then only that one single module has to be recompiled. The dependent features don't have to, since the api didn't change at all.
So @ber4444 @Morfly My question is:

Should I use api/impl separation for things like my use case modules?
If I want a single db for my app, should I use api/impl separation to share it between use cases, or should it be one single module?

Thank you again for your time and effort you've put into sharing about such a difficult topic 🚀

Hi @AradiPatrik, in my mind the narrowly scoped use case modules would be the abstractions in front of the data module, and feature modules should not depend on data any more.  And keep data as lean as possible, e.g. hold your REST interface and/or GraphQL schema + queries, but not much more.
You can measure build speed via Gradle's --scan option and see it for yourself, e.g. make an ABI-incompatible change in data/impl or common such as renaming a method, and see the speed difference between that and making it in a featureX/impl (screen destination) or usecaseX (narrowly scoped and regular) module.
Modules without api/impl separation perform extremely well (as long as they are not behind composition locals), due to Kotlin's compilation avoidance, which as per https://youtrack.jetbrains.com/issue/KT-24203/Enable-Compile-Avoidance-for-kotlin-in-gradle-builds wasn't quite available yet when @Morfly came up with this project, but it is available now.

Hi @ber4444 Thanks that's some great information! 🙌
For my next project I will definitely skip the api/impl modules for my usecase and data classes, and have the features depend on narrowly scoped use case modules.

And keep data as lean as possible, e.g. hold your REST interface and/or GraphQL schema + queries, but not much more

What about Room database? Are you suggesting having multiple databases in the app?
Anyway I will also take your advice and use the --scan option more so I can start optimizing the build speed when there's an actual issue with build speed!


Suggestion to proxy SSH from vm; basic HTML UI

I like the idea of this tool, as it allows to easily expose a VM to anyone on a tailnet. However, while the idea of SSH to control the VM state is novel, it is probably a better idea to proxy the actual SSH of the host to UI. Below is a quick mockup of a minimal interface that looks a lot like the one provided by cockpit.

For example, the dropdown allows switching between VNC Console and an SSH terminal. Under the menu that popped out is a Expand [] button to allow the desktop/terminal to take over the whole area. The URL vmproxy/#terminal would automagically open an expanded terminal:

and similarly for #desktop.
Note: Migrate has no meaning here (copy/paste) from the Cockpit UI. And Console being a placeholder for the vmname.
Also, this would mean the command line could be:
$ vmproxy <vmname> <IP address> <vnc session|:0> [--ssh]

With [--ssh] to allow SSH forwarding or not, and the VNC sessions defaulting to :0 for port 5900.
Note: --ssh should also register the node as it was run with setArgs.runSSH, and therefore allow the Dashboard to allow the javascript terminal.

Actually, in that case it becomes more of a generic machine-proxy tool, as VNC and SSH aren't only for a VM... so what about:
$ machineproxy <name> <IP address> <vnc session|:0> [--ssh] [--vm]

which only registers the Libvirt/virsh interaction when --vm is provided, and otherwise will just act as a proxy for a machine at a given IP address.

I was writing down a few thoughts I had.
For producing the SSH connection, I think there are a couple of gotchas.
The first is not knowing the IP address of the machine. If the VM has guest agent tools installed, we could get it from libvirt.
I've deployed this for a single VM, and other than the initial setup and potential recovery, its primary use is entering the LUKS password for encrypted disks at boot.
After setting up the VM, I installed Tailscale and accessed the system traditionally. There's a long tail of features that we'd chase trying to create a feature proxy when it's just as easy to reach for the VM once it's online.

I am not sure how many people are actually going to use ssh to toggle state of the VM either ;-). It would be friendlier to capture these by a simple http.HandleFunc("/vm/start", libvirtStart) and then use the SSH to proxy the VMs serial console, just like VNC can be exposed from libvirt directly.
This way you can handle the LUKS password from SSH

Regarding the serial console, you're right. That's what I see this project as primarily helpful for. Set-up, recovery, and booting with encrypted disks.
As for the guest, I'd install Tailscale on it, and assuming it's up, I'd access it directly.
--
Regarding Taislcale SSH with tsnet, I'll have to look into it more.

Taislcale SSH with tsnet

Didn't have a lot of time, but guess it should be possible to use the Prefs on tsnet to toggle RunSSH. Alhough at the moment the tsnet interface does not allow you to set this directly: https://github.com/tailscale/tailscale/blob/bdc7a61c24fda0b1a788ac76e9f70f81a3f370b8/tsnet/tsnet.go#L546. It should have something like reportSSH or similarly for this to work.


Hard for clients to distinguish local vs. cloud printers

When running cups-connector with local_printing: true, clients like Chrome OS see a local and cloud version of their printer (assuming they're on same network as printer). However, it's difficult for the client to determine which of these is currently selected without going to the selection menu every time. Consider the following scenario:

hplj200 is a cups printer shared by cups-connector and can printer both local and cloud.
user is almost always on same LAN as cups-connector and so normally has hplj200 local printer selected automatically and never changes it.
user travels but needs to print to hplj200 remotely. Since local hplj200 is not available, another printer like Google Docs is selected as the default after CTRL+P.
user clicks Change... button and selects cloud version of hplj200 and is able to print via GCP.
user returns to LAN so that hplj200 is local again.
user is not aware that they are now printing via hplj200 cloud as it has become the default.

A few possible ways this could be solved (some outside scope of cups-connector):

Chrome clients can become intelligent enough to prefer local printer when available, otherwise use cloud printer. This would be a GCP client issue, not cups-connector. Not sure if GCP clients can establish that local and cloud versions of a printer are the same printer.
Chrome clients should label (icon or text) the type of printer that is already selected. Maybe a cloud icon for the cloud printers and a LAN icon for local printers. This is outside the scope of cups-connector.
cups-connector appends " (local)" to all local printers.

The last option is the most straightforward though it may not be the ideal solution.
I'll work on a patch that adds optional values to append to local and cloud printer names (default to off).

You're right, lots of client changes in these ideas.

Unfortunately, when a printer is available both locally and via cloud, the policy is to favor cloud. This is because the cloud backend has all finishings available, applies them consistently, etc.
Icons are good, especially since there's local-system and local-GCP. I'll talk to PMs about it.
Currently, local is optional, cloud is required. In the future, local and cloud will both be optional. I think the " (local)" suffix is a good idea when both local and cloud are enabled. In fact, I favor no config option, just append it.

Fixed in #51


unhandled event MemberUp(Member(address = ...., status = Up)) in state AddingSelf

When starting multiple nodes at the same time I sometimes see this.

Unregistering from the member service seems to work asynchronously and sometimes too late, so we're getting member events in a state when we aren't interested any more. But that's not at all an issue.

Agreed - however currently it is logged at WARN level (sorry I see I didn't mention that in the issue).
Might be good to explicitly ignore those messages, suppressing the warning.

I know that FSM logs unhandled events at WARN ;-)
There's no way to ignore those messages, because ConstructrMachine doesn't know them. Take a look at the sources and you will see.

Well, for example https://github.com/raboof/constructr/commit/12f5a0af2e0f9c548a31a58ab066c2c99ac9cbad would work, but perhaps you don't want to add that dependency on akka-cluster there?

Exactly. constructr-machine is unaware of akka-cluster and must remain so, because it's also used for Cassandra and maybe others in the future.


update for pytorch 0.4

model introspection doesn't appear to be working for 0.4 after the Variable Tensor refactor

fixed


feat: Ionic Popover Position

Type: feat
Platform: all
Hi,
Can you tell me how to configure position of popover.
http://codepen.io/anon/pen/fymeD
Like in above codepen my trigger for popover is at footer, so i want to show popover above footer.... It gets opened but it is outside view.

fixed the issue using http://stackoverflow.com/questions/26669651/how-to-show-ionicpopover-by-default


Migrate to create-plugin

Migrates from grafana-toolkit to create-plugin
Fixes #179

What's the story with this compatibility check?

What's the story with this compatibility check?

I've opened an issue for it #188 , but it's basically never passed. Honestly we should probably either fix it or turn it off.


/mod score set... is unintuitive

Describe the bug
/mod score set requires you to type score after providing user mention. For example, you have to type:
/mod score set @mention score (type in number)
This is unusual because score is already selected as an option once, and then you have to select it again. I know this isn't a bug, but it results in the command failing if the user doesn't type it in correctly, so we should try to make it as intuitive to use as possible.
Expected behavior
If possible, the commands should be:
/mod score set @mention (type in number)
Or alternatively:
/mod score @mention set: (number)

That is basically how Discord slash commands work when you have an optional option. You have to choose that you want to specify a number by either typing score or selecting it from the menu. Actually, currently it is /mod score set user:@mention score:<number>, just Discord already opens the parameter for user because it is required. I think the easiest way to solve it would be to just make the number required too. Another way I see is creating the /mod score command and have an action option with preconfigured choices. For that way, I see something like /mod score user:@mention action:<add|set|remove> score/set:<number> being possible

Oh, another idea I just have how it might be possible to improve it is to make the command like /mod score user:@mentionOrId add:True/False value:(number that may be negative) (would imply setting it to 0 with add:False to remove it from the leaderboard, but currently it just writes a 0 in the DB) or maybe a command like /mod score args:add/set/remove @mentionOrId (number)
Those 2 comments being made I think it is a matter of decision on how we want to have it. Also I think the first step would be to decide if we want to make the score parameter required. After that there is a decision to make how we want to have the command. Here are the summerized options I see from my post 3 weeks ago and today:

/mod score <add|set|remove> user:@mention score:<number> (Keep it like it currently is, just maybe make it required)
/mod score user:@mention action:<add|set|remove> score/set:<number>
/mod score user:@mentionOrId add:True/False value:(number that may be negative) (See above regarding an adjustment that I think needs to be done if doing that)
/mod score args:add/set/remove @mentionOrId (number) (args being a single parameter, where the user would have to know/look up the syntax)
Configure the score (or all mod options) through a message that gets edited whenever you click a button below the message.

We can discuss it further either here or in Discord, if you want.

I just have another idea on how the command could look like additionally to my last post:
/mod score user:@mentionOrId score:(number as string. If it starts with '-' it'll subtract, if it starts with '+' it'll add, if you omit the '+' or '-' it'll set)
Examples:

/mod score user:@wissididom score:10 ⇒ Sets the score of the user to 10 points
/mod score user:@wissididom score:+10 ⇒ Adds 10 points to the current score of the user
/mod score user:@wissididom score:-10 ⇒ Subtract 10 points from the current score of the user

One disadvantage I see here is, that we cannot assume anymore that it is a number but needs to be checked manually before converting to number. Currently, the score option of the slash command is configured to be an integer.
But that shouldn't really be difficult. We probably just need to trim whitespaces, check startsWith and after that we should be able to use Number.isInteger and fail if it returns false and parseInt the number and do what the command is supposed to do, if it returns true.

I modified one of your suggestions which I liked the most:
/mod score user:@mention action:<add|set|remove> amount:<number>
This follows the most logical progression:
Mod > score > user > function (+/-/set) > amount
The benefit here is we don't reuse score which could cause confusion. I also like that the function and amount are beside each other because George + 5 makes more sense in a glance than + George 5. So from a flow perspective I think this is ideal.
I also am not sure if discord allows for integer inputs with an automatic check, but if it does than it could save us from building our own logic which is a minor convenience as well.

It does support integer options, yes. The amount option (that's currently called score) is an integer option. That was just a thought if we wanted to put the Action in the same field as the amount. I'll look into modifying the code in that regard later, probably today.


nita-jenkins: cli_scripts that use docker will need to be converted to use the kubectl equivalents

The bash cli_scripts that use docker will need to be converted to use the kubectl equivalents (for the K8S version)

I'll do this


isInternetReachable Property behavior

isInternetReachable  property behavior
I would like to check if the internet is working even when a device is connected to a network. For example, I connected the device in a Wifi and to simulate the no internet behavior inside the network.
I changed the network settings from DHCP to a manual wrong IP so it would not be able to access the internet and in fact by trying to access a browser doesn't work.
However, both isInternetReachable and isConnected are still returning true to the listener and when I force a NetInfo.fetch().
Is this the expected behavior or a known issue ?
I'm testing using  Android both Simulator and physical device had the same behavior.

code that I did to simulate the error:
import Netinfo from "@react-native-community/netinfo";

const test = async() => {
  const netinfo = await Netinfo.fetch();
  console.log('netinfo', netinfo);
}
const App = () => {
  useEffect(() => {
    console.log('called');
    const unsubscribe = Netinfo.addEventListener((state) => {
      console.log("Connection type", state.type);
      console.log("Is connected?", state.isConnected);
    });
  },[]);

  return (
    <>
      <StatusBar barStyle='dark-content' />
      <SafeAreaView>
        <ScrollView
          contentInsetAdjustmentBehavior='automatic'
          style={styles.scrollView}>
          <Header />
          <Button onPress={test} title="test" />

        </ScrollView>
      </SafeAreaView>
    </>
  );

};

System:
    OS: macOS 10.15
    CPU: (8) x64 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz
    Memory: 199.11 MB / 16.00 GB
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 10.8.0 - ~/.nvm/versions/node/v10.8.0/bin/node
    Yarn: 1.19.1 - ~/.yvm/shim/yarn
    npm: 6.9.0 - ~/.nvm/versions/node/v10.8.0/bin/npm
    Watchman: 4.9.0 - /usr/local/bin/watchman
  SDKs:
    iOS SDK:
      Platforms: iOS 12.2, macOS 10.14, tvOS 12.2, watchOS 5.2
    Android SDK:
      API Levels: 17, 18, 19, 21, 22, 23, 24, 25, 26, 27, 28
      Build Tools: 23.0.1, 23.0.3, 25.0.0, 25.0.1, 25.0.2, 25.0.3, 26.0.0, 26.0.1, 26.0.2, 26.0.3, 27.0.1, 27.0.3, 28.0.2, 28.0.3
      System Images: android-19 | Google APIs ARM EABI v7a, android-19 | Google APIs Intel x86 Atom, android-21 | Google APIs ARM EABI v7a, android-22 | Google APIs Intel x86 Atom, android-23 | Google APIs Intel x86 Atom, android-25 | Android Wear Intel x86 Atom, android-25 | Google APIs Intel x86 Atom, android-28 | Google APIs Intel x86 Atom
  IDEs:
    Android Studio: 3.5 AI-191.8<IP_ADDRESS>91312
    Xcode: 10.2.1/10E1001 - /usr/bin/xcodebuild
  npmPackages:
    react: 16.9.0 => 16.9.0
    react-native: 0.61.3 => 0.61.3
  npmGlobalPackages:
    react-native-cli: 2.0.1
```

On Android, we decide if we are connected by asking the Android OS directly. If it says that the device is connected to the internet, that's what we report to you. See here for the code we use:
https://github.com/react-native-community/react-native-netinfo/blob/f55c5684d2601f9075af76c075961a0a8b154729/android/src/main/java/com/reactnativecommunity/netinfo/NetworkCallbackConnectivityReceiver.java#L78-L81
What might be happening is that you've made it so the Wifi connection has no internet, so the Android system is falling back to the mobile connection. It will then report that it does have internet via that, even though you're connected to the Wifi.
Either way, this is the intended behaviour as we're only ever reporting what the Android OS is telling us.