Broken offline service workers (504) after migrating to @angular-devkit/build-angular:application from :browser

Command
build
Is this a regression?

[X] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
v17
Description
Background: Mostly vanilla angular application using PWA since Angular ~5 doing normal ng updates.
Service worker only being used for caching static assets and serving them offline

When upgrading to v18 I accepted the migration to :application - the schematic did not migrate the build.configurations.production.serviceWorker from a boolean (true)
ng serve --proxy-config proxy-dev.conf.json  --configuration production
Error: Schema validation failed with the following errors:
  Data path "/serviceWorker" must be string.
  Data path "/serviceWorker" must be equal to constant.
  Data path "/serviceWorker" must match exactly one schema in oneOf.

Through trial and error I discovered the "correct"  new value for the build.configurations.production.serviceWorker field: ngsw-config.json (aside -- FR: link the schema from the migration docs? 🙏)
The service worker appeared to fetch and serve assets as expected but just before promoting from canary to prod I did an extra QA pass (it was sus that my angular.json wasn't properly migrated) and discovered that refreshing while offline now reported a 504

The above 504 is being served from the Service Worker and it appears the handling logic isn't serving the cached copy on network error anymore.

⛔ Running my v18 build locally with ng serve --configuration production shared the same issue
✅ Reverting to v17 and it again worked as expected.
✅ Making a new v18 build with angular.json reverted to the v17 version (using :build rather than :application) works as expected when using ng serve --configuration production
⛔ Making a new v18 build with the reverted angular.json and substituting :build for :build-esbuild failed to register any service worker

This seems like a separate issue, the :application build does still register the SW and serve assets

Minimal Reproduction
I assume (but have not tested) this would affect any PWA-enabled repo upgraded from v17 to v18 which performs the migration.
I have a workaround that allows me to upgrade to v18 (using the old & busted webpack builder)
Below are the relevant files and commands I can think of:

ngsw-config.json
{
  "index": "/index.html",
  "assetGroups": [{
    "name": "app",
    "installMode": "prefetch",
    "resources": {
      "files": [
        "/index.html",
        "/*.css",
        "/*.js"
      ]
    }
  }, {
    "name": "assets",
    "installMode": "lazy",
    "updateMode": "prefetch",
    "resources": {
      "files": [
        "/assets/**"
      ],
      "urls": [
        "https://fonts.googleapis.com/**"
      ]
    }
  }],
  "navigationUrls": [
    "/**",
    "!/**/*.*",
    "!/admin/**"
  ]
}

Post-migration (bad) angular.json
{
  "$schema"<EMAIL_ADDRESS>  "version": 1,
  "newProjectRoot": "projects",
  "projects": {
    "anguldl": {
      "root": "",
      "sourceRoot": "src",
      "projectType": "application",
      "architect": {
        "build": {
          "builder": "@angular-devkit/build-angular:application",
          "options": {
            "allowedCommonJsDependencies": [
              "http-proxy-middleware",
              "pino-http"
            ],
            "outputPath": {
              "base": "dist"
            },
            "index": "src/index.html",
            "tsConfig": "src/tsconfig.app.json",
            "polyfills": [
              "src/polyfills.ts"
            ],
            "assets": [
              "src/assets",
              "src/manifest.json"
            ],
            "styles": [
              "src/styles.scss",
              "src/theme.scss"
            ],
            "stylePreprocessorOptions": {
              "includePaths": [
                "src"
              ]
            },
            "scripts": [],
            "extractLicenses": false,
            "sourceMap": true,
            "optimization": false,
            "namedChunks": true,
            "browser": "src/main.ts",
            "server": "src/main.server.ts",
            "prerender": true,
            "ssr": {
              "entry": "src/server.ts"
            }
          },
          "configurations": {
            "production": {
              "budgets": [
                {
                  "type": "anyComponentStyle",
                  "maximumWarning": "6kb"
                }
              ],
              "optimization": true,
              "outputHashing": "all",
              "sourceMap": false,
              "namedChunks": false,
              "extractLicenses": true,
              "fileReplacements": [
                {
                  "replace": "src/environments/environment.ts",
                  "with": "src/environments/environment.prod.ts"
                }
              ],
              "serviceWorker": "ngsw-config.json"
            }
          },
          "defaultConfiguration": ""
        },
        "serve": {
          "builder": "@angular-devkit/build-angular:dev-server",
          "options": {
            "buildTarget": "anguldl:build"
          },
          "configurations": {
            "production": {
              "buildTarget": "anguldl:build:production"
            }
          }
        },
        "extract-i18n": {
          "builder": "@angular-devkit/build-angular:extract-i18n",
          "options": {
            "buildTarget": "anguldl:build"
          }
        },
        "test": {
          "builder": "@angular-devkit/build-angular:karma",
          "options": {
            "main": "src/test.ts",
            "karmaConfig": "./karma.conf.js",
            "polyfills": "src/polyfills.ts",
            "tsConfig": "src/tsconfig.spec.json",
            "scripts": [],
            "styles": [
              "src/styles.scss"
            ],
            "assets": [
              "src/assets",
              "src/manifest.json"
            ]
          }
        }
      }
    },
    "anguldl-e2e": {
      "root": "",
      "sourceRoot": "",
      "projectType": "application",
      "architect": {
        "e2e": {
          "builder": "@angular-devkit/build-angular:protractor",
          "options": {
            "protractorConfig": "./protractor.conf.js",
            "devServerTarget": "anguldl:serve"
          }
        }
      }
    }
  },
  "schematics": {
    "@schematics/angular:component": {
      "prefix": "app",
      "style": "scss"
    },
    "@schematics/angular:directive": {
      "prefix": "app"
    }
  }
}

Pre-migration (good) angular.json
{
  "$schema"<EMAIL_ADDRESS>  "version": 1,
  "newProjectRoot": "projects",
  "projects": {
    "anguldl": {
      "root": "",
      "sourceRoot": "src",
      "projectType": "application",
      "architect": {
        "build": {
          "builder": "@angular-devkit/build-angular:browser",
          "options": {
            "outputPath": "dist/browser",
            "index": "src/index.html",
            "main": "src/main.ts",
            "tsConfig": "src/tsconfig.app.json",
            "polyfills": "src/polyfills.ts",
            "assets": [
              "src/assets",
              "src/manifest.json"
            ],
            "styles": [
              "src/styles.scss",
              "src/theme.scss"
            ],
            "stylePreprocessorOptions": {
              "includePaths": [
                "src"
              ]
            }
            "scripts": [],
            "vendorChunk": true,
            "extractLicenses": false,
            "buildOptimizer": false,
            "sourceMap": true,
            "optimization": false,
            "namedChunks": true
          },
          "configurations": {
            "production": {
              "budgets": [
                {
                  "type": "anyComponentStyle",
                  "maximumWarning": "6kb"
                }
              ],
              "optimization": true,
              "outputHashing": "all",
              "sourceMap": false,
              "namedChunks": false,
              "extractLicenses": true,
              "vendorChunk": false,
              "buildOptimizer": true,
              "fileReplacements": [
                {
                  "replace": "src/environments/environment.ts",
                  "with": "src/environments/environment.prod.ts"
                }
              ],
              "serviceWorker": true
            }
          },
          "defaultConfiguration": ""
        },
        "serve": {
          "builder": "@angular-devkit/build-angular:dev-server",
          "options": {
            "buildTarget": "anguldl:build"
          },
          "configurations": {
            "production": {
              "buildTarget": "anguldl:build:production"
            }
          }
        },
        "extract-i18n": {
          "builder": "@angular-devkit/build-angular:extract-i18n",
          "options": {
            "buildTarget": "anguldl:build"
          }
        },
        "test": {
          "builder": "@angular-devkit/build-angular:karma",
          "options": {
            "main": "src/test.ts",
            "karmaConfig": "./karma.conf.js",
            "polyfills": "src/polyfills.ts",
            "tsConfig": "src/tsconfig.spec.json",
            "scripts": [],
            "styles": [
              "src/styles.scss"
            ],
            "assets": [
              "src/assets",
              "src/manifest.json"
            ]
          }
        }
      }
    },
    "anguldl-e2e": {
      "root": "",
      "sourceRoot": "",
      "projectType": "application",
      "architect": {
        "e2e": {
          "builder": "@angular-devkit/build-angular:protractor",
          "options": {
            "protractorConfig": "./protractor.conf.js",
            "devServerTarget": "anguldl:serve"
          }
        }
      }
    }
  },
  "schematics": {
    "@schematics/angular:component": {
      "prefix": "app",
      "style": "scss"
    },
    "@schematics/angular:directive": {
      "prefix": "app"
    }
  }
}

Original upgrade command:
ng update @angular/core@18 @angular/cli@18 @angular/material@18 @angular/google-maps@18

which gave this prompt I agreed to:
This package has 1 optional migration that can be executed.
Optional migrations may be skipped and executed after the update process, if preferred.

 Select the migrations that you'd like to run [use-application-builder] Migrate application projects to the new
build system. (https://angular.dev/tools/cli/build-system-migration)

❯ Migrate application projects to the new build system.
  Application projects that are using the '@angular-devkit/build-angular' package's 'browser' and/or 'browser-esbuild' builders will be migrated to use the new 'application' builder.
  You can read more about this, including known issues and limitations, here: https://angular.dev/tools/cli/build-system-migration

Exception or Error
Included another screenshot in the repro section showing the network series of events

The linked line 1835 from ngsw-worker.json is from safeFetch copied below:

    async safeFetch(req) {
      try {
        return await this.scope.fetch(req);
      } catch (err) {
        this.debugger.log(err, `Driver.fetch(${req.url})`);
        return this.adapter.newResponse(null, {
          status: 504,
          statusText: "Gateway Timeout"
        });
      }
    }
  ...
  // bazel-out/k8-fastbuild-ST-2e5f3376adb5/bin/packages/service-worker/worker/main.mjs

### Your Environment

```text
Angular CLI: 18.2.0
Node: 20.12.2
Package Manager: npm 10.5.0
OS: linux x64

Angular: 18.2.0
... animations, cdk, cli, common, compiler, compiler-cli, core
... forms, google-maps, language-service, material
... platform-browser, platform-browser-dynamic, platform-server
... router, service-worker, ssr

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1802.0
@angular-devkit/build-angular   18.2.0
@angular-devkit/core            18.2.0
@angular-devkit/schematics      18.2.0
@schematics/angular             18.2.0
rxjs                            6.6.7
typescript                      5.4.5
zone.js                         0.14.10

Anything else relevant?
No response

I cannot reproduce the error related to the invalid serviceWorker option. The error message indicates that this option may be a string. Regarding the other error, it seems to originate from the same root cause as the issue outlined in https://github.com/angular/angular-cli/issues/28224.

I cannot reproduce the error related to the invalid serviceWorker option

Took a stab at reproing this.
$ ng new # version 13
$ cat angular.json | jq .projects.repro.architect.build.options.serviceWorker
true
$ ng update @angular/cli@14 @angular/core@14
...
$ ng update @angular/cli --name use-application-builder --migrate-only
...
$ cat angular.json | jq .projects.repro.architect.build.options.serviceWorker
"ngsw-config.json"

So this fresh repo got migrated correctly from a boolean to a string
Looking back at my app and comparing both pre-migration I see two differences:

the value is instead set in .build.configurations.production.serviceworker where the newly generated app sets it in .build.options.serviceWorker (I assume this is fine and unrelated)
I'm missing the option "ngswConfigPath": "ngsw-config.json"

This seems to be mostly untouched since back in 2018 when I upgraded to angular 6.0.4 (from 6.0.3) and initially added @angular/pwa 0.6.8 😅

Anyways if I (starting at a new ng 13 repo upgraded to ng14) drop the  "ngswConfigPath": "ngsw-config.json" field which was never added to my repo:
diff --git a/repro2/angular.json b/repro2/angular.json
index 0c0d691..982726b 100644
--- a/repro2/angular.json
+++ b/repro2/angular.json
@@ -31,8 +31,7 @@
               "src/styles.css"
             ],
             "scripts": [],
-            "serviceWorker": true,
-            "ngswConfigPath": "ngsw-config.json"
+            "serviceWorker": true
           },
           "configurations": {
             "production": {

Then run the migration:
$ ng update @angular/cli --name use-application-builder --migrate-only
...
$ cat angular.json | jq .projects.repro2.architect.build.options.serviceWorker
true

I see the serviceWorker value remains a boolean and is not migrated to a string

I don't know if it's worth bisecting the 8 versions of angular between now and and when I added the PWA to figure out why I don't have  "ngswConfigPath": "ngsw-config.json"
Maybe the use-application-builder schematic could warn when attempting to migrate the boolean to a string if there's no ngswConfigPath field?
Or maybe it's fine and I'm one of the lucky few people with a repo in this state :)


No rule to make target src/webfrontend/CustomDataTypeGEONAMES.coffee.js

The filename is CustomDataTypeGeoname.coffee.js but the Makefile refers to CustomDataTypeGEONAMES.coffee.js, which is then not found.

I suppose it is fixed, otherwise it wouldn't be delivered with easydb..


Fails when too many files changed

We create PRs with quite a lot of files changes sometimes (> 10,000 files). This causes this action to fail with a message such as:

reviewdog: post failed for actionlint: fail to parse diff: GET https://api.github.com/repos/pulumi/pulumi-azure-native/pulls/2553: 422 The diff could not be processed because too many files changed []

I'm not familiar with how this action works internally but might there be a way of adding an option to use a regular git checkout rather than attempting to fetch the diff via the API?

I don't believe this action does any diff fetch via an API. The expectation is that you are adding the checkout step in your workflow. This action doesn't perform it. Honestly, the best advice here is simply to not create large PRs as you would run into other errors anyway such as what we can post to GitHub PR checks.


Help me get a proper notification

Hello.
First of all, thank you for developing this tool. Even someone as me managed to configure it.
I created my config.yml to send notifications to telegram, but instead of recieveing them as you show in the readme, I only get a simple line text.
How can I configure it to send me the Snapshot and also the links to view the camera or the clip?
I commented an open issue asking to be able to directly attach the video in the telegram message. It would be awesome, but meanwhile, can you help me ?
frigate:
  server: httXXXXXXXXXXX3:5000/
  ignoressl: false
  public_url: httX.XXXXXXXXXX.ud
  headers:
  startup_check:
    attempts: 5
    interval: 30

  webapi:
    enabled: true
    interval:

  mqtt:
    enabled: false
    server:
    port:
    clientid:
    username:
    password:
    topic_prefix:

  cameras:
    exclude:

alerts:
  general:
    title: Alerta Alarma
    timeformat: Mon, 02 Jan 2006 15:04:05 MST
    nosnap:

  zones:
    unzoned: allow
    allow:
     - zone_0
     - zone_1
    block:

  labels:
    allow:
     - person
     - dog
    block:
     - bird

  discord:
    enabled: false
    webhook:
    template:

  gotify:
    enabled: false
    server:
    token:
    ignoressl:
    template:

  smtp:
    enabled: false
    server:
    port:
    tls:
    user:
    password:
    recipient:
    template:

  telegram:
    enabled: true
    chatid: 2XXXXXXXXXXXXXXXX
    token: 73XXXXXXXXXXXXXXXXXXXXXXXXXQZHiFtYI
    template: Detección en {{ .Camera }}. Parece {{ .Label }}!!

  pushover:
    enabled: false
    token:
    userkey:
    devices:
    priority:
    retry:
    expire:
    ttl:
    template:

  ntfy:
    enabled: false
    server:
    topic:
    ignoressl:
    headers:
    template:

  webhook:
    enabled: false
    server:
    ignoressl:
    headers:
    template:

monitor:
  enabled: false
  url:
  interval:
  ignoressl:

Well. I managed to get the snapshot enabling them in frigate itself. Nice.

Hi there - Thanks for your interest in this app!
So in your config, it looks like you've specified a custom template: template: Detección en {{ .Camera }}. Parece {{ .Label }}!!
In this case, that will override the default & you'll no longer get the messages as shown in the README. If you leave template blank, it will default to the template as shown in the README. If you would like to build your own message template, the docs have some information & you can find the default template used by Telegram here

Nice. Thank you :)

All I needed was to read a bit more. Sorry for the inconvenience


3D

I'm sure this is one's on the list of todos. Maybe you can enumerate the main obstacles?

Could you elaborate on some place to get more info on your GJK intersection algorithm? Is it in the docs in a placeI haven't read that yet?  I would like to lend a hand in maintaining this endeavor if I could, I think that it would bring life to the repo.

I'm actually going to close this since 3D is a different beast than 2D. It's better to focus on 2D only at the moment

Is the collision detection going to be a priority?


Moved postcss config to its own file. Updated webpack dependencies

Summary
This is a prep change to make storybook integration easier and simpler. I moved postcss config to its own file and removed it from webpack. This cleans up the webpack build and makes it slightly easier to understand.
I also noticed webpack errors during complication and it seems it was a version issue. I took the liberty to update all the dependencies as well which I noticed resolves a nasty issue we ran into with terra-consumer where CSS files are being concatenated in an indeterministic-order.. This makes using CSS custom properties more predictable when expecting :root{} declarations to override previously imported components props.

Hurray!


Radiobuttons not properly aligned

Radiobuttons and their corresponding label are not properly aligned, the button is displayed a little bit higher than its text. Is there a way to rectify this or is it an inherent behaviour of radio buttons?

Thanks for reposting. Fixed in 74a34e2e0df55b274097ff78f054859f73ab5a83.


Update sbt-scalajs, scalajs-compiler, ... to 1.13.2

About this PR
📦 Updates

org.scala-js:sbt-scalajs
org.scala-js:scalajs-compiler
org.scala-js:scalajs-library
org.scala-js:scalajs-library_2.13
org.scala-js:scalajs-test-bridge
org.scala-js:scalajs-test-bridge_2.13

from 1.12.0 to 1.13.2
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-js" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.scala-js" }
}]

labels: library-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #72.


Standard markdown links are converted into oneboxes

Perhaps only preview if the anchor is in its own paragraph

Was wondering why all links are previews 😆

Haha, this is fixed now. Can you update to the latest version?


chore: Fix small nits for ToRadix

Related issue(s)
(If it does not already exist, first create a GitHub issue that describes the problem this Pull Request (PR) solves before creating the PR and link it here.)
Resolves (link to issue)
Description
This does a small refactor of the ToRadix directive. Since both little endian and big endian rely on the number of witnesses being assigned to, to be more than the number of decomposed digits, this was added for both branches.
We also now pad the decomposed integer to the beginning or end depending on its endianness.
Summary of changes
(Describe the changes in this PR. Point out breaking changes if any.)
Dependency additions / changes
(If applicable.)
Test additions / changes
(If applicable.)
Checklist

[ ] I have tested the changes locally.
[ ] I have formatted the changes with Prettier and/or cargo fmt with default settings.
[ ] I have linked this PR to the issue(s) that it resolves.
[ ] I have reviewed the changes on GitHub, line by line.
[ ] I have ensured all changes are covered in the description.

Additional context
(If applicable.)

We modified ToRadix, so closing this


Is the output the entire model？

We use lora , is the output the whole model

Can you elaborate on what you meant by output?

I had some issues with my previous statement. What I meant was the model weights after fine-tuning. In previous projects I've worked on, they would only store the LoRA weights after fine-tuning. However, in this project, it stored all weight parameters after fine-tuning, and I would like to inquire whether it was an issue with my fine-tuning or if it was originally intended to be this way.

Hi @licy02,
In the Lora project, storing all weight parameters post-fine-tuning is intentional and differs from some previous projects. This is to provide the complete model state for flexibility and use-case coverage.
Hope this helps!


Visual Scripting broken on latest builds?

I noticed that having a visual script attached to any node, even if it doesn't contain any complicated logic, will cause the game window to close immediately after running the project without any error messages in the debugger. This must be a new issue, since i remember visual script working just fine a couple of weeks back.
Steps to reproduce:

create a new project and add a node to it
attach a visual script to it and add some logic (or none at all)
run the project

Seen on MacOS el capitan 10.11.6

Can't reproduce this anymore, so seems fixed.

Can't even try to reproduce this, because the new "Multiscript"-Entry in the latest build won't let me create a new Vscript and i can't select the "visual script" entry directly. I tried an older project with an existing visual script and I'm getting the same behavior as before (project window closing instantly)

Can't even try to reproduce this, because the new "Multiscript"-Entry in the latest build won't let me create a new Vscript and i can't select the "visual script" entry directly.

cc @karroffel

@Hinsbart I dont't think this is fixed. I just compiled a fresh built of godot master (with multiscript disabled for testing purposes) and can still reproduce this. I also haven't seen any commits lately, that would have adressed this issue. I'm appending my console output here:

running: /opt/godot-bin/godot3 -path /home/alex/Godot/vs_test -rdebug localhost:6007 -epid 16925 -p 0x0 res://Node2D.tscn
ERROR: load: No loader found for resource: res://test.vs
   At: core/io/resource_loader.cpp:213.
ERROR: poll: res://Node2D.tscn:3 - Parse Error: [ext_resource] referenced nonexistent resource at: res://test.vs
   At: scene/resources/scene_format_text.cpp:182.
ERROR: load: Condition ' err != OK ' is true. returned: RES()
   At: core/io/resource_loader.cpp:144.
ERROR: load: Failed loading resource: res://Node2D.tscn
   At: core/io/resource_loader.cpp:213.
ERROR: start: Failed loading scene: res://Node2D.tscn
   At: main/main.cpp:1460.
shit?: Connection reset by peer
ERROR: read: Server disconnected!

   At: drivers/unix/stream_peer_tcp_posix.cpp:275.

@novemberist can you test if VS was broken before the MultiScript PR as well?

@karroffel I can't test it right now, but I am pretty sure that it was, since I've been testing visual script regularly ever since i opened this issue back in march and it hasn't worked ever since.

Found this between lot of spam, trying on windows a build before multiscript
path not recognized
ERROR: No loader found for resource: res://visualscript.vs

Seems like it's still broken and not related to the MultiScript PR.

Awesome. Thank you very much.


fix ts perf regression

The FS actions themselves refer to Types.FsError which creates a recursive definition for the action types which causes a tremendous explosion in TS transform time.


Support complex for curve, image, cube and stack

This implementation only filter data with numpy.absolute in case of complex.
The toolbar do not display the raw data, but the filtered result
close #775

Functionality is there. Waiting for the CI to finish.

Tested with silx view.


RN upgrade - contact book not loading

does not work on iOS for sure
check on android as well

-Verified this issue on Staging and dev version 1.3.0(145)


Leaving a room does not work properly

Well, i joined the riot-room, was reading there for quite some time and left afterwards. It automatically adds me again all the time. I keep leaving, i left some messages about that too, before leaving.
On riot android the room does not show up anymore, on riot web it did. After leaving it again and deleting it in the archive it seems to not show up anymore.
And the problem now is - i still get e-mail messages about that room. How can i erase that room from my history completely?
If i'm wrong here, please point me to the correct direction.

@bmarty I'm not sure, except if this happened when @natrius was trying to preview the left room.


[Feature request] return image when QR code is read

I would like to store the image in which the QR code was recognized.  I'm looking for suggestions about how to participate properly in getting such an enhancement approved and implemented.  I'm willing to work on it.

Looking at other's comments, I see I can simply take a photo when the QR code is recognized.  Will pursue that.

How did it work? I am trying to do the same but it was a mess as i have to dispose the controller and initialize the camera controller and still somehow the image is very very dim. Did you make it work?

I did not pursue this to implementation.  Knowing the presumed solution, I went on to other things.  I still hope to store the image with the QR code in it and will either figure out how to get all the controllers aligned or perhaps dust off my Objective C or whatever and dig into the component.  Sorry I can't be of more help at this point.


publish release artifact containing only proving keys

a published artefact containing only the proving keys would simplify development and building and dependency navigation for any software that wants to interact with penumbra
yes, i know these aren't supposed to change. they have changed several times since the first day i was told this
currently, anyone using the wasm package runs a script that ad-hoc downloads the keys by manipulating URIs based on parameters such as git tag and a manually maintained mapping of key names
and it's not clear how to correctly/reliably validate the output of that download script. a provided checksum or other endorsed validation method would be nice
also, it makes sense to simply publish these large output files through the infrastructure that github provides for publishing large output files

currently, anyone using the wasm package runs a script that ad-hoc downloads the keys by manipulating URIs based on parameters such as git tag and a manually maintained mapping of key names

For posterity, this appears to be the script: https://github.com/penumbra-zone/web/blob/71b71e82bb02a4aa1f8bed7ebf8a92d5757d251e/apps/extension/src/utils/download-proving-keys.ts

yes, i know these aren't supposed to change. they have changed several times since the first day i was told this

Oh, I feel that! You're right, we've been optimistic about not making breaking changes. The web workflows in particular bear the brunt of that. At the risk of sounding out of touch, I suggest that we don't expect these changes to happen frequently, and after mainnet, I think it's safe to say they'll be stable. So, the churn is mostly 11th hour development alignment. Not dismissing the concern, just trying to describe my perspective so we can talk pragmatically about requirements.

also, it makes sense to simply publish these large output files through the infrastructure that github provides for publishing large output files

GH already automatically posts a tarball of the source code for each tagged release, but unfortunately it only bundles lfs pointers by default, which is rather silly. Based on the docs, I've enabled opt-in inclusion of lfs files for source archives, so on the next tagged release, there'll be in there.
It might be more straightforward to download the tarball and copy specific files from that—however, given that the script you've got is already written, it feels "good enough" to me for right now.

and it's not clear how to correctly/reliably validate the output of that download script. a provided checksum or other endorsed validation method would be nice

On this count, downloading the tarball with included lfs files provides a better provenance story than pulling "raw" objects from the gh api. However, what you're pointing to is a general lack of a signing authority for verifying code integrity. These artifacts are built in CI, so publishing checksums created from CI alongside them doesn't provide any meaningful security. For paranoid folks, we assume they'll build from source—but that effectively means trusting the clone of the repo they got in the first place.
In the context of assurance, what we care about is that the ones and zeroes are not tampered with from developer workstation -> monorepo -> web repo -> web store -> end user devices. There are a few places we can try to improve integrity, as part of a broader effort for a pre-launch security review.

part the validation concern is not just cryptographic provenance, but just a very practical "do i have the correct keys, and are they whole" as we change between branches, restore from caches, and bundle files during development

We discussed this at backlog grooming this week, and consensus was, unsurprisingly: we're totally for-real about to stabilize the keys. :innocent: There's a very real forcing function here: at conclusion of phase 2 of the summoning ceremony, we'll have "final" key material that we cannot change without rerunning the ceremony, which we definitely don't want to do. When the time comes to update those keys one last time—probably a few weeks from now—feel free to tag me in on review. I'm happy to perform a tediously manual review to triple-check we've got the right key material where we expect it to be.

we're totally for-real about to stabilize the keys.

The more things change.... Turns out we changed the keys again as part of https://github.com/penumbra-zone/penumbra/pull/4239. Just goes to show that @turbocrime was indeed right in being skeptical about not having to do the manual key-ferrying again. Plans still in the works about rerunning the summoning ceremony phase 2, so for now, we're still committed to updating the keys in the web code via scripts. There is good news at least for more checksums and such in #4195. I hope to get that together in time for the next published release (i.e. >0.72.0).


Export type json not working for some users

Recently, I've noticed one strange thing with Sketch APIs. I am trying to export svg, png, pdf and json of selected layers. This code is working perfect and exporting all the file types for my device.
Code we're using
const exportSelections = () => {
  const exportFormats = [
    'png',
    'svg',
    'json',
    'pdf'
  ]

  const document = sketch.getSelectedDocument()
  const selection = document.selectedLayers

  const fileManager = NSFileManager.defaultManager();
  const pathId = NSCachesDirectory;
  const paths = fileManager.URLsForDirectory_inDomains(pathId, NSUserDomainMask);

  // /Users/apple/Library/Caches/com.iconscout.icondrop.sketch
  const filePathBase = `${paths.objectAtIndex(0).path()}/sketch-temp`;

  const files = selection.map(layer => {
    sketch.export(layer, {
      formats: exportFormats,
      output: filePathBase,
      overwriting: true
    })

    return exportFormats.map(f => `${filePathBase}/${layer.name}.${f}`)
  })

  console.log('Exported Files:', files)

  return files
}

But on the device of one of our designer, it's not exporting json (not throwing errors). Also, there's no proper response of the export method using which I can detect that the selection is exported for json.
Then I copied that particular .sketch file to my device and it worked to export json format. Any help on this?

That's not really an skpm issue, but the json export has been added in Sketch 53 IIRC so on older version of Sketch, it won't work


Failing jars

Hi,
It did find the right jars, nice job. I used 1.2.0.
It failed to process these jars in some Mx projects I have access to, probably incompatible manifest formats?
12:58:14.800 getJarProps ▶ WARN 002 Failed to parse activation-1.1.1.jar
12:58:14.806 getJarProps ▶ WARN 003 Failed to parse activation-1.1.jar
12:58:14.920 getJarProps ▶ WARN 004 Failed to parse checker-qual-2.5.2.jar
12:58:14.957 getJarProps ▶ WARN 005 Failed to parse connect-json-2.4.0.jar
12:58:15.141 getJarProps ▶ WARN 006 Failed to parse jldap-4.3.jar
12:58:15.174 getJarProps ▶ WARN 007 Failed to parse kafka-clients-2.4.0.jar
12:58:15.178 getJarProps ▶ WARN 008 Failed to parse kafka-log4j-appender-2.4.0.jar
12:58:15.182 getJarProps ▶ WARN 009 Failed to parse kafka-streams-2.4.0.jar

13:16:33.996 getJarProps ▶ WARN 002 Failed to parse junit-4.11.jar
13:16:34.006 getJarProps ▶ WARN 003 Failed to parse nekohtml.jar
13:16:34.194 getJarProps ▶ WARN 004 Failed to parse xml-apis-1.4.01.jar

13:17:44.179 getJarProps ▶ WARN 002 Failed to parse activation-1.1.1.jar
13:17:44.270 getJarProps ▶ WARN 003 Failed to parse checker-qual-2.5.2.jar
13:17:44.311 getJarProps ▶ WARN 004 Failed to parse commons-httpclient-3.1.jar
13:17:44.542 getJarProps ▶ WARN 005 Failed to parse not-yet-commons-ssl-0.3.17.jar
13:17:44.686 getJarProps ▶ WARN 006 Failed to parse value-2.8.2.jar
13:17:44.707 getJarProps ▶ WARN 007 Failed to parse xercesImpl-2.12.0.jar
13:17:44.713 getJarProps ▶ WARN 008 Failed to parse xml-apis.jar

13:18:20.265 getJarProps ▶ WARN 002 Failed to parse activation-1.1.1.jar
13:18:20.355 getJarProps ▶ WARN 003 Failed to parse checker-qual-2.5.2.jar
13:18:20.444 getJarProps ▶ WARN 004 Failed to parse commons-httpclient-3.1.jar
13:18:20.643 getJarProps ▶ WARN 005 Failed to parse org.apache.poi.poi-4.1.1.jar
13:18:20.650 getJarProps ▶ WARN 006 Failed to parse org.apache.poi.poi-ooxml-4.1.1.jar
13:18:20.665 getJarProps ▶ WARN 007 Failed to parse org.apache.poi.poi-ooxml-schemas-4.1.1.jar
13:18:20.678 getJarProps ▶ WARN 008 Failed to parse org.apache.xmlbeans.xmlbeans-3.1.0.jar

13:26:58.495 getJarProps ▶ WARN 002 Failed to parse checker-qual-2.5.2.jar
13:26:58.669 getJarProps ▶ WARN 003 Failed to parse commons-httpclient-3.1.jar
13:26:59.069 getJarProps ▶ WARN 004 Failed to parse jsch-0.1.55.jar
13:26:59.219 getJarProps ▶ WARN 005 Failed to parse junit-4.11.jar
13:26:59.239 getJarProps ▶ WARN 006 Failed to parse not-yet-commons-ssl-0.3.17.jar
13:26:59.533 getJarProps ▶ WARN 007 Failed to parse org.apache.poi.poi-4.1.1.jar
13:26:59.553 getJarProps ▶ WARN 008 Failed to parse org.apache.poi.poi-ooxml-4.1.1.jar
13:26:59.578 getJarProps ▶ WARN 009 Failed to parse org.apache.poi.poi-ooxml-schemas-4.1.1.jar
13:26:59.600 getJarProps ▶ WARN 00a Failed to parse org.apache.xmlbeans.xmlbeans-3.1.0.jar
13:26:59.697 getJarProps ▶ WARN 00b Failed to parse replication.jar
13:26:59.879 getJarProps ▶ WARN 00c Failed to parse xercesImpl-2.12.0.jar
13:26:59.900 getJarProps ▶ WARN 00d Failed to parse xml-apis.jar

Did a random selection of the failed jars. Most of them are I believe too old. Their MANIFEST.MF files contain no useful metadata at all. To resolve that it would require parsing the class files and file names. For now, these are unsupported.
Newer versions of these jars do contain proper metadata. So the issue will resolve naturally.


fix doc fishr display

display of latex equations in doc fishr is not correct

Hi, Lisa, I think your documentation is really high quality, but I have not the time budget yet to look at the error, this would be less prioritized then.

@agisga , do you know?

No, sorry. What latex renderer is used here? Maybe it's restricted to only a subset of latex commands, since some of the simpler equations seem to be displayed fine..

@agisga , do you know?

Not sure if it will fix the entire issue, but there are some $$ symbols missing, for instance:

" §§\theta$$ denote "
"$$\mathcal{R}_e(\theta) be the ERM"

No problem if you have no time, it's not a priority.
I already fixed the issue you mention here.

Not sure if it will fix the entire issue, but there are some $$ symbols missing, for instance:

" §§\theta$$ denote "
"$$\mathcal{R}_e(\theta) be the ERM"

I am just talking about the display equations (like in line 25-27 in the md file)

I am just talking about the display equations (like in line 25-27 in the md file)

I'm not sure what latex renderer is being used here, but with the most common latex interpreters, if a dollar sign is missing anywhere in the document, then it can mess up the formatting everywhere in the rest of the document (like if you open an $$ in some line but never close it, then it may be assumed that the whole rest of the document is one long equation)... I think it's like with any type of programming, a bug in one line can lead to an error in an entirely different line. So, in my opinion, I would strongly suggest that to fix the syntax everywhere, not just lines 25-27...

I am just talking about the display equations (like in line 25-27 in the md file)

I'm not sure what latex renderer is being used here, but with the most common latex interpreters, if a dollar sign is missing anywhere in the document, then it can mess up the formatting everywhere in the rest of the document (like if you open an $$ in some line but never close it, then it may be assumed that the whole rest of the document is one long equation)... I think it's like with any type of programming, a bug in one line can lead to an error in an entirely different line. So, in my opinion, I would strongly suggest to fix the syntax everywhere, not just lines 25-27...

Alright I understand, thank you very much! @agisga

Codecov Report

Merging #372 (6f0bd4c) into master (2331d1d) will increase coverage by 0.04%.
Report is 75 commits behind head on master.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #372      +/-   ##
==========================================
+ Coverage   95.22%   95.27%   +0.04%
==========================================
  Files         129      129
  Lines        4464     4613     +149
==========================================
+ Hits         4251     4395     +144
- Misses        213      218       +5

Flag
Coverage Δ

unittests
95.27% <ø> (+0.04%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed
Coverage Δ

domainlab/models/model_diva.py
97.61% <ø> (ø)

domainlab/utils/hyperparameter_gridsearch.py
100.00% <ø> (ø)

... and 1 file with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

display is now fixed: https://github.com/marrlab/DomainLab/blob/lb_fix_doc_fishr/docs/docFishr.md


Could you give me some advise, how to set resize height to 64 or 128 or some else.

Thank you for your help and this project.
I think with bigger image, could get better result. So, I try to train with bigger image such as resize height=64 or 128 but not 32. But I haven't found how to do this, I didn't find how to change cnn and buckets.
Could you give me some advise, or recommend some thesises?
Looking forward to get your help. Thank you.

You have to change the source code in a few places:

model.py, line 95: you should change the shape to (None, 1, 64, None) for img_data
In cnn.py, you should add another max_2x1pool() layer. This is because the RNN expects a single-row output from the CNN, and the max-pooling makes the input height 32 times smaller by default.
For an input of height 128, add another max_2x1pool() layer and change the shape to (None, 1, 128, None).

Thank you!!!
It works, and the result increased by 3% with height=64.

@jstzsjw  glad to hearing your story.
@Fred-Erik another way is to change the net to process rows more than one in rnn step. just like @da03  image2latex project.  run rnn every rows will add more infos


Introduce a slab-based memory allocator and optimize SHAMapItem

Introduce support for a slabbed allocator:
When instantiating a large amount of fixed-sized objects on the heap the overhead that dynamic memory allocation APIs impose will quickly become significant.
In some cases, allocating a large amount of memory at once and using a slabbing allocator to carve the large block into fixed-sized units that are used to service requests for memory out will help to reduce memory fragmentation significantly and, potentially, improve overall performance.
This commit introduces a new SlabAllocator<> class that wraps around the complexity and exposes an API that is similar to the C++ concept of an Allocator.
This class is not a general-purpose allocator and should not be used unless profiling and analysis of specific memory allocation patterns indicates that the additional complexity introduced will improve the performance of the system overall and subsequenting profiling proves that.
Optimize SHAMapItem and leverage new slab allocator:
The SHAMapItem class contains a variable-sized buffer that holds the serialized data associated with a particular item inside a SHAMap.
Prior to this commit, the buffer for the serialized data was allocated separately. Coupled with the fact that most instances of SHAMapItem were wrapped around a std::shared_ptr meant that an instantiation might result in up to three separate memory allocations.
This commit switches away from std::shared_ptr for SHAMapItem and uses boost::intrusive_ptr instead, allowing the reference count for an instance to live inside the instance itself. Coupled
with using a slab-based allocator to optimize memory allocation for the most commonly sized buffers, the net result is significant memory savings. In testing, the reduction in memory usage hovers between 400MB and 650MB. Other scenarios might result in larger savings.

High Level Overview of Change

Context of Change

Type of Change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Refactor (non-breaking change that only restructures code)
[ ] Tests (You added tests for code that already exists, or your new feature included in this PR)
[ ] Documentation Updates
[ ] Release

I thought I saw a fairly recent push to this pull request.  So I tried building it.  The previously observed build failures on macOS are still present.

@scottschurr this is Nik's branch with my my additional changes which should build https://github.com/greg7mdp/rippled/tree/slab_alloc

@greg7mdp I think you should open a new PR with all of the changes, and @nbougalis should close this one. What do you think, @HowardHinnant & @scottschurr ?

I have a slight preference for keeping this one open so I can remember what I said 19 days ago.  But I won't object if everyone wants to move to a new PR.

@mtrippled I also think it would make sense to create a new PR. I suggested it a few days back, but @nbougalis said he would take care of it, probably he'd like to review the branch first.
@scottschurr I just verified that this branch builds fine on mac (with some unrelated warnings).

@greg7mdp, I'm fine either way, either updating this pull request or creating a new one.

And, @greg7mdp, I tried building your branch locally.  Yup, your version works way better on macOS than the previous version did.  Thanks!  I think I will wait until your version is in an official pull request, however, before I start a full fledged review of the revised code.

Thanks @scottschurr for trying the mac version. I did build my branch on all 3 platforms (linux, windows, mac).
@nbougalis , we should either update that PR or start a new one, so the review can proceed.

I've addressed the concerns and issues highlighted by reviewers. I have also incorporated some of the ideas by @greg7mdp into SlabAllocator, and refined the memory allocation engine.
I believe that this is ready for another review and, provided it's signed-off, a good candidate for merging.
I expect a reduction in overall memory usage of at least 15% on mainnet servers, but it depends on a number of factors. On my test bed, the reduction is closer to 22%. Performance should also improve.
Thanks.

rippled/src/test/shamap/SHAMapSync_test.cpp:101:50: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
                SHAMapNodeType::tnACCOUNT_STATE, std::move(makeRandomAS()));
                                                 ^
/Users/howardhinnant/Development/rippled/src/test/shamap/SHAMapSync_test.cpp:101:50: note: remove std::move call here
                SHAMapNodeType::tnACCOUNT_STATE, std::move(makeRandomAS()));
                                                 ^~~~~~~~~~              ~
1 warning generated.

@nbougalis please review Howard's suggestions above, at your convenience.

@scottschurr can you confirm whether your concerns have been resolved?

@intelliot, no, I can't.  I have another higher priority code review I'm working on.  I can't provide an ETA for when I'll get to this one.

I checked with @scottschurr and we agreed that he has no particular need to review this PR, and we think it would be best to assign a different reviewer in order to move this forward. scottschurr is currently focused on other reviews that have higher priority.
I'll find another reviewer shortly. If that other reviewer approves, then this PR can be merged.

Both @HowardHinnant and I have approved this already. How many reviewers does this need?

@drlongle just FYI since this was discussed with Mark today. This PR isn't blocking on your review, but please feel free to have a look and share any questions/comments.

This has a unit test failure that’s unrelated to these changes and likely traces back to the recent “simplified fee” proposal by @ximinez.

I merged develop and applied clang-format.patch. If acceptable to you, @nbougalis , this PR will be squashed when it is merged, so it's OK to leave the merge commit for the moment. Here is the commit message I suggest; @nbougalis please confirm that this looks good (or suggest an alternative).
Optimize SHAMapItem with slabbed allocator: (#4218)

* Introduce support for a slab-based memory allocator:

When instantiating a large amount of fixed-sized objects on the heap the
overhead that dynamic memory allocation APIs impose will quickly become
significant.

In some cases, allocating a large amount of memory at once and using a
slabbing allocator to carve the large block into fixed-sized units that
are used to service requests for memory out will help to reduce memory
fragmentation significantly and, potentially, improve overall
performance.

The new `SlabAllocator<>` class exposes an API that is _similar_ to the
C++ concept of an `Allocator` but it is not meant to be a
general-purpose allocator.

It should not be used unless profiling and analysis of specific memory
allocation patterns indicates that the additional complexity introduced
will improve the performance of the system overall, and subsequent
profiling proves it.

A helper class, `SlabAllocatorSet<>` simplifies handling of variably
sized objects that benefit from slab allocations.

This commit incorporates improvements suggested by Greg Popovitch
(@greg7mdp).

* Optimize `SHAMapItem` and leverage new slab allocator:

The `SHAMapItem` class contains a variable-sized buffer that holds the
serialized data associated with a particular item inside a `SHAMap`.

Previously, the buffer for the serialized data was allocated separately.
The fact that most instances of `SHAMapItem` were wrapped around a
`std::shared_ptr` meant that an instantiation might result in up to
three separate memory allocations.

This commit switches away from `std::shared_ptr` for `SHAMapItem` and
uses `boost::intrusive_ptr` instead, allowing the reference count for an
instance to live inside the instance itself. Coupled with using a
slab-based allocator to optimize memory allocation for the most commonly
sized buffers, the net result is significant memory savings. In testing,
the reduction in memory usage hovers between 400MB and 650MB. Other
scenarios might result in larger savings.

In performance testing with NFTs, this commit reduces memory size by
about 15% sustained over long duration.

* Avoid using std::shared_ptr when not necessary:

The `Ledger` class contains two `SHAMap` instances: the state and
transaction maps. Previously, the maps were dynamically allocated using
`std::make_shared` despite the fact that they did not require lifetime
management separate from the lifetime of the `Ledger` instance to which
they belong.

The two `SHAMap` instances are now regular member variables. Some smart
pointers and dynamic memory allocation was avoided by using stack-based
alternatives.

Squashing these three together feels a bit like going mixing the appetizer, the main course and the dessert together on one plate.
I think that the three commits are functionally separate, and each is self-contained. IMO, keeping them that way is cleaner, more professional and makes it easier to review and, if needed, run git bisect in the future.

We had a comparable situation with https://github.com/XRPLF/rippled/pull/4192: the PR author recommended keeping the commits as-is, and so we did. Later, we learned there was some discontent with that, and that we should have squashed all the commits together.
I don't personally have any opinion either way, but if we're going to recommend merging this as three separate commits, then we should at least confirm that each commit stands on its own and passes unit tests & clang-format. It may make sense to open 2-3 new PRs for this purpose.

What we decided in #4192 is that if reviewers feel that a commit should be merged, and the commit message does not begin with [FOLD], then the reviewer should recommend a change to the commit message.  And vice-versa.

blocked: let's get some agreement on whether this PR should end up as 3 commits, or 1.
If it's 3 commits, I think it would be best to open a couple of separate PRs the individual parts, so that we can confirm they are mergable on their own (they make sense and pass unit tests & clang-format).

@seelabs confirmed 3 commits for this is fine. I am rebasing the commits individually to ensure they pass CI on GitHub. If they pass, I will merge them no earlier than 2023-04-11 at 8:00 AM PT. If you have any objections, please comment here on this PR.

will merge after github checks pass, if I don't hear any objections

@manojsdoshi to try to corroborate the 15-22% memory use reduction


Improve diagnostics script

[!warning]
This is a public repository, ensure not to disclose:

[ ] personal data beyond what is necessary for interacting with this pull request, nor
[ ] business confidential information, such as customer names.

What kind of PR is this?
Required: Mark one of the following that is applicable:

[ ] kind/feature
[x] kind/improvement
[ ] kind/deprecation
[ ] kind/documentation
[ ] kind/clean-up
[ ] kind/bug
[ ] kind/other

Optional: Mark one or more of the following that are applicable:

[!important]
Breaking changes should be marked kind/admin-change or kind/dev-change depending on type
Critical security fixes should be marked with kind/security

[ ] kind/admin-change
[ ] kind/dev-change
[ ] kind/security
[ ] kind/adr

What does this PR do / why do we need this PR?

Fixes #2116

Information to reviewers

Checklist

[ ] Proper commit message prefix on all commits

Change checks:

[ ] The change is transparent
[ ] The change is disruptive
[ ] The change requires no migration steps
[ ] The change requires migration steps
[ ] The change upgrades CRDs
[ ] The change updates the config and the schema

Metrics checks:

[ ] The metrics are still exposed and present in Grafana after the change
[ ] The metrics names didn't change (Grafana dashboards and Prometheus alerts are not affected)
[ ] The metrics names did change (Grafana dashboards and Prometheus alerts were fixed)

Logs checks:

[ ] The logs do not show any errors after the change

Pod Security Policy checks:

[ ] Any changed pod is covered by Pod Security Admission
[ ] Any changed pod is covered by Gatekeeper Pod Security Policies
[ ] The change does not cause any pods to be blocked by Pod Security Admission or Policies

Network Policy checks:

[ ] Any changed pod is covered by Network Policies
[ ] The change does not cause any dropped packets in the NetworkPolicy Dashboard

Audit checks:

[ ] The change does not cause any unnecessary Kubernetes audit events
[ ] The change requires changes to Kubernetes audit policy

Falco checks:

[ ] The change does not cause any alerts to be generated by Falco

Bug checks:

[ ] The bug fix is covered by regression tests

lgtm
nit: do we want to add easily searchable tags to the namespace function also like i did?, e.g.  etc so its easier to ctrl f to each section?

I will add that (I thought i did)


feat(pip_repository): Enable PyPi dep cycles

Mirror of the upstream PR https://github.com/bazelbuild/rules_python/pull/1166

Don't need this; manually merged to runahead.


multinode server2 simulation: introduce attenuation

Currently we don't use any attenuation in the simulation, however we should
introduce:

a QuickCheck generator for BearerInfo (and Script BearerInfo);
test its distribution;
plug it into prop_multinode_Sim and possibly other simulations

@bolt12 is this all addressed by #3281?

Yes


FIX: tookit > toolkit

Page title also should be corrected.

fixed misspelling (tookit -> toolkit)


fix make distcheck problem in dap4_test (but turning off nczarr_test/run_scalar.sh due to issue)

Fixes #2544
Fixes #2547
Had to turn off nczarr_test/run_scalar.sh
Also turn on distcheck target in CI build.
See #2545

When I run this branch through my Jenkins server it builds well for all supported versions of HDF5, with and without zstd, and with and without szip. So that's good!
Parallel testing next...

OK, has this all been fixed in other ways?
SHould I just close this PR?

@edwardhartnett I believe it can be closed, thanks!


(PDB-2071) Add PuppetDBTerminusState class

In order to support high availability, the PuppetDB terminus needs
to implement sticky failover. That is, after failing over to another
PuppetDB instance, it should keep using that instance for future
requests as long as possible. This is tricky when running in
PuppetServer, since it hosts JRuby instances.
This adds the PuppetDBTerminusState java class as a place for the
PuppetDB terminus to put the data it needs to share between all
JRubies in the process.

 I don't suppose we have any thoughts about making this kind of thing more generic and less pdb-specific?

@kbarber Yeah, I know what you mean. I'm of two minds here; while it would be nice to make this more generic (a map in an atom?), I also don't know of anybody else who needs such a thing.

@cprice404 : we talked about this in standup and agreed (I think) that this is ok given that we're going to a Clojure terminus at some point. @kbarber are we good here?

@mullr OK; we probably aren't going to be able to do anything with this for a little while, we're blocking Ankeny right now and we don't have a Burnside branch yet.  Will review ASAP.

Sorry it's taken so long to get to this.
I have a handful of comments, but I don't know that any of them are things I'd consider mandatory for this PR.  I probably have more comments for the terminus-side code, but I'll put those on that PR/commit.

I kind of do like the idea of trying to make this more generic, and maybe have just a TerminusContext class or something.  It could be a map whose keys map to individual terminii, and then they reserve the right to decide what the values are and use them as they see fit.
This may be a little over-protective, but I have an instinct to be really careful about muddying the waters between languages whenever we have to do cross-language interop stuff like this.  For that reason, we've tried pretty hard to make sure we do not introduce any dependencies on Clojure in the JRuby code.  This PR would break that paradigm, and it's not clear to me that it's necessary.  It seems like a Java AtomicInteger would be sufficient here, and that there is no big advantage to introducing a dependency on the Clojure runtime to the Ruby side.
For all of the patterns where we've done this sort of thing in the past, we inject things into the JRuby world from the Clojure side, rather than reaching out from the Ruby side into the JVM.  I think that would be doable and desirable here as well.  Relevant code (there are a couple of different approaches in play in these examples):

https://github.com/puppetlabs/puppet-server/blob/573e915979dc0ade9cc82cc50d6987410954f40f/src/clj/puppetlabs/services/jruby/jruby_puppet_internal.clj#L207-L212
https://github.com/puppetlabs/puppet-server/blob/573e915979dc0ade9cc82cc50d6987410954f40f/src/ruby/puppet-server-lib/puppet/server/master.rb#L28-L29
And perhaps the most relevant:
https://github.com/puppetlabs/puppet-server/blob/573e915979dc0ade9cc82cc50d6987410954f40f/src/ruby/puppet-server-lib/puppet/server/config.rb#L29
If you did something like this last example, to override the default class used to store your state, then you could have a default class implemented in pure ruby in your terminus, and puppet server would override it for the case where you are being run under puppet server.  I think that would clean up a lot of that stuff you're currently doing with NameError, and also result in a much less tangled dependency graph between the various functional units of code.
I'll try to add some comments to the terminus pr to clarify, and happy to discuss in person if that would help (though I'm out all of next week).

@mullr going through our PR backlog... any new thoughts on this?  Is it blocking you?
I think I added some comments on the terminus PR as well.
Would it be easier if we set up some time to round up a few people and discuss?

@cprice404 ack, sorry, this is a simple case of neglect. I'll dive back into this later today or early tomorrow.

I kind of do like the idea of trying to make this more generic, and maybe have just a TerminusContext class or something. It could be a map whose keys map to individual terminii, and then they reserve the right to decide what the values are and use them as they see fit.

I thought about that. The tricky bit is deciding what kind of map to use, and which language's semantics it needs to obey. I'm sure we can answer this question, but I didn't want to go into it just for this. I would totally understand if that's what's needed though, and am willing to rewrite to make it so.

This may be a little over-protective, but I have an instinct to be really careful about muddying the waters between languages whenever we have to do cross-language interop stuff like this. For that reason, we've tried pretty hard to make sure we do not introduce any dependencies on Clojure in the JRuby code. This PR would break that paradigm, and it's not clear to me that it's necessary. It seems like a Java AtomicInteger would be sufficient here, and that there is no big advantage to introducing a dependency on the Clojure runtime to the Ruby side.

Totally agree on the dependency direction smell. I took this approach because I wanted to avoid bleeding the semantic concerns of the pdb terminus into puppet server. That is, the thing PuppetDB really wants in order to implement this functionality is synchronized place to store some chunk of data. That is exactly a Clojure atom.
It is true that the data is an Integer, so it could be implemented with AtomicInteger. But my feeling is that spreading this implementation detail across two code bases is more harmful than introducing a terminus -> clojure runtime dependency. (the dependency is soft, fwiw; it'll store the state in the ruby vm if it can't find the java stuff)

Relevant code (there are a couple of different approaches in play in these examples):

The big difference between these samples and the requirement here is that we need a storage location at the PuppetServer level. The static implementation is really a hack; it would be better to have an atom whose lifetime follows that of PuppetServer (getting cleared on HUP) and is passed in to the terminus. The http client class business is kind of like that, but it appears to be stored in a static location rather than with the app itself.

@cprice404 ⬆

Totally agree on the dependency direction smell. I took this approach because I wanted to avoid bleeding the semantic concerns of the pdb terminus into puppet server. That is, the thing PuppetDB really wants in order to implement this functionality is synchronized place to store some chunk of data. That is exactly a Clojure atom.

But there are other classes in java.util.concurrent that would satisfy your requirements without introducing the JRuby->Clojure dependency, right?

@mullr after our meatspace convo the other day, would it make sense for us to go ahead and close this out and wait for your follow-up PR?
Also please let us know if you need a hand with the other PR, I know it kind of gets into the gross Ruby guts of Puppet Server, and wasn't trying to totally punt that over to you :)

@cprice404 yep, closing this. Will submit a new PR.

@mullr thanks!


MSD - missing job type for Campaign and Data System job

ENVIRONMENT: production
URL: https://scrapers.peviitor.ro/
Browser: Microsoft Edge
Device: Laptop
OS: Windows 10
STEPS TO REPRODUCE:

Open the URL   https://jobs.msd.com/gb/en/job/R261532  in the browser
Look for job type

EXPECTED RESULTS:
A job type should appear
ACTUAL RESULTS:
No job type appears

the job type is not present in the job description, and the job type is not required if it is not present


chore(deps): update actions/checkout action to v4.1.7

This PR contains the following updates:

Package
Type
Update
Change

actions/checkout
action
patch
v4.1.6 -> v4.1.7

Release Notes

actions/checkout (actions/checkout)
v4.1.7
Compare Source

Bump the minor-npm-dependencies group across 1 directory with 4 updates by @dependabot in https://github.com/actions/checkout/pull/1739
Bump actions/checkout from 3 to 4 by @dependabot in https://github.com/actions/checkout/pull/1697
Check out other refs/* by commit by @orhantoy in https://github.com/actions/checkout/pull/1774
Pin actions/checkout's own workflows to a known, good, stable version. by @jww3 in https://github.com/actions/checkout/pull/1776

Configuration
📅 Schedule: Branch creation - "every weekday after 4am" (UTC), Automerge - At any time (no schedule defined).
🚦 Automerge: Enabled.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

Edited/Blocked Notification
Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.
You can manually request rebase by checking the rebase/retry box above.
⚠ Warning: custom changes will be lost.

 You will get a PR once a newer version is released. To ignore this dependency forever, add it to the ignoreDeps array of your Renovate config.
If you accidentally closed this PR, or if you changed your mind: rename this PR to get a fresh replacement PR.


COR-445-Colombia Seeder: fix seeding error

Our PR wasn't exhaustive enough and an error slipped through.

/lgtm


MemoryBlock::init is unsound

https://github.com/TimDiekmann/alloc-wg/blob/75c657273303773fd7383a1fc3fb21a331d3fad6/src/alloc.rs#L27-L44
Since the ptr field is public, you could put a bogus pointer in there, or even a pointer to an actual value, and then overwrite it with zeros or undefined bytes without using unsafe.
MemoryBlock {
    ptr: NonNull::new(ptr::null_mut()).unwrap(),
    size: usize::max(),
}.init(AllocInit::Zeroed);

As currently alloc_wg::alloc is the same as core::alloc, this crates now uses core:alloc instead, where this bug is fixed. Thanks for reporting!


Add metadata format name

Reason for this pull request
Fractional cover application depends on metadata storage format name in the source product definition to match with that of product config.
Once test database is setup and nbart products are ingested, any attempt to apply fractional cover on ingested nbart products would raise source and new product definition document mismatch.
2019-05-25 00:35:38,187 31138 datacube.ui.click DEBUG Connected to datacube index: Index<db=PostgresDb<engine=Engine(postgresql://sm9911@agdcdev-db.nci.org.au:6432/nci_envtest_db)>>
2019-05-25 00:35:38,636 31138 /g/data/v10/public/modules/dea/20190524/lib/python3.6/site-packages/fc/fc_app.py INFO Add the output product definition for ls8_fc_albers in the database.
Traceback (most recent call last):
  File "/g/data/v10/public/modules/dea/20190524/bin/datacube-fc", line 11, in <module>
    load_entry_point('fc==1.2.9+46.g89d6fce', 'console_scripts', 'datacube-fc')()
  File "/g/data/v10/public/modules/dea-env/20190522/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/g/data/v10/public/modules/dea-env/20190522/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/g/data/v10/public/modules/dea-env/20190522/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/g/data/v10/public/modules/dea-env/20190522/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/g/data/v10/public/modules/dea-env/20190522/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/g/data/v10/public/modules/dea/20190524/lib/python3.6/site-packages/datacube/ui/click.py", line 188, in new_func
    return f(parsed_config, *args, **kwargs)
  File "/g/data/v10/public/modules/dea/20190524/lib/python3.6/site-packages/datacube/ui/click.py", line 216, in with_index
    return f(index, *args, **kwargs)
  File "/g/data/v10/public/modules/dea/20190524/lib/python3.6/site-packages/fc/fc_app.py", line 345, in ensure_products
    _, out_product = _ensure_products(app_config_file, index, dry_run)
  File "/g/data/v10/public/modules/dea/20190524/lib/python3.6/site-packages/fc/fc_app.py", line 117, in _ensure_products
    output_product = index.products.add(output_product)
  File "/g/data/v10/public/modules/dea/20190524/lib/python3.6/site-packages/datacube/index/_products.py", line 92, in add
    'Metadata Type {}'.format(product.name)
  File "/g/data/v10/public/modules/dea/20190524/lib/python3.6/site-packages/datacube/utils/changes.py", line 132, in check_doc_unchanged
    ', '.join(['{}: {!r}!={!r}'.format('.'.join(map(str, offset)), v1, v2) for offset, v1, v2 in changes])
datacube.utils.changes.DocumentMismatchError: Metadata Type ls8_fc_albers differs from stored (metadata.format.name: 'NetCDF'!='NetCDF CF')

Proposed solution
Add metadata format name in to product config files

Coverage remained the same at 56.263% when pulling 8dba5f4b19441d272c0f5e4ffbd88bc6d03be4cb on metadata-format-name into 89d6fceded118a6fc300365c3ca044fffba279ea on master.

@jeremyh @omad @Kirill888 I still do not understand how separate we want to keep the concepts of a format and a driver. So the driver name is NetCDF CF but the format is NetCDF? Is that what we consistently want to implement?

@uchchwhash there could be several drivers for the same format, writer drivers and reader drivers are different also. For example netcdfs are written by netcdf driver that can only produce netcdfs, but we read those files with rasterio driver, same one that reads tiff and many other raster formats.

This change;
output_product_definition['metadata']['format'] = {'name': 'NetCDF'}
Will hard code the storage format as NetCDF.  This will break writing files to tif.
I'm not keen on this change.  Let's talk about this.

The storage driver in lsX_fc_albers.yaml  can be updated to;
storage:
driver: NetCDF
This could fix the error.

@santoshamohan I would agree with @dunkgray here that the format need not be hard-coded. Maybe check if the driver is NetCDF or GeoTIFF and set it accordingly?

Reverted the changes within fc_app.py file and updated storage driver name to NetCDF

Thanks for the change.  I don't think the metadata field in the config yaml's is used.  So consider removing them.


error: The following binaries use incompatible versions of Swift:

XCode9.3 (Swift4.1) 사용중입니다.
carthage를 이용해 github "devxoul/Then"를 설치했는데 아래와 같은 오류가 뜹니다.
error: The following binaries use incompatible versions of Swift:
/Users/gns/Library/Developer/Xcode/DerivedData/CopyPodcasts-edkbnveraiflpmhipjhtkxsbijgx/Build/Products/Debug-iphoneos/CopyPodcasts.app/CopyPodcasts
/Users/gns/Library/Developer/Xcode/DerivedData/CopyPodcasts-edkbnveraiflpmhipjhtkxsbijgx/Build/Products/Debug-iphoneos/CopyPodcasts.app/Frameworks/Then.framework/Then

해결방법이 있을까요?

Duplicated with #55


Base classes for Off Policy Agents

This should make the code much more comprehensible, especially with the number of arguments we have. And at the same time resolve a lot of maintainability issues.

There's also a lot of code duplication issues that show up pylint. Maybe work on that too.

Yup, that's the goal. Another thing to do would be properly decide the parameters to be kept/removed in agents and added/removed from the trainers.

Since we already have the On Policy Agents, I'm renaming this to Off Policy. I'll raise a PR for this soon.

I'm thinking of refactoring each of the individual off policy algorithms first so that the code is neater, more uniform and shorter.

To-do:

[ ] Refactor DDPG
[ ] Refactor TD3
[ ] Refactor SAC
[ ] Finalise BaseAgent and Base OffPolicyAgent classes
[ ] Refactor Trainer and OffPolicyTrainer (Trainer classes are also really long, would be a good idea to first shorten them then maybe we can separate them into multiple files if they're still too big)

Tracking in separate issues now. #263, #162 and #264


Classification under 'somatic cell'

We include a class 'somatic cell'

def: "A cell of an organism that does not pass on its genetic material to the
organism's offspring (i.e. a non-germ line cell)." [GOC:tfm, ISBN:0721662544]

There are lots of cell types that probably should be classified here that are
not.

Ideally these would be autoclassified somehow.

As a sample, here is the set of most general classes under 'native cell' that
are not classified as somatic. Obviously some of these are very general and
should be left as unspecified, but some such as 'supporting' include epithelial
as a child, and epithelial is not classified as somatic, when it probably
should be.

I'll bet there are some weird invertebrate cases that defy assumptions here;
e.g. some cell types whose natural lineage is go from a specialized somatic
cell to a germ line cell....

 His-Purkinje system cell
CL:0011115 ! precursor cell
CL:2000002 ! decidua cell
CL:2000020 ! inner cell mass native cell
CL:2000021 ! sebaceous gland native cell
CL:2000022 ! cardiac septum native cell

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 10 Apr 2015 at 3:13

WARNING: This issue has been automatically closed because it has not been updated in more than 3 years. Please re-open it if you still need this to be addressed – we are now getting some resources to deal with such issues.


This article has been withdrawn. Please delete this page.

Description
This article has been withdrawn at 2024 /3 / 22 09:09:45 UTC (1 KB). Please delete this page. This page influenced my article to be republished
(Optional:) Please add any files, screenshots, or other information here.
No response
(Required) What is this issue most closely related to? Select one.
References
Internal issue ID
9ce6541d-14fd-4739-8fa5-0e20767913b2
Paper URL
https://arxiv.org/html/2401.10519v1
Browser
Chrome/<IP_ADDRESS>
Device Type
Desktop

Only v2 of the article has been withdrawn. Version one is still on arXiv, so this is as intended.

sorry，it's my fault. I will withdraw  soon.
---- Replied Message ----
| From | Deyan @.> |
| Date | 10/15/2024 06:30 |
| To | @.> |
| Cc |<EMAIL_ADDRESS>|
| Subject | Re: [arXiv/html_feedback] This article has been withdrawn. Please delete this page. (Issue #2321) |
Only v2 of the article has been withdrawn. Version one is still on arXiv, so this is as intended.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>


Release CDI-Unit 4.0.0

@BrynCooke master branch now includes three pull requests with fixes and upgrades (Java 8, CDI 1.2, Weld 2.x, Mockito 2.7.15 as minimum versions): https://github.com/BrynCooke/cdi-unit/pulls?q=is%3Apr+milestone%3A4.0.0+is%3Aclosed
Would you make a 4.0.0 release, please?
Also, what form does the source code for http://jglue.org/cdi-unit/ take? Perhaps we could convert it to something like readthedocs.org for easier maintenance in future (eg update the docs in the same pull request which creates/changes a feature). Note that this would mean hosting the docs on readthedocs.org instead of jglue.org.

Done. This should appear in central shortly. Thanks for your contributions and sorry for the delay.


Allow uppercase characters in image reference hostname

Docker 1.9 allowed uppercase characters in registry hostnames. Docker 1.10 does not. This should be fixed, because otherwise there may be images that can't be accessed because their names don't parse correctly under Docker 1.10.
Note that this only applies to hostnames, not remote image names. Previous versions also prohibited uppercase letters after the hostname, but Docker 1.10 extends this to the hostname itself.
Example:
With 1.9:
# docker tag registry:2.2.1 OldRegistry:2.2.1
repository name component must match "[a-z0-9](?:-*[a-z0-9])*(?:[._][a-z0-9](?:-*[a-z0-9])*)*"
# docker tag registry:2.2.1 SomeHostname:8080/registry:2.2.1
[success]

With 1.10:
# docker tag wordpress WordpressImage
Error parsing reference: "WordpressImage" is not a valid repository/tag
# docker tag wordpress SomeHostname:8080/wordpress
Error parsing reference: "SomeHostname:8080/wordpress" is not a valid repository/tag

See https://github.com/docker/distribution/issues/1433

If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
For more information about reporting issues, see CONTRIBUTING.md.
You don't have to include this information if this is a feature request
(This is an automated, informational response)

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

Fixed by #20175.


Introduce hard-coded serializers for the internal messages for the agent assignment et al

Just a way to make things faster, and also they're a touch problematic anyway

Done now


Is possible list all videos from a playlist link?

Hello, I am showing a playlist with a fragment, all right, but I was wondering if I can list all the videos in a playlist in some way, so the user can choose which video to go to instead of skipping one by one?

New, I really need this feature

This library's responsibility is to play videos not to list them. The listing can be done in a recyclerview.


Master refactor

No more for loop.
@SkamDart @rohansub @cmalk

Looks great when it runs on my machine.


Add max_tools_use_per_run to AgentConfiguration

Description
Just adds the new field with default value = 1
This will be backfilled here from front/migrations/20240412_force_use_at_iteration.ts
Risk

Deploy Plan
Run init_db

Where is the migration ?

We added as part of this: https://github.com/dust-tt/dust/pull/4694/files#r1562586081

🙏


MINOR: tune KIP-631 configurations

Since we expect KIP-631 controller failovers to be fairly cheap, tune
the default raft configuration parameters so that we detect node
failures more quickly.
Reduce the broker session timeout as well so that broker failures are
detected more quickly.

Can we add a comment to the code explaining why these timeouts are lower than the ZK equivalent ones (similar to what's in the PR description).

I added some JavaDoc to the RaftConfig class

@hachikuji can you please review these timeouts?

Overall, I'm ok with the lower timeouts for 2.8. I do think it's highly likely we'll end up increasing them after we get some operational experience with the new code. It's just the nature of these things. Flaky networks are much more common than true node failures.

We have a hard-coded fetch max wait time here: https://github.com/apache/kafka/blob/trunk/raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java#L143. I think it would be a good idea to reduce this to 500ms to give a little more margin for response delays given the lower default timeout. (We should probably also consider making this a configuration, but we can do it separately)

good catch. will change


Update screenshot on chrome web store

It's pretty old now, I need to upload a few more screenshots especially showing the options and various display modes. Maybe I'll do it after:

https://github.com/hankxdev/one-click-extensions-manager/issues/137


simple class without dllexport lead to ctor_0 EntryPointNotFoundException

Brief Description
OS: Windows
Used headers
class Spi
{
public:
virtual void Callback() = 0;
};
class __declspec(dllimport) Api
{
public:
static Spi *CreateApi();
virtual void RegisterSpi(Spi *pSpi) = 0;
};
Used settings
options.GeneratorKind = GeneratorKind.CSharp;
Target: MSVC
Stack trace or incompilable generated code
Test.cs
SuppressUnmanagedCodeSecurity]
[DllImport("Test", CallingConvention = global::System.Runtime.InteropServices.CallingConvention.Cdecl,
EntryPoint="??0Spi@@QEAA@XZ")]
internal static extern global::System.IntPtr ctor_0(global::System.IntPtr instance);
protected Spi()
{
__Instance = Marshal.AllocHGlobal(sizeof(global::Test.Spi.__Internal));
__ownsNativeInstance = true;
NativeToManagedMap[__Instance] = this;
        // failed,because Spi don't export,only callback interface
        __Internal.ctor_0((__Instance + __PointerAdjustment));

        SetupVTables(GetType().FullName == "Test.Spi");

}
System.EntryPointNotFoundException "??0Spi@@QEAA@XZ"

I'm quite surprised that you've got an error. CppSharp hasn't failed yet for me even for really big headers there weren't any dll import failures.
However, are you sure that the dll file is correct?

@JiangHeWei this can be fixed but requires a little setting up on your part. So in case you have control over the C++ you wrap, it would be easiest to you to instead just add an explicit empty constructor (and destructor) to Spi.

C++ dll is third part.so spi is not exported. maybe control don't "call __Internal.ctor_0((__Instance + __PointerAdjustment))"?

class Spi only exist virtual method for export C++ interface

if delete  "__Internal.ctor_0((__Instance + __PointerAdjustment))". it is ok. dll sure ok.

@JiangHeWei if you delete that line, your object might not be properly constructed which in turn might lead to undefined behaviour. So try the following instead:

Find a *-.symbols.cpp among the generated files;
Add it to a C++ project and compile it;
In your configuration add the resulting library to your input i.e. Module.Libraries.Add(...);
Regenerate and run.

Let me know if this helps.

in C#:
public SpiImp:Spi
{
public override Callback(){...}
}
Api api = ApiInternal.CreateApi();
api.RegisterSPi(new SpiImp());
error: System.EntryPointNotFoundException "??0Spi@@QEAA@XZ"
because in auto genegrated code call "__Internal.ctor_0((__Instance + __PointerAdjustment))"->
SuppressUnmanagedCodeSecurity]
[DllImport("Test", CallingConvention = global::System.Runtime.InteropServices.CallingConvention.Cdecl,
EntryPoint="??0Spi@@QEAA@XZ")]
internal static extern global::System.IntPtr ctor_0(global::System.IntPtr instance);

I understand what your problem is and why you think deleting the line "fixes" it. What I am telling you that this "fix" might cause you other problems. So please try my steps above instead.

this is callback class case. "Spi" only include virtual method. don't auto generate C# wrapper class into c++?

Spi is inherited by some real class, for example that SpiImpl. SpiImpl silently calls the base constructor, that is, the constructor of Spi. This is standard behaviour of C++ and all other object-oriented languages. So you'd be better off actually calling that constructor.

@ddobrev i will create new c++ project to solve this.thanks.


WIP: UInt flags in sys

Part of https://github.com/gtk-rs/gir/issues/575
cc @GuillaumeGomez, @sdroege, @tmiasko

All other base gtk-rs crates don't need manual changes.


no join() and avoid property name collision

new pr: https://github.com/gamerson/liferay-blade-cli/pull/411


Inference Stuck for Pattern of Empty String ""

If passing "" as one of the pattern strings to run, the current prompt might fail for such case because it will be ignored. Such problem leads to hang of inference.

https://github.com/jeffrey82221/regex_inference/pull/7

Fix at this PR: https://github.com/jeffrey82221/regex_inference/pull/7


feat: update 404 page

This pull request brings update for 404 error page. And also solve issue #376
Steps to check:

open preview
make sure that everything looks and feels as expected

Hi @xmulligan , could you please trigger a rebuild of the pull request preview without the cache? Unfortunately, I don't have the necessary access to do this myself.

Sorry lost this at KubeCon last week. Done now

Hey @xmulligan please take a look here too


graphviz.backend.CalledProcessError: Command '['dot.bat', '-Tpng']' returned non-zero exit status 1.

Hi!
I'm sorry, actually I'm a doctor, so again, I'm sorry for my code and explanation, but here is the error
Code:
xgb.plot_tree(model, num_trees=0)
rcParams['figure.figsize'] = 80, 50
'C:\Users\Ruslan' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
File "C:/Users/Ruslan Oliinyk/PycharmProjects/XGB_Facebook/income.py", line 89, in
xgb.plot_tree(model, num_trees=0)
File "C:\Users\Ruslan Oliinyk\Anaconda3\lib\site-packages\xgboost\plotting.py", line 281, in plot_tree
s.write(g.pipe(format='png'))
File "C:\Users\Ruslan Oliinyk\Anaconda3\lib\site-packages\graphviz\files.py", line 138, in pipe
quiet=quiet)
File "C:\Users\Ruslan Oliinyk\Anaconda3\lib\site-packages\graphviz\backend.py", line 231, in pipe
out, _ = run(cmd, input=data, capture_output=True, check=True, quiet=quiet)
File "C:\Users\Ruslan Oliinyk\Anaconda3\lib\site-packages\graphviz\backend.py", line 174, in run
output=out, stderr=err)
graphviz.backend.CalledProcessError: Command '['dot.bat', '-Tpng']' returned non-zero exit status 1. [stderr: b"'C:\Users\Ruslan' is not recognized as an internal or external command,\r\noperable program or batch file.\r\n"]
Thank you!

Thanks for the report! Could you please include information from the environment where this was run? Should just be conda list if the environment is active 🙂

I have seen a similar bug report in pydot/pydot#207. Both here and there, the user has a space character in the directory path. A possible solution may be to wrap the executable in double quotes in line 37 of recipe/bld.bat. Change that line from:
echo %%~dp0.\graphviz\%%~nf.exe %%* >> %%~nf.bat

to:
echo "%%~dp0.\graphviz\%%~nf.exe" %%* >> %%~nf.bat

I did a very quick, simple test of a batch script like that in Windows 10 and the resulting dot.bat contained:
"%~dp0.\graphviz\dot.exe" %*

Hopefully that allows for spaces in the directory path. I only have limited access to Windows, so please test it yourself as well.
@oliinykmd Maybe you can try this out by finding dot.bat on your system. Maybe in a directory like:
C:\Users\Ruslan Oliinyk\Anaconda3\Library\bin\dot.bat

It probably contains a line like:
%~dp0.\graphviz\dot.exe %*

Put double quotes around that first part:
"%~dp0.\graphviz\dot.exe" %*

Please report back if that prevents the error.
Some resources in case you are wondering about the meaning of all those percent signs:

https://stackoverflow.com/questions/14509652/what-is-the-difference-between-and-in-a-cmd-file
https://stackoverflow.com/questions/5034076/what-does-dp0-mean-and-how-does-it-work

@peternowee, could you submit a PR with your fix here?

@ccordoba12 Sorry, but I do not feel certain enough about the fix to do that. I do not use Windows, conda or conda-forge myself right now, so I cannot oversee possible side-effects of this and I am not able to do extensive testing. I only did a very quick manual run of a manually created batch file containing only this snippet of code on a Windows CI VM to see if the dot.bat file was written correctly and it was. I did not check if dot.bat actually executed dot.exe correctly, with all its arguments, for example. Someone with more experience with this project and better access to Windows needs to also have a look at it and maybe do a test conda install in a path with a space character in it to see if it really works. If it works, feel free to make the commit under your own name.

I have seen a similar bug report in pydot/pydot#207. Both here and there, the user has a space character in the directory path. A possible solution may be to wrap the executable in double quotes in line 37 of recipe/bld.bat. Change that line from:
echo %%~dp0.\graphviz\%%~nf.exe %%* >> %%~nf.bat

to:
echo "%%~dp0.\graphviz\%%~nf.exe" %%* >> %%~nf.bat

I did a very quick, simple test of a batch script like that in Windows 10 and the resulting dot.bat contained:
"%~dp0.\graphviz\dot.exe" %*

Hopefully that allows for spaces in the directory path. I only have limited access to Windows, so please test it yourself as well.
@oliinykmd Maybe you can try this out by finding dot.bat on your system. Maybe in a directory like:
C:\Users\Ruslan Oliinyk\Anaconda3\Library\bin\dot.bat

It probably contains a line like:
%~dp0.\graphviz\dot.exe %*

Put double quotes around that first part:
"%~dp0.\graphviz\dot.exe" %*

Please report back if that prevents the error.
Some resources in case you are wondering about the meaning of all those percent signs:

https://stackoverflow.com/questions/14509652/what-is-the-difference-between-and-in-a-cmd-file
https://stackoverflow.com/questions/5034076/what-does-dp0-mean-and-how-does-it-work

This worked for me!
Windows 10, pydot 1.4.1, graphviz-2.44.1 (installed from https://www2.graphviz.org/Packages/stable/windows/10/msbuild/Release/Win32/)
Trying to run the line keras.utils.plot_model(model, "my_first_model.png") from https://keras.io/guides/functional_api/
My anaconda3 environments are everything were installed in Users/Bryan Lanier/...  and I was also getting same error since it would complain about 'C:\Users\Bryan...'
The quotes around the code in dot.bat was all I needed.  Only thing to note was, that for me, I had to find the dot.bat that was in
environment I had installed it in!   For me, that was C:\Users\Bryan Lanier\anaconda3\envs\py37\Library\bin
Thanks a million for finding this simple fix

It took me so many search to find this simple fix. Thanks!!!!!!

Could someone please retest this with the latest packages?

Going to tentatively close. We are now building Windows from source with PR ( https://github.com/conda-forge/graphviz-feedstock/pull/59 ). So no longer include or need this batch script


exhibit libs unaccessible

we're seeing this message:
Failed to load resource: net::ERR_CERT_DATE_INVALID for
https://api.simile-widgets.org/exhibit/HEAD/exhibit-api.js
Our Exhibit apps are offline
Please update the certificates

It seems that the www.simile-widgets.org certificate ended yesterday
11/2/2020
-- luismiguel  (@lmorillas)
El mié., 12 feb. 2020 a las 13:56, jay gray<EMAIL_ADDRESS>escribió:

we're seeing this message:
Failed to load resource: net::ERR_CERT_DATE_INVALID for
https://api.simile-widgets.org/exhibit/HEAD/exhibit-api.js
Our Exhibit apps are offline
Please update the certificates
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/simile-widgets/exhibit/issues/194?email_source=notifications&email_token=AAF6EZ7HNE6CXCSIQPEBCEDRCPWXBA5CNFSM4KTZHSY2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IM5V62Q,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAF6EZYHSZ6A63GA34RVUYLRCPWXBANCNFSM4KTZHSYQ
.

good now - thanks


release: Use PyPI Trusted Publishing

Instead of using the secret stored in environment secrets, allow the publish action to use the OIDC identity to authenticate to pypi.org. This repository/workflow/environment combo has been marked as a "Trusted Publisher" in pypi.org: this means PyPI should give the publish action a short lived token to use for publishing.
This enables #2370: but the secret should still be removed before closing the issue (maybe after one successful release with Trusted Publishing).

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 97.692%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1320

Relevant Lines:
1343

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.001%) to 97.693%

Totals

Change from base Build<PHONE_NUMBER>:
0.001%

Covered Lines:
1321

Relevant Lines:
1344

💛  - Coveralls


Add a Communications section to CONTRIBUTING.md

Part of the MVP for pulumi/home#1779.

[ ] Link to Code of Conduct at the top of the document
[ ] Link to Community Slack
[ ] Clarify PR workflow (fork-and-clone) and review expectations
[ ] Link to issue triage and project priority guidelines, if any
[ ] Link to any Pulumi community events such as office hours, podcasts, etc

https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/adding-a-code-of-conduct-to-your-project
It appears that we can and should add a "github blessed" code of conduct to our repositories do that our community profile looks good and checks out: https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/about-community-profiles-for-public-repositories

As per https://github.com/pulumi/pulumi/pull/8894 this will be rolled up into that work a bit - I'll include links and verbiage that can hopefully be templated into other repositories.


Refactor the cost function to provide more api endpoints

This PR adds two new API endpoints to Terrain Layer to give other modules more granular access to terrain information.
Effectively, this PR splits up the cost function into two separate APIs:

listTerrain: Returns a list of terrain that's relevant for the given set of options. All the terrain in this list follows a shared interface (that's what the new TerrainInfo class is for), which makes it easy for consumers of the list to work with the provided information.
costWithTerrain: Does what the cost function does, but allows passing in a list of terrain previously retrieved via listTerrain.

The cost function retains its functionality, but now only consists of a call to listTerrain and a call to costWithTerrain, to avoid duplicate code.
I've marked this PR as a draft, because while I carefully tried to not change any logic, the amount of changes in the code is quite significant, and I haven't tested the change extensively yet. I will start working with the new API Endpoints in Drag Ruler soon, which is an opportunity for more testing. If anything crops up, I'll add the fixes to this PR. However, I don't intend to add any big changes to the PR at this point (that is unless you suggest some), so it should be fine for review.

My implementation of TokenTerrainInfo was a bit buggy. I've added a commit that addresses those bugs. I think it's worth squashing these changes upon merge, but for now I'll leave separate to make it easier for you to track my changes.

The initial version of Drag Ruler, that taps into the new API, is done. I've tested it with all three types of Terrain (regular terrain, measurement templates, tokens) and they all worked nicely, so I guess this is ready to merge.

Looks good.  This makes a lot more sense to me now, seeing the code.
Could I request one more quick change?
Split the listTerrain into a function for each type.  So listTerrain, listMeasuredTerrain and listTokenTerrain.  And call all three of those from a listAllTerrain which will essentially concat the results from the first three.
That way if for some reason you only want to get the cost from only one of those terrains, you can.

Enhanced Terrain is top of the list this weekend.  I'm going to clean up some tickets and then hopefully release either tomorrow or Sunday.

Sure thing!


WIP:build centos node image for bml test

for building image for bml

/test-integration-metal3-dev-tools-centos

/test-integration-metal3-dev-tools-centos

/test-integration-metal3-dev-tools-centos

/test-integration-metal3-dev-tools-centos

testing is done. Closing

testing is done. Closing


Product update page tuneup

@featurepeek /product-updates

https://app.clubhouse.io/featurepeek/story/3560/tune-up-product-updates-page


[WIP] Adding run length encoding exercise

This should cover all the previous change requests on my last pull request (#862)

Thanks! :tada:

@MatForsberg great first PR. Thanks for your patience!


SyntaxError: Unexpected token 'export'

Hi, I'm not being able to use colors-convert using typescript with ts-node-dev. I'm getting the following error:
yarn dev
yarn run v1.22.10
$ tsnd --respawn --transpile-only --ignore-watch node_modules src/index.ts
[INFO] 22:18:37 ts-node-dev ver. 1.1.6 (using ts-node ver. 9.1.1, typescript ver. 4.3.4)
/Users/daltonmenezes/projetos/github/new-aura/node_modules/colors-convert/dist/index.js:1
export { isHex, isRgb, isRgba, isCmyk, isHsl, isHsla, isColor } from './lib/types/isType';
^^^^^^

SyntaxError: Unexpected token 'export'
    at Object.compileFunction (node:vm:355:18)
    at wrapSafe (node:internal/modules/cjs/loader:1022:15)
    at Module._compile (node:internal/modules/cjs/loader:1056:27)
    at Module._compile (/Users/daltonmenezes/projetos/github/new-aura/node_modules/source-map-support/source-map-support.js:547:25)
    at Module._extensions..js (node:internal/modules/cjs/loader:1121:10)
    at Object.nodeDevHook [as .js] (/Users/daltonmenezes/projetos/github/new-aura/node_modules/ts-node-dev/lib/hook.js:63:13)
    at Module.load (node:internal/modules/cjs/loader:972:32)
    at Function.Module._load (node:internal/modules/cjs/loader:813:14)
    at Module.require (node:internal/modules/cjs/loader:996:19)
    at require (node:internal/modules/cjs/helpers:92:18)
[ERROR] 22:18:37 SyntaxError: Unexpected token 'export'

Hi Dalton, I'll take a look. Thank you!

I've never used ts-node-dev before. I tested here and everything seems to work well. I used your ts-node-dev version (1.1.6) and not the last one. Let me know if there are updates.

Hello! Can I have a reproducible example?

Sure, @ilariaventurini.
https://github.com/daltonmenezes/colors-convert-issue-examples
I made two examples, one using TypeScript and the other using JavaScript vanilla in Node.js with esmodules. You can run both by running yarn dev.

Still not working in node.js