Karpenter launches wrong instance type/Ignoring instance-family

Description
Observed Behavior:
Karpenter appears to be ignoring the instance-family setting in the Provisioner. "g3" is launched but not in the instance-family list. However, "g" is in the instance-category.
2023-08-01T22:23:53.205Z	DEBUG	controller.provisioner	23 out of 667 instance types were excluded because they would breach provisioner limits	{"commit": "34d50bf-dirty", "provisioner": "gpu"}
2023-08-01T22:23:53.214Z	INFO	controller.provisioner	found provisionable pod(s)	{"commit": "34d50bf-dirty", "pods": 1}
2023-08-01T22:23:53.214Z	INFO	controller.provisioner	computed new machine(s) to fit pod(s)	{"commit": "34d50bf-dirty", "machines": 1, "pods": 1}
2023-08-01T22:23:53.229Z	INFO	controller.provisioner	created machine	{"commit": "34d50bf-dirty", "provisioner": "gpu", "requests": {"cpu":"165m","memory":"184Mi","nvidia.com/gpu":"1","pods":"6"}, "instance-types": "g2.2xlarge, g2.8xlarge, g3.16xlarge, g3.4xlarge, g3.8xlarge and 23 other(s)"}
2023-08-01T22:23:53.936Z	DEBUG	controller.machine.lifecycle	created launch template	{"commit": "34d50bf-dirty", "machine": "gpu-sgbdg", "provisioner": "gpu", "launch-template-name": "karpenter.k8s.aws/2135683832065095291", "id": "lt-01f701678584180e6"}
2023-08-01T22:23:54.090Z	DEBUG	controller.machine.lifecycle	created launch template	{"commit": "34d50bf-dirty", "machine": "gpu-sgbdg", "provisioner": "gpu", "launch-template-name": "karpenter.k8s.aws/2064232833418684968", "id": "lt-06d5e5f91bd309abd"}
2023-08-01T22:23:54.227Z	DEBUG	controller.machine.lifecycle	created launch template	{"commit": "34d50bf-dirty", "machine": "gpu-sgbdg", "provisioner": "gpu", "launch-template-name": "karpenter.k8s.aws/6849786203555880075", "id": "lt-04813f97e93ed9808"}
2023-08-01T22:23:54.348Z	DEBUG	controller.machine.lifecycle	created launch template	{"commit": "34d50bf-dirty", "machine": "gpu-sgbdg", "provisioner": "gpu", "launch-template-name": "karpenter.k8s.aws/12266669837394155381", "id": "lt-0b9958b65db9c1334"}
2023-08-01T22:23:54.485Z	DEBUG	controller.machine.lifecycle	created launch template	{"commit": "34d50bf-dirty", "machine": "gpu-sgbdg", "provisioner": "gpu", "launch-template-name": "karpenter.k8s.aws/16420496653598597600", "id": "lt-049f6ba0882d8008a"}
2023-08-01T22:23:55.624Z	DEBUG	controller.deprovisioning	waiting on cluster sync	{"commit": "34d50bf-dirty"}
2023-08-01T22:23:56.413Z	INFO	controller.machine.lifecycle	launched machine	{"commit": "34d50bf-dirty", "machine": "gpu-sgbdg", "provisioner": "gpu", "provider-id": "aws:///us-west-2b/i-0d14d857c2052fc2d", "instance-type": "g3.16xlarge", "zone": "us-west-2b", "capacity-type": "spot", "allocatable": {"cpu":"63770m","ephemeral-storage":"17Gi","memory":"453771Mi","nvidia.com/gpu":"4","pods":"737"}}

Expected Behavior:
Karpenter should select from the types specified in the Provisioner.
Reproduction Steps (Please include YAML):
Configured a provisioner:
apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"karpenter.sh/v1alpha5","kind":"Provisioner","metadata":{"annotations":{},"name":"gpu"},"spec":{"limits":{"resources":{"cpu":"1000","memory":"1000Gi"}},"providerRef":{"name":"gpu-spot"},"requirements":[{"key":"karpenter.k8s.aws/instance-generation","operator":"Gt","values":["0"]},{"key":"karpenter.k8s.aws/instance-category","operator":"In","values":["g","p"]},{"key":"karpenter.k8s.aws/instance-cpu","operator":"Gt","values":["2"]},{"key":"node.kubernetes.io/instance-family","operator":"In","values":["p4d","p5d","g5","g4ad","g4dn"]},{"key":"topology.kubernetes.io/zone","operator":"In","values":["us-west-2a","us-west-2b","us-west-2c"]},{"key":"kubernetes.io/arch","operator":"In","values":["amd64"]},{"key":"karpenter.sh/capacity-type","operator":"In","values":["spot"]}],"taints":[{"effect":"NoSchedule","key":"nvidia.com/gpu","value":"true"}],"ttlSecondsAfterEmpty":30,"ttlSecondsUntilExpired":86400,"weight":10}}
  creationTimestamp: "2023-07-31T23:47:39Z"
  generation: 17
  name: gpu
  resourceVersion: "325766"
  uid: 087d67c0-5209-49d6-972b-329ffc7f422e
spec:
  limits:
    resources:
      cpu: 1k
      memory: 1000Gi
  providerRef:
    name: gpu-spot
  requirements:
  - key: karpenter.k8s.aws/instance-generation
    operator: Gt
    values:
    - "0"
  - key: karpenter.k8s.aws/instance-category
    operator: In
    values:
    - g
    - p
  - key: karpenter.k8s.aws/instance-cpu
    operator: Gt
    values:
    - "2"
  - key: node.kubernetes.io/instance-family
    operator: In
    values:
    - p4d
    - g5
    - g4ad
    - g4dn
  - key: topology.kubernetes.io/zone
    operator: In
    values:
    - us-west-2a
    - us-west-2b
    - us-west-2c
  - key: kubernetes.io/arch
    operator: In
    values:
    - amd64
  - key: karpenter.sh/capacity-type
    operator: In
    values:
    - spot
  - key: kubernetes.io/os
    operator: In
    values:
    - linux
  taints:
  - effect: NoSchedule
    key: nvidia.com/gpu
    value: "true"
  ttlSecondsAfterEmpty: 30
  ttlSecondsUntilExpired: 86400
  weight: 10
status: {}

And a simple pod:
---
apiVersion: v1
kind: Pod
metadata:
  name: example
  namespace: example
spec:
  containers:
    - name: app
      image: centos
      command: ["/bin/sh"]
      args: ["-c", "while true; do echo $(date -u) >> /dev/null; sleep 5; done"]
      resources:
        requests:
          memory: 64Mi
          cpu: 10m
        limits:
          nvidia.com/gpu: 1
          memory: 128Mi
          cpu: 40m
  tolerations:
    - key: "nvidia.com/gpu"
      value: "true"
      operator: "Equal"
      effect: "NoSchedule"

Versions:

Chart Version: v0.29.2
Kubernetes Version (kubectl version):

Client Version: v1.27.3
Kustomize Version: v5.0.1
Server Version: v1.27.3-eks-a5565ad

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

I'm looking at this page for the values: https://karpenter.sh/docs/concepts/instance-types
But if I'm reading it wrong, please let me know!

node.kubernetes.io/instance-family is not a valid label. The correct label is karpenter.k8s.aws/instance-family. Can you try with this label and see if Karpenter respects the requirements then?

D'oh! 🤦 Yes, that was it. Easy fix. Thanks so much!


Azure Automation job gets suspended

I have setup an Automation Account and followed the README guide. I did testing on a Management group with single subscription and the report came out nicely. However, when I switch it to our production Management group which has a lot many subscriptions and underlying resources, the job gets suspended after running for a few minutes with exception message:
The runbook job was attempted 3 times, but it failed each time.  Common reasons that runbook jobs fail can be found here:  https://docs.microsoft.com/en-us/azure/automation/automation-troubleshooting-automation-errors
There were a lot of entries in the Error Field where each error retry refers to couple of variables $VNET and $SUBNET. I reviewed ARI_Automation.ps1, but couldn't find any reference of those two variables.
You cannot call a method on a null-valued expression. At line:45 char:17 + $VNET = $data.virtualNetworkSubnetId.split("/")[8] + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNull
You cannot call a method on a null-valued expression. At line:46 char:17 + $SUBNET = $data.virtualNetworkSubnetId.split("/")[10] + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNull
You cannot call a method on a null-valued expression. At line:45 char:17 + $VNET = $data.virtualNetworkSubnetId.split("/")[8] + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNull
You cannot call a method on a null-valued expression. At line:46 char:17 + $SUBNET = $data.virtualNetworkSubnetId.split("/")[10] + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [], RuntimeException + FullyQualifiedErrorId : InvokeMethodOnNull

Seems like the offending script is: ARI/Modules/Compute/APPSERVICES.ps1.
Can someone please help me fix this?

Hello,
The Automation Account method is outdated, we are working on a newer version.
To be honest I'm surprised it worked that far.
Do you have App Services in your environment? By the error message I believe it is complaining by empty values in that resource.

Hello,
Yes, we indeed have quite a few App Services running in our environment. How do I go around this error message? Could you please advise?

Hello,
Can you try the new automation account method we release in the version 3.0?
Thanks

Hello @Claudio-Merola , the new script worked better than the previous one. I still see error messages in the job section, this makes me wonder did it pull out all the resources?

Few of the messages from the Job streams:
InvalidOperation: Line | 45 | $VNET = $data.virtualNetworkSubnetId.split("/")[8] | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | You cannot call a method on a null-valued expression.
InvalidOperation: Line | 46 | $SUBNET = $data.virtualNetworkSubnetId.split("/")[10] | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | You cannot call a method on a null-valued expression.
Remove-Variable: Line | 174 | … Remove-Variable PIP, vmnic, vmnsg, VNET, Subnet | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | Cannot find a variable with the name 'PIP'.
InvalidOperation: Line | 121 | … $PIP = $vmnic.properties.ipConfigurations.properties.publ … | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | You cannot call a method on a null-valued expression.
Is there something I am doing differently?

I believe the root cause of the automation script is due to this part.

Invoke-WebRequest unable to get list url for the modules. Hence, causing a null error. (Nothing from the module list).
Been trying to filter but to no success. The list of *.ps1 files are lazy loaded as json result and populated using react.

Hi @Sabrthor
Last week I fixed a bug in the script that might have relation with your issue.
Can you try the updated version of the Automation Script and see how it goes?
Thanks

Hi @Claudio-Merola,
Unfortunately, this still hasn't been solved.
I have done following:

Reinstalled all the required modules with Runtime 7.2.
Ensured the Automation Account has Reader permissions from the Management Group.
Ensured the Automation Account has Storage Blob Data Contributor permissions to the Storage Account.

When I ran the automation, the output still shows bunch of errors.

What I could not believe is the difference in the resource list.
The spreadsheet shows a total resource count of 13004, whereas, I have about ~27k varied resources.
The same spreadsheet overview page says, I have 786 Virtual Machines in the Overview page:

The Virtual Machine sheet shows ~1400 machines:

In the output of the Automation script, it says ~2200 VMs:

While in reality, I have 1894 VMs. And this is just not limited to VMs, but almost any resource type (such as Virtual Network).
What I also observe is the Subscription list in the last sheet is completely wrong. I have about 40+ subscriptions and the sheet shows 24.

Hi @Sabrthor ,
The Excel sheet will present more VMs, because of the extra resources, like VMs with more than 1 NIC, or the extensions, so is expected to have more lines. But the script should present the correct number of VMs...
I have updated the script some time ago with an extra option at the line 95, can you change the: $RunDebug to $true? So we can have some more details of might be going on.


Cannot parse ADD GENERATED

Hi, I tried this library with my schema dump and find out that sqlparser 0.38.0 cannot parse this ADD GENERATED clause: ... ALTER COLUMN xxx ADD GENERATED ALWAYS AS IDENTITY .... The error is

Expected SET/DROP NOT NULL, SET DEFAULT, SET DATA TYPE after ALTER COLUMN, found: ADD at Line: 18, Column 71

That's a line from my PostgreSQL schema dump, it's for setting up identity column.

Text form in case you want to copy:
create table t(id int primary key);
alter table t alter column id add generated always as identity;

I just replaced the SQL statement in the example.
use sqlparser::dialect::PostgreSqlDialect;
use sqlparser::parser::Parser;

fn main() {
    let sql = "ALTER TABLE t ALTER COLUMN id ADD GENERATED ALWAYS AS IDENTITY";
    let dialect = PostgreSqlDialect {};
    let ast = Parser::parse_sql(&dialect, sql).unwrap();

    println!("AST: {:?}", ast);
}


Create assay plugin shortcut collections for landing zones

It was reported that e.g. the root RawData collection defined in the meta_ms assay plugin is not created for landing zones with "create collections" enabled.
Upon closer inspection, it seems we simply skip creating these "shortcut" collections in ZoneModifyMixin.submit_create(). Hence it's a missing feature.
This should be trivial to implement, so I will do so for v0.14.2.

It looks like we simply skip creating such root-level extra collections in ZoneModifyMixin.submit_create(). This means this concerns other plugins too if they define such collections.
I'll have to fix this and also make sure the way to define such collections is clear, consistent and documented.

Done.


[WIP] Feature/Error checking middleware to reduce boilerplate code

@katedee, what do you think?
Reduce boilerplate by putting error checking in middleware
Since most of our error checks are pretty similar, I built out a middleware that runs through the error check. If it finds an error, it

returns a 422 status
passes in the session as data
passes in any request parameters as body
passes in the errors as errors

Add extra check to birthdate format validation
Found a wee bug and added a check.
Previously, if the value was coming back like "1909", then the birthdate array wouldn't have a 2nd or 3rd entry, and then calling birthDateSplit[1].length would return the error message "can't call length on undefined".

Thanks! Will reconcile this with Nick's PR that just went in and then merge.


chore: connect failurePolicy to package.json onError

Description
@bburky brought up a potential security thread related to the webhook configurations's failurePolicy. Currently, the failurePolicy is always ignore and there is no way to change it, this prove be danger to a ValidatingAdmission Webook or even a Mutating that is supposed to remove capability and securityContexts because it can cause the webhoook to ignore the object and pass it into the cluster.
During npx pepr init there is a prompt about:
? How do you want Pepr to handle errors encountered during K8s operations? › - Use arrow-keys. Return to submit.
❯   Ignore - Pepr will continue processing and generate an entry in the Pepr Controller log.
    Log an audit event
    Reject the operation

Based on this response, the package.json is populated with onError section:
 "pepr": {
    "name": "pepr-test-module",
    "uuid": "static-test",
    "onError": "reject", // <----- HERE
    "alwaysIgnore": {
      "namespaces": [],
      "labels": []
    },
    "includedFiles": []
  },

If reject is present on the onError section then the failurePolicy generated during a npx pepr build will be Fail.
Related Issue
Fixes #434

Relates to #
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Other (security config, docs update, etc)

Checklist before merging

[x] Test, docs, adr added or updated as needed
[x] Contributor Guide Steps followed

This looks good. The only other note is the uptime and stability of a webhook is critical when it is failurePolicy: Fail. I think you already said the webhook pods of Pepr are HA though, so I think you're fine.


Launcher.LaunchFileAsync fails after explorer.exe has been restarted

I know this isn't the right repository for it, but I honestly don't know where else to post this.
On Windows 10 (I believe all versions but confirmed on 18363), there is some sort of core functionality that is not restarted when the main explorer.exe instance (the one hosting the start menu and the desktop) is killed (e.g. taskkill /f/ im explorer.exe) and then subsequently restarted (e.g. by running explorer.exe in the task manager's run dialog), UAP apps are no longer able to use Launcher.LaunchAsync() to launch files associated with non-UAP applications.
e.g.

before restarting explorer.exe, trying to open foo.jpg or foo.pdf, Launcher.LaunchAsync returns true
after restarting explorer.exe, trying to open foo.jpg associated with the UAP photos app: Launcher.LaunchAsync() returns true
after restarting explorer.exe, trying to open foo.pdf associated with Adobe Acrobat DC (non-UAP app), Launcher.LaunchAsync() returns false

@mqudsi This is an internal Windows bug, so you should report this to Microsoft via Feedback hub.

Thanks for the report, we're looking at this via http://task.ms/26890318

Thanks, @jonwis. If at all possible, I'd appreciate a tip on the resolution, whenever that may be.

fwiw, it also happens if explorer was exited cleanly (via ctrl+shift+right-click on taskbar followed by "Exit Explorer") then restarted.

Hi @mqudsi - thanks for filing this issue. We have investigated, but unfortunately we cannot reproduce the issue, so I suspect it might be something specific to your machine - might be worth checking you have all the latest updates, etc.

I'm still on 18363, so I guess there's hope it's been fixed in an update!


CollectionUtilTest extension

Hi,
PR consists of :

added tests for method jodd.util.CollectionUtil#asStream(Iterator iterator)
licence added for jodd.util.CollectionUtilTest

Bye,
Sascha

Thank you! Btw, I thought we have fixed the missing license issue with the gradle task, yet missing header appears from time to time, have to make a mental note to check that :)


Can't see list of records

for some reason I can add/edit records but i can't any list of records in my panel.
I can't even see the list of the default tables, i.e. Links, Roles, Permissions, Admin, but again I can add or edit records in every tables. In my composer.json the require parts are these:
"require": {         "php": ">=5.5.9",         "laravel/framework": "5.2.*",         "laravelcollective/html": "5.2.*",         "james-heinrich/getid3": "^1.9",         "serverfireteam/panel": "^1.5"     },     "require-dev": {         "fzaninotto/faker": "~1.4",         "mockery/mockery": "0.9.*",         "phpunit/phpunit": "~4.0",         "phpspec/phpspec": "~2.1",         "symfony/css-selector": "2.8.*|3.0.*",         "symfony/dom-crawler": "2.8.*|3.0.*"     },
and in config/app.php my providers are these:
`    'providers' => [
    /*
     * Laravel Framework Service Providers...
     */
    Illuminate\Auth\AuthServiceProvider::class,
    Illuminate\Broadcasting\BroadcastServiceProvider::class,
    Illuminate\Bus\BusServiceProvider::class,
    Illuminate\Cache\CacheServiceProvider::class,
    Illuminate\Foundation\Providers\ConsoleSupportServiceProvider::class,
    Illuminate\Cookie\CookieServiceProvider::class,
    Illuminate\Database\DatabaseServiceProvider::class,
    Illuminate\Encryption\EncryptionServiceProvider::class,
    Illuminate\Filesystem\FilesystemServiceProvider::class,
    Illuminate\Foundation\Providers\FoundationServiceProvider::class,
    Illuminate\Hashing\HashServiceProvider::class,
    Illuminate\Mail\MailServiceProvider::class,
    Illuminate\Pagination\PaginationServiceProvider::class,
    Illuminate\Pipeline\PipelineServiceProvider::class,
    Illuminate\Queue\QueueServiceProvider::class,
    Illuminate\Redis\RedisServiceProvider::class,
    Illuminate\Auth\Passwords\PasswordResetServiceProvider::class,
    Illuminate\Session\SessionServiceProvider::class,
    Illuminate\Translation\TranslationServiceProvider::class,
    Illuminate\Validation\ValidationServiceProvider::class,
    Illuminate\View\ViewServiceProvider::class,
    Collective\Html\HtmlServiceProvider::class,
    App\Providers\AppServiceProvider::class,
    App\Providers\AuthServiceProvider::class,
    App\Providers\EventServiceProvider::class,
    App\Providers\RouteServiceProvider::class,
    Serverfireteam\Panel\PanelServiceProvider::class,
],`

is it my fault, or I need something else to run LaravelPanel properly?

Is not your fault , it was a bug , do a composer update and it will work


Add gke-gcloud-auth-plugin to dockerfile

Motivation

This PR adds gke-gcloud-auth-plugin to turing-api can access gke clusters without having to provide static credentials (ie client cert key, private key)
This PR can be merged without affecting any existing logic

Reason:

GKE clusters now support authentication via plugin

gke-gcloud-auth-plugin is a go binary. Source code can be found here

There are 3 ways to install the plugin:

via gcloud cli: gcloud components install gke-gcloud-auth-plugin --quiet
via go install <package>

Howver, go install <plugin> returns this error

➜ go install github.com/kubernetes/cloud-provider-gcp/cmd/gke-gcloud-auth-plugin@929b1354f904a55d42389ae2a75021
042e62e40f
go: downloading github.com/kubernetes/cloud-provider-gcp v0.0.0-20221223184326-929b1354f904
go: github.com/kubernetes/cloud-provider-gcp/cmd/gke-gcloud-auth-plugin@929b1354f904a55d42389ae2a75021042e62e40
f (in github.com/kubernetes/cloud-provider-gcp@v0.0.0-20221223184326-929b1354f904):
        The go.mod file for the module providing named packages contains one or
        more replace directives. It must not contain directives that would cause
        it to be interpreted differently than if it were the main module.

via apt-get install google-cloud-sdk-gke-gcloud-auth-plugin

The go binary is built using go-alpine image, so this approach is not suitable as apt-get is only applicable to debian images

Final approach uses gcloud-sdk-cli:alpine image as a build stage to download the gke-gcloud-auth-plugin binary, and copy the binary into the final image.
The only disadvantage of installing gke-gcloud-auth-plugin this way is that gcloud cli is not available. gke-gcloud-auth-plugin defaults to using gcloud to retrieve gcp access token.
To get around this, the argument --use_appliaction_default_credentials needs to be supplied when environment config used in
https://github.com/caraml-dev/turing/pull/303

gke-gcloud-auth-plugin --use_application_default_credentials

@shydefoo thanks for the detailed PR description! When is this commend expected to be run? Is it

To get around this, the argument --use_appliaction_default_credentials needs to be supplied when environment config used in ...

I don't see this flag being supplied in https://github.com/caraml-dev/turing/pull/303. Is that PR still WIP ?

@shydefoo thanks for the detailed PR description! One question:

To get around this, the argument --use_appliaction_default_credentials needs to be supplied when environment config used in ...

I don't see this flag being supplied in #303. Is that part still WIP ?

This flag is supposed to be applied in the configuration directly, or helm values.yaml. For the github workflow e2e tests that connect to a k3d cluster, we use client certificates in the K8sConfig, not the gke-gcloud-auth-plugin.
I've just added some examples to config-dev.yaml and environments-dev.yaml in #303


Delete webpanel functions

Webpanel has been offline for like 2 years now; why still continue to have webpanel scripts that are basically just abandonware?

pof:


@orta: Displays thumbnail on detail view while fullsize image is being downloaded

Fixes #429.

:+1: - might also make sense to do this as a category, I think we do that in folio/eigen


It is possible to create unreferenced data entites, resulting in invalid crates

According to the specification, every Data Entity MUST be linked to the Root Data Entity:

Where files and folders are represented as Data Entities in the RO-Crate JSON-LD, these MUST be linked to, either directly or indirectly, from the Root Data Entity using the hasPart property.

Adding a Data Entity like this is valid according to the library:
crate.addDataEntity(new DataSetEntity.DataSetBuilder().setId("./some_file").build(), false);

Now at this point, the crate is not valid. If you add a DataSetEntity that has a hasPart property linking to ./some_file and the Root Data Entity has a hasPart property linking to this DataSetEntity it would be valid again.

For clarification:

The builder's builder.addDataEntity(DataEntity) does add it correctly.
The crate's crate.addDataEntity does this only optionally, with crate.addDataEntity(DataEntity, true). It does not if the second parameter is false or without it (crate.addDataEntity(DataEntity)).

At first glance, it seems to be used with the boolean being false only once in the whole code, at the RoCrateReader, for each DataEntity. Currently, I do not understand why, but I assume that it is because in this case, it is already inside. I think we should:

[ ] change the default from "not being added" to "being added" at least.
[ ] But ideally, we should figure out if we can remove this "feature" completely in the end. For example, if this is about not adding something twice, we may use a set in the background (this is already the case!) and simply add it anyway. It should not change anything, then.

🎉


Exception: Request failed. Error was: Backend Error

I'm getting this error for every request, get or put. I don't know why. The API console only knows this.

Methods
Error codes

datastore.datasets.commit
500

datastore.datasets.lookup
500

What adds to the confusion is yet another gcloud, which is really gcloud-cli-go from what I can tell.
To reproduce this bug, disable a project in Application Settings in the old Appengine Dashboard. This function does not seem to be available in the new Cloud Console. Run the [steps] (https://cloud.google.com/datastore/docs/activate#accessing_an_existing_app_engine_datastore_from_another_platform) to access a Datastore tied to an Appengine project, in particular the Wizard to enable the API.
from gcloud import datastore
a = datastore.get_dataset('app-id'<EMAIL_ADDRESS>'path/key')

a.entity('item').save()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/entity.py", line 244, in save
    exclude_from_indexes=self.exclude_from_indexes())
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 425, in save_entity
    result = self.commit(dataset_id, mutation)
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 329, in commit
    datastore_pb.CommitResponse)
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 81, in _rpc
    data=request_pb.SerializeToString())
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 59, in _request
    raise Exception('Request failed. Error was: %s' % content)
Exception: Request failed. Error was: Backend Error

# project re-enabled
a.entity('item').save()
<Entity[{'kind': u'item', 'id':<PHONE_NUMBER>244544L}] {}>

What adds to the confusion is yet another gcloud, which is really gcloud-cli-go from what I can tell.
To reproduce this bug, disable a project in Application Settings in the old Appengine Dashboard. This function does not seem to be available in the new Cloud Console. Run the [steps] (https://cloud.google.com/datastore/docs/activate#accessing_an_existing_app_engine_datastore_from_another_platform) to access a Datastore tied to an Appengine project, in particular the Wizard to enable the API.
from gcloud import datastore
a = datastore.get_dataset('app-id'<EMAIL_ADDRESS>'path/key')

a.entity('item').save()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/entity.py", line 244, in save
    exclude_from_indexes=self.exclude_from_indexes())
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 425, in save_entity
    result = self.commit(dataset_id, mutation)
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 329, in commit
    datastore_pb.CommitResponse)
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 81, in _rpc
    data=request_pb.SerializeToString())
  File "/home/user/.local/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 59, in _request
    raise Exception('Request failed. Error was: %s' % content)
Exception: Request failed. Error was: Backend Error

# project re-enabled
a.entity('item').save()
<Entity[{'kind': u'item', 'id':<PHONE_NUMBER>244544L}] {}>

Thanks for giving me a way to replicate @pdknsk.
I added a debug statement before the error is raised and actually printed the body and headers of the response.

With writes disabled, but the app still enabled I got
('The API call datastore_v4.Commit() is temporarily unavailable: Datastore '
 'writes have been disabled by an application administrator. Writes can be '
 're-enabled in the admin console.')
# Word-wrapping by me

{
  "status": "403",
  "alternate-protocol": "443:quic,p=0.02",
  "content-length": "178",
  "x-xss-protection": "1; mode=block",
  "x-content-type-options": "nosniff",
  "transfer-encoding": "chunked",
  "expires": "Fri, 02 Jan 2015 18:28:40 GMT",
  "vary": "Origin, X-Origin",
  "server": "GSE",
  "-content-encoding": "gzip",
  "cache-control": "private, max-age=0",
  "date": "Fri, 02 Jan 2015 18:28:40 GMT",
  "x-frame-options": "SAMEORIGIN",
  "content-type": "text/html; charset=UTF-8"
}

With the app disabled (as @pdknsk suggests) I got
'Backend Error'

{
  "status": "503",
  "alternate-protocol": "443:quic,p=0.02",
  "content-length": "13",
  "x-xss-protection": "1; mode=block",
  "x-content-type-options": "nosniff",
  "transfer-encoding": "chunked",
  "expires": "Fri, 02 Jan 2015 18:31:42 GMT",
  "vary": "Origin, X-Origin",
  "server": "GSE",
  "-content-encoding": "gzip",
  "cache-control": "private, max-age=0",
  "date": "Fri, 02 Jan 2015 18:31:42 GMT",
  "x-frame-options": "SAMEORIGIN",
  "content-type": "text/html; charset=UTF-8"
}

Do you think we should also print the status code? Here the information was 403 Forbidden when writes aren't allowed vs. 503 Service Unavailable when the app is turned off.
Do you think the backend should return more than just Backend Error if the app is turned off? FWIW I also tried this on apps that don't exist and apps that my credentials can't see and in both cases got a non-descript error and a 503. This may indicate that the error is a security protection.

I don't know how this works behind the scenes, but I've been thinking that the Datastore should perhaps be able to operate separately from App Engine. Storage does. Every App Engine project also gets a default bucket named app-id.appspot.com which can be accessed with this API even when the app is disabled. I just tried.

@pdknsk Thanks for the feedback.
@silvolu What's the right channel for this?


Rebuild for RHSA-2022:6206

https://access.redhat.com/errata/RHSA-2022:6206
Description
A detailed explanation of the changes in your PR.
Feel free to remove this section if it is overkill for your PR, and the title of your PR is sufficiently descriptive.
Checklist

[ ] Investigated and inspected CI test results
[ ] Updated documentation accordingly

Automated testing

[ ] Added unit tests
[ ] Added integration tests
[ ] Added regression tests

If any of these don't apply, please comment below.
Testing Performed
TODO(replace-me)
Use this space to explain how you tested your PR, or, if you didn't test it, why you did not do so. (Valid reasons include "CI is sufficient" or "No testable changes")
In addition to reviewing your code, reviewers must also review your testing instructions, and make sure they are sufficient.
For more details, ref the Confluence page about this section.

The errors in the integration tests are unrelated to the change, errors are:

VMs no longer available in GCP
Kernel modules not built for some versions needing a backported fix.
One error on the container-stats container.

In the interest of moving 3.8.3 along, I will merge the PR as is.


feat: experimental /_nitro/openapi.json and /_nitro/swagger endpoints for dev mode

🔗 Linked issue
Resolves #3
Resolved #4
❓ Type of change

[ ] 📖 Documentation (updates to the documentation or readme)
[ ] 🐞 Bug fix (a non-breaking change that fixes an issue)
[ ] 👌 Enhancement (improving an existing functionality like performance)
[x] ✨ New feature (a non-breaking change that adds functionality)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description
This PR exposes available routes and an initial meta using Open API 3 standard.
It exposes two endpoints for development:

/_nitro/openapi.json: JSON response that can be integrated with external tools and setups such as Postman and Nuxt devtools to access available routes meta (/cc @antfu)
/_nitro/swagger: A basic UI that makes local debugging easier

This feature opens the next steps in supporting generic and extendable route meta via a well-defined standard.

📝 Checklist

[ ] I have linked an issue or discussion.
[ ] I have updated the documentation accordingly.

That would be great! 🔥

Hot damn, I forgot I even commented on that issue but it has just become relevant again for a project. Will definitely check it out ASAP (which might sadly not be too soon as I'm quite busy :/)!

Great feature!
It seems that the port 3000 in swagger is hardcoded, and the the port used in Nuxt. I am using port 3001 for this application, and the try out calls, won't work that way.
Is this a bug, or just wrong configuration of nitro on my part?

I guess https://github.com/unjs/nitro/blob/4111798a09fde7bf5916500637e27d6acc0189d1/src/runtime/routes/openapi.ts#L20 should reuse the host+port where the application is exposed

I have found some problems using this.
Generating the json works, although with some errors. When opening the swagger page I get a json syntax error. I believe this comes from swagger itself. When copying the generated schema into the online swagger editor it has errors. The versioning and parameters are missing an option.
https://github.com/unjs/nitro/blob/2ad43894a6039207fd535d7fdc30ba92d4847ee6/src/runtime/routes/openapi.ts#L16
In the code the version is given as null but the openapi specs are given this as required
Regarding the parameters openapi states this:
A parameter MUST contain either a schema property, or a content property, but not both.

See here
When generating the paths, these properties are not set
https://github.com/unjs/nitro/blob/2ad43894a6039207fd535d7fdc30ba92d4847ee6/src/runtime/routes/openapi.ts#L42
It would be great if there would an option we can define openapi properties for ourself. Maybe directly in the routes via a openapi config object that gets automatically picked up by the feature?

I still don't understand how to use it, can you give me a sample code?

I still don't understand how to use it, can you give me a sample code?

Just update your Nitro config:
// https://nitro.unjs.io/config
export default defineNitroConfig({
  experimental: {
    openAPI: false,
  },
})

Hey there.
How do we define the params for routes?
I tried following the openapi specification and it doesn't update the UI or JSON.
import _ from 'lodash'
/**
 * @swagger
 * /api/accounts:
 *   get:
 *     summary: Retrieve a list of accounts
 *     description: Get all users from the system
 *     responses:
 *       '200':
 *         description: A list of accounts
 *         content:
 *           application/json:
 *             schema:
 *               type: array
 *               items:
 *                 type: object
 *                 properties:
 *                   id:
 *                     type: integer
 *                   username:
 *                     type: string
 *                   balance:
 *                     type: string
 */
export default defineEventHandler(async (e) => {
  try {
    let { limit, page } = e.context
    let params = getQuery(e)
    const query = buildQuery(params)
    const fieldsToPopulate = [
      { from: 'users', localField: 'user' },
      { from: 'banks', localField: 'bank' },
      { from: 'branches', localField: 'branch' },
      { from: 'transactions', localField: 'transactions' },
    ]
    const pipeline = buildPipeline(query, page, limit, fieldsToPopulate)
    const results = await Account.aggregate(pipeline)
    let { data, totalCount, pageCount } = results[0]
    if (!_.isEmpty(totalCount)) {
      totalCount = totalCount[0].total
    }
    const response = {
      meta: {
        page,
        pageCount,
        totalCount,
      },
      data,
    }
    return response
  } catch (error) {
    console.log({
      error,
    })
  }
})

These routes don't get updated that that codedoc/comment.
http://localhost:3005/_nitro/swagger
http://localhost:3005/_nitro/openapi.json

Apologize if that's a WIP and is expected to not work yet. I read through the commit but wasn't sure as I'm not an expert on the openapi spec.
TIA~!

Also are there plans to enable configuring which openapi.json file to use?
If I understand correctly, it's generated via Nitro.
If thats the case then my previous comment is brought up. Regardless, I feel it makes the code too verbose.
I checked the discussions tab and only found this.
My use case is I generating/build routes programmatically and would prefer to generate/build the openapi.json as well and then have nitro point toward that if possible.

How do we define the params for routes?
Apologize if that's a WIP and is expected to not work yet.

It is still an experimental feature and to my knowledge there is no easy mechanism to hook into the generation, yet.

Also are there plans to enable configuring which openapi.json file to use?

If you do not want to use the autogenerated one you could just serve a static file and register the UI endpoint manually.

My use case is I generating/build routes programmatically and would prefer to generate/build the openapi.json as well and then have nitro point toward that if possible.

I assume that once https://github.com/unjs/nitro/issues/938 gets merged these features could be combined to infer the types of the OpenAPI spec automatically.

In the v2.4.0 release notes it states We are working to bring route-level meta definition for an even better and more detailed docs generator. I was wondering what the status of that is?

I would also like to be updated on the status of adding the route level meta definitions.

I don't know if this is a good place to report this but, I am using this feature in a Nuxt project and it works as long as I'm using npm. When I run an identical project in Bun, I get the "Unable to render this definition. The provided definition does not specify a valid version field." Though openapi.json is requested and looks the same.

Hey there.
How do we define the params for routes?
I tried following the openapi specification and it doesn't update the UI or JSON.
import _ from 'lodash'
/**
 * @swagger
 * /api/accounts:
 *   get:
 *     summary: Retrieve a list of accounts
 *     description: Get all users from the system
 *     responses:
 *       '200':
 *         description: A list of accounts
 *         content:
 *           application/json:
 *             schema:
 *               type: array
 *               items:
 *                 type: object
 *                 properties:
 *                   id:
 *                     type: integer
 *                   username:
 *                     type: string
 *                   balance:
 *                     type: string
 */
export default defineEventHandler(async (e) => {
  try {
    let { limit, page } = e.context
    let params = getQuery(e)
    const query = buildQuery(params)
    const fieldsToPopulate = [
      { from: 'users', localField: 'user' },
      { from: 'banks', localField: 'bank' },
      { from: 'branches', localField: 'branch' },
      { from: 'transactions', localField: 'transactions' },
    ]
    const pipeline = buildPipeline(query, page, limit, fieldsToPopulate)
    const results = await Account.aggregate(pipeline)
    let { data, totalCount, pageCount } = results[0]
    if (!_.isEmpty(totalCount)) {
      totalCount = totalCount[0].total
    }
    const response = {
      meta: {
        page,
        pageCount,
        totalCount,
      },
      data,
    }
    return response
  } catch (error) {
    console.log({
      error,
    })
  }
})

These routes don't get updated that that codedoc/comment.
http://localhost:3005/_nitro/swagger
http://localhost:3005/_nitro/openapi.json

Apologize if that's a WIP and is expected to not work yet. I read through the commit but wasn't sure as I'm not an expert on the openapi spec.
TIA~!

It's 2024, but still no news about that

I'm nearly driven to despair by @scalar/scalar, a persistent hardcoded bug that's remained unresolved for months. If this resolves it, I'll be immensely pleased.


Add force close option to powertoys run

Description of the new feature / enhancement
when running powertoys run to find applications currently there is 4 icons 1. run as an admin 2. run as another user 3. open folder 4. copy path in console
the request is to add another icon to force close the application
Scenario when this would be used?
inset of opening task manager navigate to desired application right click and select end task just having a kill switch in run will make it much easier to do
Supporting information
No response

Hi @moses-evergreen
This feature is provided by the Window Walker plugin: https://learn.microsoft.com/windows/powertoys/run#window-walker-plugin
Can you take a look?
/needinfo

Hi @moses-evergreen
This feature is provided by the Window Walker plugin: https://learn.microsoft.com/windows/powertoys/run#window-walker-plugin Can you take a look?
/needinfo

Hi dave it seems you are right this is an option the walker plugin already it was my bad i did not notice it
however maybe it an indication of the immense number of options available and with this suggestion it will be a little streamlined to have all app related options in the same query instead of multiple terms
but you are right the functionally is exactly what I meant


Timestamp of log entry is not getting created

I have tested it by running examples/basic.go and it is not working. Am I missing something?

Nevermind... I have figured it out...


Update Get-SPEnterpriseSearchCrawlCustomConnector.md

Update example/formatting.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

sharepoint/sharepoint-ps/sharepoint-server/Get-SPEnterpriseSearchCrawlCustomConnector.md
:white_check_mark:Succeeded
View (sharepoint-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Error fetching keys from keyserver

Hello,
During installation I receive the following error when debian-libseccomp-update.yml is handled by the ansible script
TASK [Add Buster backports apt key.] ******************************************************************************************************************************
failed: [<IP_ADDRESS>] (item=04EE7237B7D453EC) => {"ansible_loop_var": "item", "changed": false, "cmd": "/usr/bin/apt-key adv --no-tty --keyserver keyserver.ubuntu.com --recv 04EE7237B7D453EC", "item": "04EE7237B7D453EC", "msg": "Error fetching key 04EE7237B7D453EC from keyserver: keyserver.ubuntu.com", "rc": 2, "stderr": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: Server indicated a failure\n", "stderr_lines": ["Warning: apt-key output should not be parsed (stdout is not a terminal)", "gpg: keyserver receive failed: Server indicated a failure"], "stdout": "Executing: /tmp/apt-key-gpghome.iOC1uIfC5w/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv 04EE7237B7D453EC\n", "stdout_lines": ["Executing: /tmp/apt-key-gpghome.iOC1uIfC5w/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv 04EE7237B7D453EC"]}
failed: [<IP_ADDRESS>] (item=648ACFD622F3D138) => {"ansible_loop_var": "item", "changed": false, "cmd": "/usr/bin/apt-key adv --no-tty --keyserver keyserver.ubuntu.com --recv 648ACFD622F3D138", "item": "648ACFD622F3D138", "msg": "Error fetching key 648ACFD622F3D138 from keyserver: keyserver.ubuntu.com", "rc": 2, "stderr": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: Server indicated a failure\n", "stderr_lines": ["Warning: apt-key output should not be parsed (stdout is not a terminal)", "gpg: keyserver receive failed: Server indicated a failure"], "stdout": "Executing: /tmp/apt-key-gpghome.60gZR0SUhA/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv 648ACFD622F3D138\n", "stdout_lines": ["Executing: /tmp/apt-key-gpghome.60gZR0SUhA/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv 648ACFD622F3D138"]}

When I check if I can fetch the keys manually I receive a "not found" message from keyserver.ubuntu.com
ansible version: 2.11.2
pyrthon3 version: 3.7.2 (default)
raspberry PI 4 with buster light (may 2021) fully updated

Hey,
Did you check if your Pi could ping the kyserver.ubuntu.com?
Thanks,
MiguelNdeCarvalho

Hey,
I did check that first. Time-out on three different networks (private, work and cell phone). The website (https) is reachable and replies with a "not found" after entering the two keys.
Thanks,
Rene

I've got the same error problem see below on error msg.  I cannot ping on my pi or locally on my mac in terminal.  ~ % ping kyserver.ubuntu.com
ping: cannot resolve kyserver.ubuntu.com: Unknown host
Error on playbook run:
TASK [Add Buster backports apt key.] ******************************************************************************
failed: [<IP_ADDRESS>] (item=04EE7237B7D453EC) => {"ansible_loop_var": "item", "changed": false, "cmd": "/usr/bin/apt-key adv --no-tty --keyserver keyserver.ubuntu.com --recv 04EE7237B7D453EC", "item": "04EE7237B7D453EC", "msg": "Error fetching key 04EE7237B7D453EC from keyserver: keyserver.ubuntu.com", "rc": 2, "stderr": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: Server indicated a failure\n", "stderr_lines": ["Warning: apt-key output should not be parsed (stdout is not a terminal)", "gpg: keyserver receive failed: Server indicated a failure"], "stdout": "Executing: /tmp/apt-key-gpghome.nLjFZURuf4/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv 04EE7237B7D453EC\n", "stdout_lines": ["Executing: /tmp/apt-key-gpghome.nLjFZURuf4/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv 04EE7237B7D453EC"]}
failed: [<IP_ADDRESS>] (item=648ACFD622F3D138) => {"ansible_loop_var": "item", "changed": false, "cmd": "/usr/bin/apt-key adv --no-tty --keyserver keyserver.ubuntu.com --recv 648ACFD622F3D138", "item": "648ACFD622F3D138", "msg": "Error fetching key 648ACFD622F3D138 from keyserver: keyserver.ubuntu.com", "rc": 2, "stderr": "Warning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: keyserver receive failed: Server indicated a failure\n", "stderr_lines": ["Warning: apt-key output should not be parsed (stdout is not a terminal)", "gpg: keyserver receive failed: Server indicated a failure"], "stdout": "Executing: /tmp/apt-key-gpghome.ESUQK9zPhu/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv 648ACFD622F3D138\n", "stdout_lines": ["Executing: /tmp/apt-key-gpghome.ESUQK9zPhu/gpg.1.sh --no-tty --keyserver keyserver.ubuntu.com --recv 648ACFD622F3D138"]}

Hey,
Can you edit the content of this line
https://github.com/geerlingguy/internet-pi/blob/f1da3fccd2f162b3cc95fd8d00e8d51c1b18a753/tasks/debian-libseccomp-update.yml#L4 to: keyserver: hkp://keyserver.ubuntu.com:80 and try to run it again, please?
Thanks,
MiguelNdeCarvalho

Hello Miguel,
Adding HTTP keysaerver protocol did the trick. The script runs fine now and the keys are added.
TASK [Add Buster backports apt key.] ******************************************************************************************************************************
changed: [<IP_ADDRESS>] => (item=04EE7237B7D453EC)
changed: [<IP_ADDRESS>] => (item=648ACFD622F3D138)
Thanks for the answer and work.
Rene

that worked perfectly - TY!!!! @MiguelNdeCarvalho

Hey,
I will do a a PR whenever I can. Can you confirm me how you fixed it please?
Thanks,
MigielNdeCarvalho

Changed this line to: as noted above
keyserver: hkp://keyserver.ubuntu.com:80
needed to put a CRLF at the end of the line as well but I think thats how the config roll anyways in ansible.

Hey,
I will do the PR ASAP.
Thanks,
MiguelNdeCarvalho


Updated tests

@ebdrup added new tests and updated meta-schema to match with the changes in JSON-Schema org  (https://github.com/json-schema-org/json-schema-spec/blob/dba92b702c94858162f653590230e7573c8b7dd0/schema.json)
Quite a few validators failed additional tests...

You should make the new changed test not required to be included in the benchmark, so the list of tested schema validators remains the same.
What has actually changed in the spec? To me it seems weird that they would change an old spec like that.

Oh @epoberezkin BTW, thanks a lot for the contribution. Could you make the PR in a way where all the reposts changing are not part of the PR. Then I can run the benchmarks myself? That would also make it a lot easier to review the change.

What has actually changed in the spec? To me it seems weird that they would change an old spec like that.

@ebdrup The spec hasn't really changed - only the tests were added to provide a better coverage to the spec.

Can you elaborate please, what's this change about? I see it breaks some tests
Thanks!

@korzio This is the draft-04 meta-schema that was updated in json-schema org, as it wasn't correct previously. Schema ID should be uri-reference, but this format was only added in the later versions of the spec, so "format" keyword was just removed. "uri" format is much stricter than the requirement to "id". Not sure how any tests can fail because of removing it - Could you clarify which tests failed?

Could you make the PR in a way where all the reposts changing are not part of the PR. Then I can run the benchmarks myself? That would also make it a lot easier to review the change.

@ebdrup There were two commits - the first is the change, the second - the updated reports. I removed the second.

@epoberezkin Thanks for comment, will have a look at my package

@epoberezkin Do you know, are there plans to release a new npm package version of JSON-Schema-Test-Suite - https://github.com/json-schema-org/JSON-Schema-Test-Suite#nodejs
?
My validator tests depend on this package right now, and there are bunch of changes in the release branch
https://github.com/json-schema-org/JSON-Schema-Test-Suite/compare/86f965e53dda0b6c57e70ddd726243e1e061cf84...15ba997f9b937150a0ab88244d1d0fbf58526c48

I use the submodule - probably worth raising an issue there to auto-release all merged changes

https://github.com/korzio/djv/issues/78

I can't merge this until this is addressed:
You should make the new changed test not required to be included in the benchmark, so the list of tested schema validators remains the same.

I'm currently working on an update here instead:https://github.com/ebdrup/json-schema-benchmark/pull/44
I'll close this PR.
I'm giving djv a chance to fix all the failing tests before I'll merge the PR.


Possible MintQuote Bug - Checking both asset amounts are equal in case of new mint

In case the liquidity is 0, the code compares the input asset amounts instead of checking that they are non null values.
https://github.com/geoffodonnell/dotnet-tinyman-sdk/blob/main/src/Tinyman/V1/Model/PoolExtensions.cs#L183
// Pool exists does not contain assets
} else {

if (amount1 != amount2) {
	throw new Exception("Amounts required for both assets for first mint!");
}

liquidityAssetAmount = Convert.ToUInt64(
	Math.Sqrt((double)BigInteger.Multiply(amount1.Amount, amount2.Amount)) - 1000);
slippage = 0;
}

This should be resolved in v<IP_ADDRESS>.


[Blazor][Server-side] Limit the rate at which events can be delivered and the max size for events

Do you have any data or examples of tests that help us understand the priority of this?

In process, so far with a client I can cause exceptions on the server
fail: Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher[8]
      Failed to invoke hub method 'BeginInvokeDotNetFromJS'.
System.IO.InvalidDataException: Error binding arguments. Make sure that the types of the provided values match the types of the hub method being invoked.
 ---> System.IO.InvalidDataException: Deserializing object of the `String` type for 'argument' failed.
 ---> System.IO.EndOfStreamException: Attempted to read past the end of the stream.
   at MessagePack.MessagePackReader.GetStringLengthInBytesSlow(Byte code)
   at Microsoft.AspNetCore.Components.Server.BlazorPack.BlazorPackHubProtocol.DeserializeObject(MessagePackReader& reader, Type type, String field) in C:\work\AspNetCore\src\Components\Server\src\BlazorPack\BlazorPackHubProtocol.cs:line 474
   --- End of inner exception stack trace ---
   at Microsoft.AspNetCore.Components.Server.BlazorPack.BlazorPackHubProtocol.DeserializeObject(MessagePackReader& reader, Type type, String field) in C:\work\AspNetCore\src\Components\Server\src\BlazorPack\BlazorPackHubProtocol.cs:line 474
   at Microsoft.AspNetCore.Components.Server.BlazorPack.BlazorPackHubProtocol.BindArguments(MessagePackReader& reader, IReadOnlyList`1 parameterTypes) in C:\work\AspNetCore\src\Components\Server\src\BlazorPack\BlazorPackHubProtocol.cs:line 258
   --- End of inner exception stack trace ---
   at Microsoft.AspNetCore.Components.Server.BlazorPack.BlazorPackHubProtocol.BindArguments(MessagePackReader& reader, IReadOnlyList`1 parameterTypes) in C:\work\AspNetCore\src\Components\Server\src\BlazorPack\BlazorPackHubProtocol.cs:line 265
   at Microsoft.AspNetCore.Components.Server.BlazorPack.BlazorPackHubProtocol.CreateInvocationMessage(MessagePackReader& reader, IInvocationBinder binder, Int32 itemCount) in C:\work\AspNetCore\src\Components\Server\src\BlazorPack\BlazorPackHubProtocol.cs:line 123

And I get the memory from the server to grow up to 3GB

In process, so far with a client

Doesn't SignalR enforce a default limit on the message size from a client?
How will an external client run code in process on a deployed application?

In process, so far with a client

I meant with 1 client.

Doesn't SignalR enforce a default limit on the message size from a client?

Yes, I'm playing within those rules. Sending events with 31K

Ok good, this seems relevant then 👍

Closing this as we've found out that things look good enough and we don't need to do anything here.


DOC: readme and contributing doc cleanup

Took a pass through README.rst and CONTRIBUTING.md, updating existing text to be more succinct and adding new info relevant to scikit-bio developers. Instructions for installing the development version of scikit-bio are now included in CONTRIBUTING.md instead of README.rst.
Fixes #850. Addresses part of #867.
@gregcaporaso @ebolyen @Jorge-C can you please review?

Looks good, just one comment.

Thanks for reviewing! I addressed your comment.

:+1:

Looks like all comments have been addressed, merging. Thanks!


Buttons no longer working

The 'Enable Live Updates' and 'Expand Table' buttons are no longer working. I see no errors in the console and nothing happens to the page when either button is clicked. I noticed during a refresh that 'Disable Live Updates' changed to 'Enable Live Updates'.
There seem to be a lot of changes going on. Can untested changes be pushed to a development branch instead of master? I sent this site to many people and they're all asking me why it is constantly having different issues.
Windows 10
Edge (latest) - the behavior is the same in regular mode as well as incognito mode
It was working 30 minutes ago. The last real commit has a lot of changes, but I can't tell if any are related to the current behavior.

They just started working again right after I posted this. Closing for now.


Upgrade to new findbugs groupId

Findbugs libraries (annotations and jsr305) have been relocated from net.sourceforge.findbugs to com.google.code.findbugs.
This causes duplicates/conflicts between libraries depending on codegen, such as querydsl-jpa, and up-to-date libraries such as guava.
Sincerely,
Stéphane Landelle

Another question : why use both annotations (the tentative) and jsr305 (the official jsr)?

I removed the jsr305 annotations completely. I will further continue to slim down the codegen dependencies.

released in 0.4.8

Thanks!
Do you plan on doing the same thing for querydsl?
Stephane
2012/2/5 Timo Westkmper <
<EMAIL_ADDRESS>

released in 0.4.8

Reply to this email directly or view it on GitHub:
https://github.com/mysema/codegen/issues/1#issuecomment-3820342

Maybe in the future.


Release individual crates upon merge into master

Right now, the CI job is releasing creates when it finds a ci-release-tag on the commit. However, we want the job to run on every merge into master and release all crates where versions have been changed compared to what is released on crates.io at the moment. From a technical perspective this only requires switching the unleash job to run on master, but before we can do that we have to consider a few aspects:

the unleash-check-CI job is currently disabled (I don't know why) but must run the release job, at least on the PR that intends to bump versions, to ensure things aren't breaking (ideally on every PR to make sure we've catch bugs early and not only when we intend to release it)
can we enforce that changelogs are not only kept but upgrade instructions from the unreleased-section are moved properly? Maybe through a helper job and a flag on cargo unleash that checks the title isn't present upon release?
can we provide helpers to the developers to know which bump they have to do (MAJOR, MINOR, etc)
what is our tagging-story for these releases? Do we want to create a tag per each crate-x.x.x-combo? Would make it easier to find chances to apply hot-fixes, but also pretty messy huge list of tags in this repo?
How do we prevent upgrade-fatigue? How about upgrade-incompatibilities? If we release very often, maybe even bumping major versions, this requires downstream tooling to upgrade at least their Cargo.toml as well. Can we help with dependabot-setups to make sure everyone is on latest? How do we deal with the problem that some downstream tooling might needs a while to upgrade to latest MAJOR while others does and the user that wants to combine both ends up in an unrecoverable situation (cargo doesn't know semver version ranges)?

the unleash-check-CI job is currently disabled (I don't know why) but must run the release job, at least on the PR that intends to bump versions, to ensure things aren't breaking (ideally on every PR to make sure we've catch bugs early and not only when we intend to release it)

Digging into the history of this, it seems it was temporarily deactivated to wait for an external fix that has been merged already Sept 15th – just no one re-activated the jobs properly. Gonna give it a try and see if this is a quick one to sort out.


Make Github Actions optional for eventing-istio

PRs stuck on shellchecks on git submodules.
/assign @pierDipi
(same as https://github.com/knative/test-infra/pull/3846, but test-infra repo is not used anymore)

/lgtm
/approve


Suggestions for constructing a ResNet with the revlib

Hi
I would like to use this library to build a ResNet20 model, I've tried several times but I still have the mismatched dimension error. My model is shown as follows:
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.nn.init as init

from torch.nn.modules.batchnorm import _BatchNorm
from torch.nn.modules.instancenorm import _InstanceNorm

hidden_size = [16, 32, 64]

class View(nn.Module):
    def forward(self, x):
        batch_size = x.size(0)
        return x.view(batch_size, -1)

class LambdaLayer(nn.Module):
    def __init__(self, lambd):
        super(LambdaLayer, self).__init__()
        self.lambd = lambd

    def forward(self, x):
        return self.lambd(x)

class BasicBlock(nn.Module):
    expansion = 1

    def __init__(self, in_planes, planes, stride, norm_layer, conv_layer, option='A'):
        super(BasicBlock, self).__init__()

        self.bn1 = norm_layer(in_planes)
        self.conv1 = conv_layer(in_planes, planes, kernel_size=3, stride=stride, padding=1, bias=False)
        self.bn2 = norm_layer(planes)
        self.conv2 = conv_layer(planes, planes, kernel_size=3, stride=1, padding=1, bias=False)

        self.shortcut = nn.Sequential()
        if stride != 1 or in_planes != self.expansion * planes:
            if option == 'A':
                """
                For CIFAR10 ResNet paper uses option A.
                """
                self.shortcut = LambdaLayer(lambda x:
                                            F.pad(x[:, :, ::2, ::2], (0, 0, 0, 0, planes//4, planes//4), "constant", 0))
            elif option == 'B':
                self.shortcut = conv_layer(in_planes, self.expansion * planes, kernel_size=1, stride=stride, bias=False)

    def forward(self, x):
        out = F.relu(self.bn1(x))
        shortcut = self.shortcut(out) if hasattr(self, 'shortcut') else x
        print(1, shortcut.size())
        out = self.conv1(out)
        out = self.conv2(F.relu(self.bn2(out)))
        print(2, out.size())
        out += shortcut
        return out

class ResNet(nn.Module):

    def __init__(self, hidden_size, block, num_blocks, num_classes=10, bn_type='bn',
                 share_affine=False, track_running_stats=True):
        super(ResNet, self).__init__()
        self.in_planes = 16

        self.bn_type = bn_type
        if bn_type == 'bn':
            norm_layer = lambda n_ch: nn.BatchNorm2d(n_ch, track_running_stats=track_running_stats)
        elif bn_type == 'gn':
            norm_layer = lambda n_ch: nn.GroupNorm(4, n_ch) # 3 can be changed -- # of groups
        else:
            raise RuntimeError(f"Not support bn_type={bn_type}")
        conv_layer = nn.Conv2d
        first = conv_layer(3, hidden_size[0], kernel_size=3, stride=1, padding=1, bias=False)
        layer1 = self._make_layer(block, hidden_size[0], num_blocks[0], stride=1,
                                       norm_layer=norm_layer, conv_layer=conv_layer)
        layer2 = self._make_layer(block, hidden_size[1], num_blocks[1], stride=2,
                                       norm_layer=norm_layer, conv_layer=conv_layer)
        layer3 = self._make_layer(block, hidden_size[2], num_blocks[2], stride=2,
                                       norm_layer=norm_layer, conv_layer=conv_layer)

        self.rev_layers = revlib.ReversibleSequential(*[layer1, layer2, layer3])

        norm = norm_layer(hidden_size[2] * block.expansion)
        linear = nn.Linear(hidden_size[2] * block.expansion, num_classes)

        self.full_model = nn.Sequential(first, self.rev_layers, nn.ReLU(), norm, \
                                        nn.AdaptiveAvgPool2d((None, 1)), View(), linear)

    def _make_layer(self, block, planes, num_blocks, stride, norm_layer, conv_layer):
        strides = [stride] + [1] * (num_blocks - 1)
        layers = []
        for stride in strides:
            layers.append(block(self.in_planes, planes, stride, norm_layer, conv_layer))
            self.in_planes = planes * block.expansion
        return nn.Sequential(*layers)

    def forward(self, x):
        out = self.full_model(x)
        return out

def init_param(m):
    """Special init for ResNet"""
    if isinstance(m, (_BatchNorm, _InstanceNorm)):
        m.weight.data.fill_(1)
        m.bias.data.zero_()
    elif isinstance(m, nn.Linear):
        m.bias.data.zero_()
    return m

def resnet20(**kwargs):
    model = ResNet(hidden_size, BasicBlock, [3,3,3], **kwargs)
    model.apply(init_param)
    return model

I've tried to modify self.in_planes = 8 and hidden_size = [8, 16, 32], respectively, but it still does not work. Could you provide any hints? Is it possible to build a model in a forward way instead of wrapping reversible model with non-reversible layers like model = nn.Sequential(conv, rev_layer, conv)? I appreciate your help.

For RevNet to work, you must ensure that twice as many features get passed in as each branch uses. Otherwise, RevLib can't split the input into two equal-sized inputs, one for each of the two branches.
To do this, change the output channels of first from hidden_size[0] to hidden_size[0] * 2.
Optionally, you could also tell RevLib to feed the same input into both branches by setting split_dim=None when constructing the ReversibleSequential module.
Keep in mind that not only do you need twice as many features in the input, but you also get twice as many features in the output. So, the final norm and linear both need to work on twice as many features as well.
Unfortunately, I'm unsure what you mean by "build a model in a forward way." Could you elaborate?

@ClashLuke Thank you a lot for your reply.
Doubling the output channels of first will raise RuntimeError,
File ~\Anaconda3\lib\site-packages\revlib\core.py:183, in additive_coupling_forward(other_stream, fn_out)
    181 fn_out = split_tensor_list(fn_out)
    182 if isinstance(fn_out, torch.Tensor):
--> 183     return other_stream + fn_out
    184 return [other_stream + fn_out[0]] + fn_out[1]

RuntimeError: The size of tensor a (32) must match the size of tensor b (16) at non-singleton dimension 3

I've printed the output size in the feedward of the BasicBlock, layer 1 and layer 2 work well but I can not print the output of the layer 3.
There is the other problem due the the doubling operation, the number of parameters of the RevNet will not equal to the the original ResNet. I may not fully understand this library and how RevNet works, and implement RevNet with revlib in a wrong way.
About the question  build a model in a forward way, I would like to know if I can stack revlib.ReversibleSequential(), specifically,
layer 1 = self._make_layer(...); layer 2 = self._make_layer(...); layer 3 = self._make_layer(...)
rev_layers = nn.Sequantial (layer 1, layer 2, layer 3)

or

def forward(x):
    out = layer 1 (x)
    out = layer 2 (out)
    out = layer 3 (out)

where ._make_layer(...) returns revlib.ReversibleSequential(*layers), which is different from the previous one --
self.rev_layers = revlib.ReversibleSequential([layer1, layer2, layer3]), where layer is nn.Sequantial().
I've tried this in my previous RevNet20 code, and I got the same RuntimeError but the error happens in layer 2 instead of layer 3.

The top problem you faced is that RevNet requires all inputs and outputs to be the same size. As the second layer has more output features than the first, RevNet will have to add a tensor with 32 features to one with 16, which isn't possible.
Think about it like in a ResNet. In ResNet, you only have the residual path within each resolution+feature size, but not across them. To get the residual stream across, you usually use downsampling (such as AvgPool2d) and add its output to the output of your "residual" block. In RevNet, the second thing doesn't exist. Instead, you would have to use PixelShuffle and feature padding to arrive at a similar result (see #2).
The easiest way forward would be to have multiple ReversibleSequential modules, one for each _make_layer()-call, and put these into a standard nn.Sequential-container. This is how the original RevNet did it. Their method uses marginally more parameters but otherwise gives the same results:

Another alternative would be to avoid this multi-stage assembly and construct one large ReversibleSequential module instead. Using one large block saves memory, and i-RevNet documented how they achieved marginally worse ImageNet accuracy with this kind of architecture:

Yes, you can define the reversible architecture in forward. However, I'd advise against it, as ReversibleSequential is a thin wrapper around things you have to do anyway.
If you want to do what ReversibleSequential would usually handle for you manually, you'd have to wrap your modules in ReversibleModules like so manually:
https://github.com/HomebrewNLP/revlib/blob/34dad19318e2f861ea6b0ce263506625a934b568/revlib/core.py#L471-L487
and call these modules one-by-one, just like in a normal nn.Sequential module:
https://github.com/HomebrewNLP/revlib/blob/34dad19318e2f861ea6b0ce263506625a934b568/revlib/core.py#L509-L511

Thanks a lot for your detailed explanation! Now I can understand why my code does not work.
I also read the source codes of revnet and i-revnet, and they provide a downsampling to match the dimension of other_stream and fn_out (the mismatch is caused by the changing of number of channels). If I am not misunderstand, it seems that your code does not provide this feature, right? Do you have a plan to add this feature?

Sorry, I'm not planning to add these, as the most common functions (pooling, pixelshuffle, upsample) are already part of PyTorch.


Admin Dashboard - Modules - Execute button should have validator to prevent "Choose action..." submit

Branch: Dev
http://localhost:30321/OrchardLocal/Admin/Modules/Features
If you tick several options and then click Execute then the page submits without issue:

However because Choose action... is selected, no action is taken and so the ticked modules are not acted upon.
Solution
Add a validator to prevent submit when Actions is set to Choose action...
If this requires extra coding a quick fix would be add a notifier call to show a warning saying no action taken

This is going to be fixed when the new admin theme is finished.


Traefik 2.x support for kvs atomic configs?

Sort of a question more than a bug per se.
Traefik 1.x KVS config backend had the "traefik.alias" key that could be use to make atomic updates to the whole configuration (instead of whatching the whole prefix, only the traefik.alias key is watched and then the target used as config). I'm having a hard time finding the same functionality in the 2.x versions.
Is there a similar way to achieve full atomic config updates in 2.x? If not, what is the suggested way to achieve something like that (won't traefik watches trigger an update for every key update having intermediate, wrong, configuration?).

Hi! I'm Træfiker :robot: the bot in charge of communication regulation.
Thanks for your interest in Traefik!
We dedicate the issue tracker to bug reports and feature requests only. My advanced AI has spotted that your issue might be a configuration problem or relates to something that doesn't look like a bug.
To confirm this, please join our Community Forum and reach out to us on the Traefik section.
In case I'm wrong (well, that would be embarrassing :sweat_smile:), my developers will re-open the issue and fix me!
In the meantime, you can double check Traefik's documentation.


Additional options for control inputs

Ongoing discussion for adding acceleration and steering velocity as control inputs.

Addressed in #72.


fix: Make Pinecone API key optional in dev 🚀

🔧 fix: Make Pinecone API key optional in dev 🚀
Description
The API fails to start in development due to a missing Pinecone API key, despite most features not requiring Pinecone integration to work locally.
Error Message
PineconeConfigurationError: The client configuration must have required property: apiKey. You can find the configuration values for your project in the Pinecone developer console at https://app.pinecone.io.

Steps to Reproduce

Start Docker containers:

yarn dx:up

Start applications:

yarn dev:apps

Expected Behavior
All applications should start successfully:
@oyster/member-profile:dev:   ➜  Local:   http://localhost:3000/
@oyster/member-profile:dev:   ➜  Network: use --host to expose
@oyster/admin-dashboard:dev:   ➜  Local:   http://localhost:3001/
@oyster/admin-dashboard:dev:   ➜  Network: use --host to expose
@oyster/api:dev: API is up and running! 🚀

Current Behavior
API fails to start due to missing Pinecone configuration.
Screenshots
Expected behavior:

Current behavior:

Proposed Solution
Make Pinecone configuration optional in development environment since it's not required for most local development features.

This is a duplicate of issue #505


[MNOE-783] Fix registration route not being loaded

Fix an issue when the registration is disabled in the settings.yml and
then enabled via the MnoHub settings.
Since the routes are being build based on the User#devise_modules which
is built when loading the class, the registration route is never added.
The Devise::Models::Registerable is only adding a class method to the
user model (.new_with_session), so it's ok to always add it.

"In progress" as I'll do more tests tomorrow to check if there are any unintended side effects.
Happy so far with my testing.

@fgourichon @x4d3 rebased and happy with testing. Please review.


Scoring & Exact Matches

Sometimes, exact matches don't show up as the first result. Is there a way to configure BRFullTextSearch/Lucene to make sure that exact matches show up as the first result? Sometimes it's the first one and sometimes not.

Can you provide an example test case? By default Lucene is sorting by relevance, and "exact match" can be different from what you expect, for example when stemming is enabled.
You might also find it useful to use a tool like Luke (you can get an old version that uses a 2.3.x version of Lucene to be more similar to CLucene) to debug your queries and see how documents are scored. You just have to be careful to use the same analyzer for your searches as then one you generated your index with.

Closing from lack of discussion.


.NET Standard compatibility?

Hi,
I'm currently studying how to use Electron.NET inside a Blazor application.
Following this tutorial I'm able to configure the appropriate dependencies in the server project (I'm using a client-side, ASP.NET-hosted solution) and use electronize to start the app and open the main window.
However, I can't use Electron.NET from the Client project. It must use netstandard2.0 as the target framework, and NuGet rejects the ElectronNET.API package when I try to add it:
C:\Users\miguel\Code\blazor\blazorclienthosted\blazorclienthosted.Client
λ  dotnet add package ElectronNET.API --version 5.22.12
  Writing C:\Users\miguel\AppData\Local\Temp\tmpCBC1.tmp
info : Adding PackageReference for package 'ElectronNET.API' into project 'C:\Users\miguel\Code\blazor\blazorclienthosted\blazorclienthosted.Client\blazorclienthosted.Client.csproj'.
info : Restoring packages for C:\Users\miguel\Code\blazor\blazorclienthosted\blazorclienthosted.Client\blazorclienthosted.Client.csproj...
error: Package ElectronNET.API 5.22.12 is not compatible with netstandard2.0 (.NETStandard,Version=v2.0). Package ElectronNET.API 5.22.12 supports: netcoreapp2.1 (.NETCoreApp,Version=v2.1)
error: Package 'ElectronNET.API' is incompatible with 'all' frameworks in project 'C:\Users\miguel\Code\blazor\blazorclienthosted\blazorclienthosted.Client\blazorclienthosted.Client.csproj'.

Is Electron.NET going to be available under .NET Standard 2.0 at some point?
Thanks!

Hi Miguel,
Electron.NET should be used inside the Server project since it is the startup project.
Best regards,
Maher


Jump to 'Backup your key' screen when user has assets in wallet

App should display the 'backup your key' screen when the user finally has eth or ticket assets.
The screen should advise user on where to put the data, and that it's locked with a password.

@JamesSmartCell bump

@JamesSmartCell bump

@JamesSmartCell is this done?

No longer relevant since UI keystore implementation #770


tabletserver/txthrottler: Fix bug that config update error was not ch…

…ecked.
An internal tool found this :)

LGTM


None of the test users are in stub-idp

Fixed - https://github.com/alphagov/verify-stub-idp/pull/25


Error init passpie on windows 8

After successfully installed passpie, I typed this
passpie init

And this error occur :
raise RuntimeError(str(ae))
RuntimeError: Homedir ''C:\Users\XXX~1\AppData\Local\Temp\tmpmp4852an'' needs
 read/write permissions

He, i think i have the same bug.
I'm on windows 8.1, i use pip to install passpie and use Cmder as console
There is my log in the console :
λ passpie init
Passphrase:
Repeat for confirmation:
ERROR:gnupg:[Error 123] La syntaxe du nom de fichier, de rÚpertoire ou de volume est incorrecte: "E:\\'e:"
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\gnupg\_util.py", line 322, in _create_if_necessary
    os.makedirs(directory, 0x1C0)
  File "C:\Python27\lib\os.py", line 150, in makedirs
    makedirs(head, mode)
  File "C:\Python27\lib\os.py", line 150, in makedirs
    makedirs(head, mode)
  File "C:\Python27\lib\os.py", line 157, in makedirs
    mkdir(name, mode)
WindowsError: [Error 123] La syntaxe du nom de fichier, de rÚpertoire ou de volume est incorrecte: "E:\\'e:"
Traceback (most recent call last):
  File "C:\Python27\lib\runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\Python27\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Python27\Scripts\passpie.exe\__main__.py", line 9, in <module>
  File "C:\Python27\lib\site-packages\click\core.py", line 664, in __call__
    return self.main(*args, **kwargs)
  File "C:\Python27\lib\site-packages\click\core.py", line 644, in main
    rv = self.invoke(ctx)
  File "C:\Python27\lib\site-packages\click\core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Python27\lib\site-packages\click\core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Python27\lib\site-packages\click\core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "C:\Python27\lib\site-packages\passpie\cli.py", line 159, in init
    with Cryptor(config.path) as cryptor:
  File "C:\Python27\lib\site-packages\passpie\crypt.py", line 38, in __init__
    self._gpg = gnupg.GPG(binary=self._binary, homedir=self._homedir)
  File "C:\Python27\lib\site-packages\gnupg\gnupg.py", line 125, in __init__
    ignore_homedir_permissions=ignore_homedir_permissions,
  File "C:\Python27\lib\site-packages\gnupg\_meta.py", line 177, in __init__
    self.homedir = os.path.expanduser(home) if home else _util._conf
  File "C:\Python27\lib\site-packages\gnupg\_util.py", line 744, in __set__
    getattr(obj, self.fset.__name__)(value)
  File "C:\Python27\lib\site-packages\gnupg\_meta.py", line 430, in _homedir_setter
    raise RuntimeError(str(ae))
RuntimeError: Homedir ''e:\temp\tmp9dawsu'' needs read/write permissions

I'm experiencing the same issue using both the Windows command prompt and Git Bash.

Looks like that on Windows open files cannot be accessed by other processes in this case "gpg" process. I could reproduce this issue and I am working on for the next version.
Thanks for reporting

@marcwebbie: Hey, any progress on this? Anything to share?
Yesterday I dug this one a bit while trying and debugging a build failure on Win8. Looks like python-gnupg can't handle paths quite right. Both homedir and path to GnuPG.exe (used when guessing version, for example). It's a known, wonfix, feature/bug isislovecruft/python-gnupg#62, by the way.
Maybe upstream python-gnupg is an option...

Hello,
I am working on some rewrite of gpg and git backend for passpie, I don't have windows so it is hard to test local for me. I be writting a gpg module for Windows installations and push it up to a branch so people can try.
I will keep this issue up to date with newss

Glad to know that, @marcwebbie! I'm eager to help, I already have a Win8 box all setup, shiny – phew, that was pain for a Linux/OSX user :smile:
Merci!

@marcwebbie do we have any updates on this? passpie has had no commits for almost 3 months and this issue is 4 months old. It's sad because passpie is legitimately the best CLI based password manager for Windows users.

Hello @zQueal
Actually, we were experiencing some issues with python-gnupg and I've been rewriting passpie commands backend not to depend on python-gnupg anymore. This will probably become the 1.0 version of Passpie and it is on a different branch.
You can take a look at the current work here: https://github.com/marcwebbie/passpie/tree/redesign_dependencies
Thanks for keeping up with Passpie
Regards

Awesome to know this, Marc. I'll have a close look at and experiment it as soon as I can. Thanks for sharing.

Hey Marc, this is great news!
I just loaded this up via pip and got farther than with master.
λ passpie init --force
Passphrase:
Repeat for confirmation:
Traceback (most recent call last):
  File "C:\Python27\Scripts\passpie-script.py", line 9, in <module>
    load_entry_point('passpie==0.3.3', 'console_scripts', 'passpie')()
  File "C:\Python27\lib\site-packages\click\core.py", line 700, in __call__
    return self.main(*args, **kwargs)
  File "C:\Python27\lib\site-packages\click\core.py", line 680, in main
    rv = self.invoke(ctx)
  File "C:\Python27\lib\site-packages\click\core.py", line 1027, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Python27\lib\site-packages\click\core.py", line 873, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Python27\lib\site-packages\click\core.py", line 508, in invoke
    return callback(*args, **kwargs)
  File "C:\Python27\lib\site-packages\click\decorators.py", line 63, in new_func
    return ctx.invoke(f, obj, *args[1:], **kwargs)
  File "C:\Python27\lib\site-packages\click\core.py", line 508, in invoke
    return callback(*args, **kwargs)
  File "C:\Python27\lib\site-packages\passpie\cli.py", line 99, in init
    key_length=db.config['key_length'])
  File "C:\Python27\lib\site-packages\passpie\crypt.py", line 64, in create_keys
    keysfile.write(export_keys(temp_homedir))
AttributeError: 'NoneType' object has no attribute 'write'

Hopefully that's some good debug information.

Hey All!
I have something of an update on this issue.  I forked the source and seem to have gotten passpie working on Windows 10, however this involved changing the dependency from gnupg to python-gnupg, which I'm still unsure if this'll break anything.  It seems that on windows there's a known conflict between the python-gnupg and gnupg packages, however I've had better luck using the python-gnupg packages to avoid this particular issue.
I've got the code on my github, and would welcome discussion and feedback on if there's anything else that breaks moving passpie to python-gnupg, as opposed to the gnupg library, and if there's any other considerations involved with a library change like this.  Or if there's any particular reason against using the python-gnupg libs.  Lemme know!  I'd love to have passpie working on my windows box alongside Mac OS X and Linux!
https://github.com/PoppySeedPlehzr/passpie

Hello @chtiprog @scorphus @zQueal @wcastand @jeffbonhag
Development version of Passpie is now working on Windows. Try it out with:
pip install -U https://github.com/marcwebbie/passpie/tarball/master

Tested on WIndows 10

@marcwebbie tested on Windows 8 x64 with Python 2.7.10 and pip 8.0.2, I can init, add, and copy!  Thanks a lot for the awesome project!!

@PoppySeedPlehzr happy to read that it was working for you!
This is now pushed to upstream passpie.
pip install passpie

Thanks for your contribution

Will be closing this issue, seems to be fixed in upstream.
Thanks for your contribution: 😃

I get this issue on Windows 10. Seems to be similar to the one described above.
M:>passpie init
Traceback (most recent call last):
File "m:\miniconda3\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "m:\miniconda3\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "M:\Miniconda3\Scripts\passpie.exe_main.py", line 9, in
File "m:\miniconda3\lib\site-packages\click\core.py", line 716, in call
return self.main(*args, **kwargs)
File "m:\miniconda3\lib\site-packages\click\core.py", line 695, in main
with self.make_context(prog_name, args, **extra) as ctx:
File "m:\miniconda3\lib\site-packages\click\core.py", line 620, in make_context
self.parse_args(ctx, args)
File "m:\miniconda3\lib\site-packages\click\core.py", line 1012, in parse_args
rest = Command.parse_args(self, ctx, args)
File "m:\miniconda3\lib\site-packages\click\core.py", line 874, in parse_args
value, args = param.handle_parse_result(ctx, opts, args)
File "m:\miniconda3\lib\site-packages\click\core.py", line 1398, in handle_parse_result
self.callback, ctx, self, value)
File "m:\miniconda3\lib\site-packages\click\core.py", line 77, in invoke_param_callback
return callback(ctx, param, value)
File "m:\miniconda3\lib\site-packages\passpie\validators.py", line 43, in validate_config
configuration = config.setup_crypt(configuration)
File "m:\miniconda3\lib\site-packages\passpie\config.py", line 71, in setup_crypt
configuration['recipient'] = get_default_recipient(configuration['homedir'])
File "m:\miniconda3\lib\site-packages\passpie\crypt.py", line 132, in get_default_recipient
output, _ = process.call(command)
File "m:\miniconda3\lib\site-packages\passpie\process.py", line 40, in call
with Proc(*args, **kwargs) as proc:
File "m:\miniconda3\lib\subprocess.py", line 707, in init
restore_signals, start_new_session)
File "m:\miniconda3\lib\subprocess.py", line 966, in _execute_child
args = list2cmdline(args)
File "m:\miniconda3\lib\subprocess.py", line 461, in list2cmdline
needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: argument of type 'NoneType' is not iterable


motions to cit and ciT

Does this feature exist in Emacs orgmode core?
N/A
Orgmode link
No response
Feature value
No response
Additional context
I think your work is wonderful, I just miss vim motions to move states.
e.g.:
STATES -> { 'TODO', 'NEXT', 'DOING', '|', 'DELEGATED', 'DONE', 'ARCHIVED', 'TRANSFERED'}
			  				   /\ I am here, but i want to ARCHIVED, so i would type 3cit instead of cit 3 times.

Is it plausible?

If you are jumping more than one step ahead, I suggest checking out the "fast access" feature. More in this section of readme https://github.com/kristijanhusak/orgmode.nvim/blob/master/DOCS.md#org_todo_keywords

I read it, but it feels like cheating vim motions, I will give it a try.

these cit mappings are not really a motion mappings, since there is no visual/yank/delete representation of it. It's just a regular normal mode mapping. Count could be added though, so I'll check it out when I get some time.

Okay, thanks I'm new to orgmode.


Profile Search Page

Create a new component to search users by username or show all users sort by date with pagination
route should be /profile
use supabase to search
Display the user profile cards (username, avatar_url, and a link to their profile)
show error case No profiles found
Loading profile card case
Let me know if u have design ideas for user profile cards

I want to work on it please assign me.


Business can enter negative working hours while registering

1. Severity:
medium
2. Reported by:
Abdelrahman El Meniawy
3. Description:
A business can enter the working hours as a negative number when registering
4. Steps to reproduce the issue:

register as a business
get confirmation email from the admin
open the link in the email
get the form page
enter a negative number in the working hours field
gets a success message
5. Expected result:
Expected a validation error message

Working hours is a formatted textfield i.e you enter the details of your working hours not actual time
Sat   10 AM - 12 PM
Mon 10 AM - 12 PM
Fri    Closed

The form is not descriptive enough then.


[New Feature] - Add new Bank Sync Functionality

Thanks for all the hard work putting this together.  Please consider this request to add the new bank sync functionality:
https://github.com/actualbudget/actual/pull/2683

Hello @latetedemelon , thanks for your interest in the project! I was planning to integrate such functionality. I mapped the API calls from the
I think this info is for now enough for the API models, but the resolution of the transactions is unfortunatelly not that straight forward (at least I know by lookimg at actual source code that you also have to resolve edge cases, like transactions added manually, ending balance that does not match, etc). Since this is a bigger issue I'll focus first on getting the pip release ready (https://github.com/bvanelli/actualpy/issues/14) and then try to implement this functionality.
For reference, here is the flow for Simplefin (since they have a beta token easily available):

Frontend registers by posting to /secret using the body {"name": "simplefin_token", "value": "raw token"}. Response is {"status": "ok"}
There is a post request to figure out which syncronization methods are already configured (for some reason, this is a POST with empty dict): /gocardless/status and /simplefin/status. Response is {"status":"ok","data":{"configured":true}}
You can then query, in this case, which account you want to syncronize. For Simplefin,  POST to /simplefin/accounts with empty dict. The response somehow seems to contain exactly (a very arbitrary number of) 57 of the oldest transactions. The response is something like:

{
   "status":"ok",
   "data":{
      "accounts":[
         {
            "org":{
               "domain":"example.com",
               "sfin-url":"https://beta-bridge.simplefin.org/simplefin"
            },
            "id":"Demo Savings",
            "name":"SimpleFIN Savings",
            "currency":"USD",
            "balance":"115524.49",
            "available-balance":"115524.49",
            "balance-date":1719792000,
            "transactions":[
               {
                  "id":"1717315200",
                  "posted":1717315200,
                  "amount":"-10.00",
                  "description":"Fishing bait",
                  "payee":"John's Fishin Shack",
                  "memo":"JOHNS FISHIN SHACK BAIT"
               }
            ],
            "holdings":[
               {
                  "id":"25bc4910-4cb4-437b-9924-ee98003651c5",
                  "created":345427200,
                  "cost_basis":"55.00",
                  "currency":"USD",
                  "description":"Shares of Apple",
                  "market_value":"105884.8",
                  "purchase_price":"0.10",
                  "shares":"550.0",
                  "symbol":"AAPL"
               }
            ]
         },
         {
            "org":{
               "domain":"example.com",
               "sfin-url":"https://beta-bridge.simplefin.org/simplefin"
            },
            "id":"Demo Checking",
            "name":"SimpleFIN Checking",
            "currency":"USD",
            "balance":"26134.41",
            "available-balance":"26134.41",
            "balance-date":1719792000,
            "transactions":[
               {
                  "id":"1717315200",
                  "posted":1717315200,
                  "amount":"-13.31",
                  "description":"Fishing bait",
                  "payee":"John's Fishin Shack",
                  "memo":"JOHNS FISHIN SHACK BAIT"
               }
            ],
            "holdings":[

            ]
         }
      ]
   }
}

Apparently we can also query the transactions via the endpoint /simplefin/transactions with POST data {"accountId":"Demo Checking","startDate":"2024-03-05"}. The second call used start date as 2024-04-03 that was the latest date from the previous sync. The response for the first call was:

{
   "status":"ok",
   "data":{
      "balances":[
         {
            "balanceAmount":{
               "amount":"26134.41",
               "currency":"USD"
            },
            "balanceType":"expected",
            "referenceDate":"2024-07-01"
         },
         {
            "balanceAmount":{
               "amount":"26134.41",
               "currency":"USD"
            },
            "balanceType":"interimAvailable",
            "referenceDate":"2024-07-01"
         }
      ],
      "startingBalance":2613441,
      "transactions":{
         "all":[
            {
               "booked":true,
               "bookingDate":"2024-04-03",
               "date":"2024-04-03",
               "debtorName":"John's Fishin Shack",
               "remittanceInformationUnstructured":"Fishing bait",
               "transactionAmount":{
                  "amount":"-19.96",
                  "currency":"USD"
               },
               "transactionId":"1712131200",
               "valueDate":"2024-04-03"
            },
            {
               "booked":true,
               "bookingDate":"2024-04-03",
               "date":"2024-04-03",
               "debtorName":"Grocery store",
               "remittanceInformationUnstructured":"Grocery store",
               "transactionAmount":{
                  "amount":"-163.36",
                  "currency":"USD"
               },
               "transactionId":"1712160000",
               "valueDate":"2024-04-03"
            },
         ],
         "booked":[
            {
               "booked":true,
               "bookingDate":"2024-04-03",
               "date":"2024-04-03",
               "debtorName":"John's Fishin Shack",
               "remittanceInformationUnstructured":"Fishing bait",
               "transactionAmount":{
                  "amount":"-19.96",
                  "currency":"USD"
               },
               "transactionId":"1712131200",
               "valueDate":"2024-04-03"
            },
            {
               "booked":true,
               "bookingDate":"2024-04-03",
               "date":"2024-04-03",
               "debtorName":"Grocery store",
               "remittanceInformationUnstructured":"Grocery store",
               "transactionAmount":{
                  "amount":"-163.36",
                  "currency":"USD"
               },
               "transactionId":"1712160000",
               "valueDate":"2024-04-03"
            }
         ],
         "pending":[

         ]
      }
   }
}

I literally came here to ask the same! But thinking about it - I wonder if integrating with SimpleFin directly and then adding the transactions with the API would give extra control. Though that is likely beyond the scope of this project.t


feat: merge from dev to main branch

Description
Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

We identified new issues on unchanged lines of code. Navigate to the Amazon CodeGuru Reviewer console to view the recommendations to fix them.


Good Ending Day becomes "war"?

No longer affects Day variable, now replaces Time variable with "The Keeon Wars".


Install system call support for tests in travis

I think this will fix the test failures in travis. Currently gopherjs test warns about missing syscall support. The added commands are directly pulled from the gopherjs documentation, which has worked fine for me on gitlab CI.
https://github.com/gopherjs/gopherjs/blob/master/doc/syscalls.md

This is ready

Awesome, thanks for doing this @bzub ! This looks great 😄

(feel free to send a PR to add yourself to contributors file if you want btw, I forgot to mention in my last PR)


Is there a way to get the client related entities, such as RedirectUris, ClientSecrets, AllowedScopes, etc.

Can you elaborate?

Please see my code below.  I want to use the IdentityServer4.EntityFramework to manage the clients. I got a collection of clients use the following code, but the entity cannot obtain the related RedirectUris, etc. Please see my attached screenshot.

   `private readonly IConfigurationDbContext _context;
    private readonly IClientStore _clientStore;

    public ClientsController(IConfigurationDbContext context, IClientStore clientStore)
    {
        _context = context;
        _clientStore = clientStore;
    }

    // GET: Clients
    public ActionResult Index()
    {
        var clients = _context.Clients.ToList();

        return View(clients);
    }`

You need to include the collections (there's no lazy loading in EF Core):
_context.Clients.Include(x => x.RedirectUris).ToList()

Thanks a lot for your answer.
Yes, I know the way you say. But there is an error that DbSet does not contain a definition for "Include". Could you please let me know Which assembly reference should I use?


Invalid API Key

General information

Description
Value

Project version
2.4.4

Operating system and version
windows 11 home

NodeJS version
16.17.0

NPM version
8.15.0

Yarn version
1.22.19

Truffle version
5.5.31

I get the next issue when I try to verify the contract
Error in plugin @nomiclabs/hardhat-etherscan: Invalid API Key

Hi @bolisteward,
which chain are you working on? Please remember that any chain has its own verification API key (e.g. Etherscan API key won't work on Polygon, and so on). If you haven't already, I also highly recommend you to watch my video tutorials (YouTube) to get instructions about how to take every single step correctly.
Let me know if this helps.


BindingsGenerator+HTML: Partially implement HTMLInputElement's selection{Start,End}

Get the functionality we already have working in HTMLTextAreaElement put into a common place for use in HTMLInputElement.

@shannonbooth The new test added here is flaky on master:
 --- /home/runner/work/ladybird/ladybird/Tests/LibWeb/Text/expected/./input-selection-start-selection-end.txt
+++ /home/runner/work/ladybird/ladybird/Tests/LibWeb/Text/expected/./input-selection-start-selection-end.txt
@@ -1,5 +1,5 @@
-Well hello friends    text selectionStart: 0
-text selectionEnd: 0
+Well hello friends    text selectionStart: 5
+text selectionEnd: 5
 date selectionStart: null
 date selectionEnd: null
 text selectionStart: 18

https://github.com/LadybirdBrowser/ladybird/actions/runs/10196223641/job/28206559460

@shannonbooth The new test added here is flaky on master:

Was this ever resolved?

(I suppose #1193 deleted the test again  – I mean before then :))

Was this ever resolved?

Yep, the issue was the test was affected by a cursor position retained from the previous test/page load. Fixed in https://github.com/LadybirdBrowser/ladybird/pull/932.


Licensing?

Just wondering what license debate-cards falls under.
Thanks.

Went ahead and added an MIT License. But I'm open to suggestions if you think a different one would be better.