Random crashes on iOS after calling execute method

Describe the bug
There are random crashes in X seconds after calling recaptchaClient.execute(RecaptchaAction(action: action)).
Integration Method
Select the method used to integrate with reCAPTCHA Mobile.

[x] SPM (iOS)
[ ] Cocoapods (iOS)
[x] Direct Download (iOS)
[ ] GMaven (Android)

SDK Version (e.g. 18.0.1):
18.0.1, 18.0.3, 18.1.0
To Reproduce
Steps to reproduce the behavior:

Call recaptchaClient.execute(RecaptchaAction(action: action))
Interact with UITextField in some way: open keyboard, write some text, close keyboard, open keyboard and write again, etc.
Optional suspend and return to the app after some short time
Optional call recaptchaClient.execute(RecaptchaAction(action: action)) again
Optional interact with UITextField again
Optional Move to another screen in the app
Wait X seconds
Observe the crash

The order of 3-6 steps could be mixed
Expected behavior
Eliminate all crashes produced by the framework
Screenshots
malloc: Heap corruption detected, free list is damaged at 0x282a948f0
*** Incorrect guard value:<PHONE_NUMBER>14432

EXC_BAD_ACCESS: hash >
Attempted to dereference garbage pointer 0x1de9bf0.

EXC_BAD_ACCESS
isEqual: >
Attempted to dereference garbage pointer 0x400000020.

Xcode version for iOS (please complete the following information):

Xcode 14.2

Device (please complete the following information):

iPhone 7, iPad (9th generation), iPad Pro (2nd generation)
iOS: 15.7.1, 16.2, 16.3.1
Real devices, release build
Doesn't reproduce on simulators and debug build

Additional context
It's randomly reproduced in X seconds after calling execute() method.
We are assuming there are some bugs somewhere in the Objective-C code of the framework, eg. pointers.
Could be connected to: https://github.com/GoogleCloudPlatform/recaptcha-enterprise-mobile-sdk/issues/22
@mcorner please take a look and advice us how to eliminate crashes or when we can expect a new version of the framework which fixes such issues?

Also when you say "Quit the app" do you mean force quit? Or suspend the application in the background?

The SDK does not interact with the UITextField or the keyboard.

We don't know it, so it's just our observation when crashes appear. UITextField and keyboard may not influence at all on it, but we are sure that it appears randomly after X seconds after calling execute method.

It sounds like you have replicated it locally.

We reproduced it on TF builds, but I assume that we will have something similar as @SurglogsGithubUser here after changing build config to release.

a) how often does it occur?

In every app session after the execute method has been called

Or how quickly can you make it happen?

~15 seconds - 5 minutes after the above method was called

b) if you have a sample project that seems to have the right combination of things, would you be willing to share it?

Unfortunately, we don't have a sample project. I can share a screen recording of the crash with you privately if it may help.

Also when you say "Quit the app" do you mean force quit?

Nope, I mean just suspend and return to the app after some short time, ~ max 3 mins.

Thus far no luck reproducing this on a real device, also using a release build.  We haven't seen the crash so far and AddressSanitizer isn't yielding any clues thus far.

Obviously, without a clear repro, we will have trouble finding it here.

@mcorner
No, we haven't run AddressSanitizer yet.
We discovered interesting information yesterday that might be useful for you: we were not able to reproduce such behavior on the 18.0.2 version of the framework installed via SPM. But the issue unfortunately still occurs on 18.0.1, 18.0.3, 18.1.0.
Additional info: we were not able to install 18.0.2 via direct download xcframework due to compiling errors, attaching below:

That means you are missing protobuf.  Starting with v18.0.3 we embed protobuf in the SDK. In v18.0.2 it was a separate direct download dependency that needed to be included.  SPM and pods did it for you.
This might give us a clue that it is something in a particular part of our code.  We will keep digging.

@mcorner we followed the integration guide from here. Nothing was mentioned about "protobuf". We installed via SMP and via direct download xcframework. v18.0.1 also doesn't work for us.
Could you write more about "protobuf"? What is it and what should we do? Thank you!

This compilation problem is a separate issue, please open a new one so we can keep this one focused on the original one.

Great news @mcorner
Can't wait for the next release.
Thanks for the updates!

Ok....finally!  ios 18.1.2 just went out which hopefully fixes this issue. Official release notes will be up soon, but in the meantime:

Fix for memory corruption bug #38
Refinement of fix for bug affecting execute on iOS 11, 12 and 13 devices
Returns network error instead of internal error in cases where the network is extremely slow, but not doesn't fail by the timeout

There will be an Android release real soon.
Keep in mind that the SDK is only part of the equation.    The vast majority of issues we see are related to poor networking conditions (much more typical on Android phones that are used more globally).  We have a long series of improvements to help address those situations and are committed to having reCAPTCHA work for every customer globally.


As a developer I want centralised styling so I can swap in brand styles without editing 100s of files

We talked about using Styled Components for this, because it's CSS syntax, well known, well supported, works with both Native and Web, and has robust built-in support for centralised theming while accepting props so can be tailored at the component level from a central source.

[ ] Check that Styled Components does indeed suit our needs (there are some reports of performance issues on Native, we suspect these are out of date or due to it being implemented badly but we should check). If not, find something that does.
[ ] Set up cross-platform centralised styling architecture
[ ] Make it look good enough to demo and sell to clients, and on-brand for Nearform
[ ] Document how to replace the Nearform styles with another brand

From the last conversation we had about this, we were imagining something like:

Brand/client-specific styles applied from a central theme passed in through ThemeProvider (if we use styled components; or something similar if we don't)
Necessary structural CSS for each component (the styles we don't expect to change per-project) in a styles.js file alongside the component
If we use styled-components, put cross-platform styles in css template and platform-specific attached to a base element like:

import styled, { css } from 'styled-components';

const commonStyles = css`
   /* some CSS here */
`

const styles = Platform.OS === 'web' ? styled.div`
  ${commonStyles}
  /* web-specific CSS here */
`; : `
  ${commonStyles}
  /* native-specific CSS here */
`


Switch project browse view to use resized thumbnails

[x] Download images
[x] Resize images
[x] Upload thumbnail to S3
[x] Switch apps to use function to get thumbnail URLs

[x] simulation run view
[x] projects browse

[x] Document image size

Done


helm init --wait gets a context deadline exceeded

Running helm init --wait --upgrade --service-account tiller
$HELM_HOME has been configured at /root/.helm.
Error: could not ping Tiller: context deadline exceeded

Could be related to using upgrade?
v2.8.0

I'm also getting this when I run helm init --wait

@TheOriginalAlex would you mind taking a look into this?

@bacongobbler: Will take a look this upcoming weekend!  :)

Wait algorithm changed here, https://github.com/kubernetes/helm/pull/3506.

i still get this error when i use helm v2.13.1? what's the solution for it?


Incorrect number of products display in cart if product already added in cart is deleted from back-end.Also showing incorrect price in this case, previous product price is also added.

Framework Version : 0.1.2
Test Steps:

Add a product in cart.
Delete that product from back-end.
Refresh the cart page.

Result: Product will be removed by cart but count remains the same.

This issue is marked as critical bug because it breaks up cart instance from its summary information to item information as items get deleted when the product is also deleted from the backend.
We are looking for possible solutions to tackle this issue and the best one will be applied soon.
As deleting the the products is a rare or an action done by mistake and this will prevent this situation form coming up again and again.
Possibly v0.1.4 will have the fix for this, we have left it out of release v0.1.3.


Is there any way to disable specific error messages?

Given the following code:
def a() -> dict[int, list[int]]:
    return {3: [1, 2]}

pyright says that:

Pyright: Subscript for class "dict" will generate runtime exception; enclose type annotation in quotes
Pyright: Subscript for class "list" will generate runtime exception; enclose type annotation in quotes

But acutally I don't need my code supporting old python version, I'm using python 3.11 which supports the syntax.
I have configure virtual env to use python3.11, but the error still exists.  Is there any way to disable the message?  Thanks!

If you are using Python 3.11 (or any version of Python after 3.8), you should not see these errors unless you've explicitly configured pyright's pythonVersion setting to an older version of Python.
If you look in the Output logs for pylance, you should see a line that says something like "Assuming Python version 3.11.0.final.3" or something like that. If you don't see that, then check your pyproject.toml or pyrightconfig.json file to see if you've specified a "pyrightVersion".

Thanks!  I never set pythonVersion in any of my config files.
But I can set it manually in pyrightconfig.json.


Removing storage client

Please add a meaningful description for your change here

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Can we start with the PR that adds Datastore Client and then remove the GCS part as a separate PR?


[BEAM-4832] Fix concurrent race in datamgr.go

Concurrent access to the DataChannel maps wasn't guarded when deleting readers. This manifested on a user's streaming pipeline where bundles were small and frequent on a single worker.

Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.
Post-Commit Tests Status (on master branch)

Lang
SDK
Apex
Dataflow
Flink
Gearpump
Samza
Spark

Go

---
---
---
---
---
---

Java

Python

---

---
---
---
---

R: @herohde

Please add comments to the mutex to indicate that it protects the readers and writers maps.

Good call, Done!


[Bug]: Access to undeclared static property App\Filament\Resources\CourseResource\Pages\CreateCourseLesson::$relationship

What happened?
When I am trying to navigate to the create page I am getting the below error.
Access to undeclared static property App\Filament\Resources\CourseResource\Pages\CreateCourseLesson::$relationship

This is the content of my CourseResource\Pages\CreateCourseLesson.php file.
<?php

namespace App\Filament\Resources\CourseResource\Pages;

use App\Filament\Resources\CourseResource;
use Guava\FilamentNestedResources\Concerns\NestedPage;
use Guava\FilamentNestedResources\Pages\CreateRelatedRecord;

class CreateCourseLesson extends CreateRelatedRecord
{
    use NestedPage;

    protected static string $resource = CourseResource::class;

}

Did I miss something here?
How to reproduce the bug
When creating lesson from the relation page.
Package Version
1.0
PHP Version
8.3
Laravel Version
11
Which operating systems does with happen with?
macOS
Notes
No response

Also could you please provide the source code of the example given for Artist/Albums/Songs too?

@z4yed You need to implement the $relationship property. I guess in your case it will be something like:
protected static string $relationship = 'lessons';

This worked. However, when I am in view/edit page of any lesson, the root navigation section Courses is not active (highlighted). Please see attached image.

This worked. However, when I am in view/edit page of any lesson, the root navigation section Courses is not active (highlighted). Please see attached image.

Great catch, thanks. Will fix it in the next release. :)

dear all i use to create but it don't fixin with my require
here is what i want i have series, seasons,episodes
but record is only one get how do i make
Model
Series have many Season
Season have belongto series and have many episodes
episodes have belongto series
So route should be like that
series/{record}/seasons/{record}/episodes


GUACAMOLE-1594: Update frontend dependencies for 1.5.0

Combines changes from
https://github.com/apache/guacamole-client/pull/725, https://github.com/apache/guacamole-client/pull/724, https://github.com/apache/guacamole-client/pull/718, and https://github.com/apache/guacamole-client/pull/723 (mistakenly targeted at master)

@mike-jumper and @necouchman What do you think? Should I update just the dependencies that dependabot identified in the aforementioned PRs, or as many NPM dependencies as possible? I can pull https://github.com/apache/guacamole-client/pull/726/commits/74a34337f36f4b9dda94504a5b9b9dbc66369a14 out of this PR if the former is the way to go.

I think it's OK to update everything that we can. Since we have the automated license check in place, we can be certain that the vast majority of these changes are specific to the build and do not affect the actual contents of the webapp.


Make host_inventory["kernel"]["machine"] always available

Since get_kernel inventory command is unavailable on Darwin, the value of host_inventory["kernel"]["machine"] is unavailable though it is provided from os_info:
$ ruby -Ilib -rspecinfra -e "p Specinfra::HostInventory.instance['kernel']['machine']"
No backend type is specified. Fall back to :exec type.
/Users/mrkn/src/github.com/mizzy/specinfra/lib/specinfra/command_factory.rb:20:in `get': get_kernel is not implemented in Specinfra::Command::Darwin::Base::Inventory (NotImplementedError)
	from /Users/mrkn/src/github.com/mizzy/specinfra/lib/specinfra/host_inventory/kernel.rb:8:in `get'
	from /Users/mrkn/src/github.com/mizzy/specinfra/lib/specinfra/host_inventory.rb:42:in `[]'
	from -e:1:in `<main>'

Ignoring NotImplementedError occurred in CommandFactory#get makes it always available.

I noticed that it is better than this approach to defining get_kernel with false command in Specinfra::Command::Darwin::Base::Inventory.
I'll change the commit.

I replaced the commit.

Thanks!

Released as v2.83.2.


How to add ListNumber or ListBullet in .DOCX file?

@scanny
@tooh
Hi
I am taking user input (below link ) in formatted form and saving these data in MySQL database.

But when i'm trying to retrieve data using markdown and python-docx its generating unformatted docx file like this:-
demo.docx
Can somebody tell me how to fix it ?
Thanks in advance.
Regards,
Hansraj

@hansraj12 -- this issues list is for bugs and feature requests. Please use the mailing list or Stack Overflow for support questions.


[CLOSED] Should we use a "default value" type class for Panic.panic?

 Issue by sweirich
Monday Oct 16, 2017 at 13:07 GMT
Originally opened as https://github.com/antalsz/hs-to-coq/issues/30

Many uses of Panic.panic in CmmTypes.hs are with concrete result types (integer, width, etc).
If we have a class for default values, then we can safely parameterize the panic function with this class.
Parameter panic {a} `{Default a} : String -> a

 Comment by antalsz
Monday Oct 16, 2017 at 15:39 GMT

Is this actually the right design?  One thing we'll presumably want to prove is that panic is never called…

 Comment by antalsz
Monday Oct 16, 2017 at 16:11 GMT

Yeah, that's very fair.

 Comment by sweirich
Monday Oct 16, 2017 at 15:44 GMT

The correctness theorem would prove that panic is never called.
I'm somewhat pessimistic that we will get this result just through our translation. It would take a lot of rewriting to replace all of the calls of panic with something else.  (And this is better than just returning the default value without wrapping it in panic.)

 Comment by sweirich
Tuesday Oct 17, 2017 at 01:00 GMT

Ok, tried it out with my recent commit.
commit 15c4060405095e4d75f6e18f23dd485ac3c5c3ed

 Comment by antalsz
Tuesday Oct 17, 2017 at 19:07 GMT

@sweirich How would that work with the "type class with no instances" approach?  I don't think that would damage the translation in 15c4060405095e4d75f6e18f23dd485ac3c5c3ed, and it might make the proofs easier.

 Comment by sweirich
Tuesday Oct 17, 2017 at 19:10 GMT

I don't understand your question. What do you mean the "type class with no instances" approach?

 Comment by sweirich
Tuesday Oct 17, 2017 at 19:12 GMT

BTW, there is probably a better place (and perhaps name) for the Default type class. Furthermore, we can tell Coq to be particularly aggressive about finding instances for it --- we will never care about coherence.

 Comment by antalsz
Tuesday Oct 17, 2017 at 19:13 GMT

Ah, sorry.  Something like this:
Class Default (a : Type) := { default : a }.

(* No instances  *)

Definition panic {a} `{Default a} (_ : String) : a := default.

Of course, now I recall the problem with this, which is that we'd have to propagate the constraint to the types of the translated functions.  Never mind :upside_down_face:

 Comment by sweirich
Tuesday Oct 17, 2017 at 19:16 GMT

Yes, I was only intending this to be used with concrete types, not propagating instances.
Furthermore, note that panic is a Parameter.  We never want to simplify it away during a proof because it marks a part of the program that is an error and we don't want to lose that information. But, we are guaranteed that this axiom is consistent because we could give panic a definition---precisely the one you have above.
Finally, if we extract, we can extract it back to the original panic.

 Comment by sweirich
Monday Apr 30, 2018 at 13:21 GMT

Closing this issue --- we've done this.


Feat: add a back to home button on the login/signup page

Describe the feature
A back to home button on the top left corner of the screen increases ease of user while navigating through pages.
Add ScreenShots

Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

please assign to me


log extra properties on Errors as extra data

Do you want to request a feature or report a bug?
Feature
What is the current behavior?
class FooError extends Error {
    constructor(msg, ham, spam) {
        super(`A foo error happened ${msg}`) ;
        this.ham = ham;
        this.spam = spam;
    }
}

setTimeout(() => throw new FooError('oh no', 'the ham', 'the spam', 1);

I only get info about the message.
What is the expected behavior?
I'd like to get all the properties on FooError eg ham and spam.

Hi @graingert - this makes good sense to me but it's not high priority yet because I don't believe this pattern is very common in JS. good suggestion and worth implementing eventually though!


Update TypesScript to 4.9.3.

It catches one new error: parseTweet can never return a boolean or even a falsy value AFAICT.

LGTM ! 👍


natural-pong: init at 0.1.0

This is a fresh looking pong game.
Description of changes

Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[X] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[X] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[X] Fits CONTRIBUTING.md.

@neosam are you still interested it getting this done?

Sorry, I was busy the last few days.  I will look at it tomorrow.

If you need more time take more time, no pressure. Just wondered since I didn't get any feedback and I like getting the things I'm involved in done :D

I converted the PR to a draft now since I have to do several modifications.  I'm very positive I can work on it this week.


[Bug] doris-streamloader memory leak

Search before asking

[X] I had searched in the issues and found no similar issues.

Version
1.0.1
What's Wrong?
Translation: When using the apache/doris-streamloader script for testing and importing, the doris-streamloader process experienced rapid memory escalation, ultimately causing the BE process to crash. It is suspected that there is a memory leak in doris-streamloader. However, using HTTP stream load functioned normally. The test data used was clickbench hits.tsv.
What You Expected?
Successful import was achieved with minimal memory usage.
How to Reproduce?
No response
Anything Else?
./doris-streamloader --source_file="/opt/hits_benchmark/1_2_hits.tsv" --url="http://localhost:8030" --header="group_commit:sync_mode?columns: WatchID,JavaEnable,Title,GoodEvent,EventTime,EventDate,CounterID,ClientIP,RegionID,UserID,CounterClass,OS,UserAgent,URL,Referer,IsRefresh,RefererCategoryID,RefererRegionID,URLCategoryID,URLRegionID,ResolutionWidth,ResolutionHeight,ResolutionDepth,FlashMajor,FlashMinor,FlashMinor2,NetMajor,NetMinor,UserAgentMajor,UserAgentMinor,CookieEnable,JavascriptEnable,IsMobile,MobilePhone,MobilePhoneModel,Params,IPNetworkID,TraficSourceID,SearchEngineID,SearchPhrase,AdvEngineID,IsArtifical,WindowClientWidth,WindowClientHeight,ClientTimeZone,ClientEventTime,SilverlightVersion1,SilverlightVersion2,SilverlightVersion3,SilverlightVersion4,PageCharset,CodeVersion,IsLink,IsDownload,IsNotBounce,FUniqID,OriginalURL,HID,IsOldCounter,IsEvent,IsParameter,DontCountHits,WithHash,HitColor,LocalEventTime,Age,Sex,Income,Interests,Robotness,RemoteIP,WindowName,OpenerName,HistoryLength,BrowserLanguage,BrowserCountry,SocialNetwork,SocialAction,HTTPError,SendTiming,DNSTiming,ConnectTiming,ResponseStartTiming,ResponseEndTiming,FetchTiming,SocialSourceNetworkID,SocialSourcePage,ParamPrice,ParamOrderID,ParamCurrency,ParamCurrencyID,OpenstatServiceName,OpenstatCampaignID,OpenstatAdID,OpenstatSourceID,UTMSource,UTMMedium,UTMCampaign,UTMContent,UTMTerm,FromTag,HasGCLID,RefererHash,URLHash,CLID" --db="hits" --table="hits" --workers 1 --max_byte_per_task 10485760
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Thank you for your feedback. We will verify it.

I encountered a similar issue where writing a script to load 100 files into Doris results in memory exhaustion, causing the script to become unresponsive and stuck.


Default export not Node.js ESM compatible

In Node.js ESM mode:
// test.mjs

import isHotkey from "is-hotkey";

console.log(isHotkey);

$ node test.mjs

Yields:
{
  default: [Function: isHotkey],
  isHotkey: [Function: isHotkey],
  isCodeHotkey: [Function: isCodeHotkey],
  isKeyHotkey: [Function: isKeyHotkey],
  parseHotkey: [Function: parseHotkey],
  compareHotkey: [Function: compareHotkey],
  toKeyCode: [Function: toKeyCode],
  toKeyName: [Function: toKeyName]
}

Expected
The default export should be a function, and not an object with a default property.
// test.mjs

import isHotkey from "is-hotkey";

console.log(typeof isHotkey === 'function'); // true

Fwiw – this is also a problem when using "type": "module" in a Vite project.

That's because, despite this module being written in ESM, it's transpiled into CJS using Babel, which is what Node is loading. It could be easily fixed by loading the ESM source code, instead of the transpiled version. But, since this repository seems abandoned (no updates in three years) it's better to just use a fork: this one seems to have fixed it and is popular.


Fix issues in tests introduced by previous series

The main issue was that args where no more tested because of a bad variable scoping in bash

+1

+1


Place of GL backend in the new low-level world

The GL support in LL is not an obvious question. There are multiple views on it:

Drop it, focus on next-gen only. GL is outdated, poorly emulated via LL frontend, and would be a burden to support.
Have GL as a backup backend, just to have an alternative on each platform. This is especially helpful on platforms where next-gen is not available, including older Android devices and WebGL.
Have a fully featured GL as one of the primary targets. This would make sure that applications like Firefox/Servo keep running fast when WebRender updates from GL to gfx-rs LL.

I'm in favor of 3, given that I want WebRender to not suffer :)

One of the pain points of GL comparing to next-gen is context handling. Currently, we have the Surface implementing Instance and used like this:
#[cfg(feature = "gl")]
    let (adapters, mut surface) = {
        let surface = back::Surface::from_window(window);
        let adapters = surface.enumerate_adapters();
        (adapters, surface)
    };

We expose a single device and a single queue, both associated with the same rendering context. Contrary to other backends, these are not sendable, which restricts what you can do with threading when you want to support it, also crippling our core interfaces a bit.
Here is my proposal for an alternative context handling, at first to be implemented behind a feature gate:

have Instance created out of thin air, exactly how other backends do it. GL backend would create a headless context for it.
surface creation would establish a new GL context created with shared lists of the main headless one.
initially, the single adapter returned by enumerate_adapters would have a single queue family with a single queue, like now. The adapter's device would be constantly bound to the headless context.
with each surface created, the adapter would expose one extra family with a single queue. This family would be constantly associated with the GL context of that surface, and support presentation on it.
that would allow the users to create multiple surfaces and operate on multiple queues. I believe we could enforce the Send on core devices and queues even. There would still be an issue of the queue+device of the headless context going into different threads, which we'll probably just enforce at run-time, so that the compile time interfaces are nice.

We don't need multiple issues for effectively the same thing. Let it be a dupe of #1619


adding SYS_pidfd_send_signal/SYS_pidfd_getfd constants to linux uclib…

…c arm flavor.
closes #3008

r? @JohnTitor
(rust-highfive has picked a reviewer for you, use r? to override)

:pushpin: Commit 23c4ff629b21d3ed08aac4f32c799d1d5aede2f7 has been approved by JohnTitor
It is now in the queue for this repository.

:hourglass: Testing commit 23c4ff629b21d3ed08aac4f32c799d1d5aede2f7 with merge 6eaf46379432c08f03a1b7ea21dcf012ebf3f37a...

:broken_heart: Test failed - checks-actions

@bors retry

:hourglass: Testing commit 23c4ff629b21d3ed08aac4f32c799d1d5aede2f7 with merge e364b0ff3f216fe8d013efec47331ccfcd7dc7a4...

:broken_heart: Test failed - checks-actions

@bors retry

:hourglass: Testing commit 23c4ff629b21d3ed08aac4f32c799d1d5aede2f7 with merge d5691a08a8d44177a99749734e854367086c9535...

:sunny: Test successful - checks-actions, checks-cirrus-freebsd-12, checks-cirrus-freebsd-13, checks-cirrus-freebsd-14
Approved by: JohnTitor
Pushing d5691a08a8d44177a99749734e854367086c9535 to master...

:sunny: Test successful - checks-actions, checks-cirrus-freebsd-12, checks-cirrus-freebsd-13, checks-cirrus-freebsd-14
Approved by: JohnTitor
Pushing d5691a08a8d44177a99749734e854367086c9535 to master...

:eyes: Test was successful, but fast-forwarding failed: 422 Changes must be made through a pull request.


Support for jcs-based proofs according to specification

Note: I'm opening an issue here as there seems to be a plan to phase out https://github.com/digitalbazaar/jsonld-signatures/issues/178, but the problem we're facing concerns both libraries.
After having fixed an issue with jcs-based proofs not verifying properly in jsonld-signatures a few months back (https://github.com/digitalbazaar/jsonld-signatures/issues/176), I want to circle back to the issue of adding context definitions to the document and proof before signature generation. My concern is that the current praxis of altering @context properties during proof generation and verification may simply not be viable for proof mechanisms that directly sign the json representation of linked data.
Specifically, I see two issues:

Adding a proof (suite) specific context to the document can invalidate pre-existing proofs. This praxis is adopted both in this library as well as in jsonld-signatures. Adding, for the purpose of supporting a wider range of verifiers, a Ed25519Signature2020 to a document that has already been signed with a DataIntegrityProof using eddsa-jcs-2022 will alter the document, after which the eddsa-jcs-2022 suite will no longer verify the existing proof.

Adding the document context prior to signing and verification, as done in this library and, inconsistently (only on verification), in jsonld-signatures seems to be at odds with the prescribed proof configuration of existing jcs based cryptosuite specifications authored by Digital Bazaar (https://www.w3.org/TR/vc-di-eddsa/#eddsa-jcs-2022 & https://www.w3.org/TR/vc-di-ecdsa/#proof-configuration-ecdsa-jcs-2019). Even though this behaviour can be patched for digitalbazaar/data-integrity by reimplementing createVerifyData in your cryptosuite, jsonld-signature's behaviour will break verification in this case, which leads into a dilemma where implementing the specs correctly actually harms interoperability between implementations.

I'm hoping to start a thread here to discuss improvements that can be made to the way this is handled.
To kick this off, I can throw in two suggestions:

Problem 1) could be solved by no longer requiring a proof/suite context to be present on the signed document; instead, this context could be added to the proof object where the document context does not already include it. From what I can see this model would continue to assure that all terms are defined and definitions are not redeclared.
Problem 2) is probably easiest solved by aligning the proof configuration of jcs based cryptosuites with that of linked data proofs / rdf-based suites. Adding the document context to the proof object prior to canonicalization has no function for jcs-based proof schemes, but allows interoperability with the existing tooling without substantial changes to it.

@rflechtner hmm, yes, you're right. I'm just commenting to confirm that we need to address this issue as it makes it difficult to use JCS-based cryptosuites with the RDFC-based cryptosuites (which was a design goal).
I expect that using DataIntegrityProof for everything would solve the issue for VCs... but that's a point solution that we should avoid... and it doesn't work if you want to mix in stuff like Ed25519Signature2020.
We could give advice like: If you're going to use JCS-based and RDFC-based cryptosuites on the same payload, do your signatures using the RDFC-based cryptosuites first (which would do any context injection necessary) and then the JCS-based cryptosuites (which would just sign over the injected contexts)... but that feels like brittle advice.
Embedding the context values in JCS proofs feels brittle as well... special processing for a cryptosuite used by people that tend to not want to deal w/ @context values.
We'll have to think about what the right path forward here is... it's not immediately obvious given the various other forms of cryptosuites that are in the works (like the one for Data Integrity-based CL Signatures).

@msporny I agree we should aim for a robust solution rather than patch the problem with brittle advice, even if that takes a bit more investment - if we spare that effort now it will most certainly come back to haunt us further down the line.
Happy to contribute here in any way I can!
I do think we should give this option some thought: making context injection an internal feature of linked data proofs, such that documents are no longer edited in any way during signing. I get that some do not like the idea of having a separate @context property on a subsection of the document (i.e., the proof), but biting that bullet as far as I can see allows linked data proofs to continue to achieve the goals for which context injection has been introduced (making sure all proof properties are defined and cannot be redefined) while not interfering with proofs that do not need and cannot properly handle context injection.
What's your opinion on that direction, does this seem feasible?

Hi @msporny, any updates on this? Also wondering if this thread should move to a different forum (e.g., https://github.com/w3c/vc-data-integrity/issues) and if the topic warrants discussion in the relevant DIF WG?

Which DIF WG were you thinking of? I didn't think that DIF worked on Data Integrity-related items?

Yup, sorry, that was a brain-glitch, don't think they do. I was thinking of the W3C VCWG.

I will also note that the JCS implementations are dragging and we'll have to remove those features from the specifications if we don't have more demonstration of implementations (and test suites).

We've been working on an implementation of a eddsa-jcs-2022 suite in typescript, ready for use with @digitalbazaar/data-integrity, which we plan to release & open source in the next couple of weeks. Would it help to include this in the implementation report?

All that said, yes, please, raise an issue so we can officially bring it to the VCWG's attention.

Will do, and I'll ping Dave in the hopes that he remembers details about the solution you came up with. Thanks!

I believe the solution was, for just the JCS cryptosuite proofs, to include the proof context from the document (and any JCS cryptosuite context, if necessary) in the JCS cryptosuite proof object itself (proof: [{"@context": "<here>", cryptosuite: "eddsa-jcs-2022", ...}, ...]) -- and update the algorithm steps for JCS cryptosuites to use it to override the document's context prior to signing or verifying.
This would avoid disturbing other cryptosuites that do not have this issue that rely on using the top-level @context and have done so for quite some time. Changing that around for every cryptosuite in an effort to be consistent / add additional separation starts messing with other requirements and usage patterns that we don't have time or resources to fully investigate.

We've been working on an implementation of a eddsa-jcs-2022 suite in typescript, ready for use with @digitalbazaar/data-integrity, which we plan to release & open source in the next couple of weeks. Would it help to include this in the implementation report?

Yes, it would help a bunch. We plan on doing an implementation and you need a minimum of two independent implementations.
One caveat is that you'll have to set up a server (or Docker image) that exposes an API to "issue" and "verify" so we can run the W3C VCWG tests against your software:
https://github.com/w3c/vc-di-ecdsa-test-suite#implementation
Do you think you'd be able to do that in the next 3-5 months?

I believe the solution was, for just the JCS cryptosuite proofs, to include the context from the document (and any JCS cryptosuite context, if necessary) in the JCS cryptosuite proof object itself (proof: [{"@context": "<here>", cryptosuite: "eddsa-jcs-2022", ...}, ...]) -- and update the algorithm steps for JCS cryptosuites to use it to override the document's context prior to signing or verifying.

Thanks for clarifying! This seems like a reasonable compromise so far. I'll have to take some time to wrap my head around how this must be done to maintain integrity protection for the context values and hence for the semantics of the secured document, but I'll open an issue including your suggestion tomorrow. Feel free to correct me where I'm misrepresenting your ideas. I'll also make sure to mention the constraint that we can't break existing rdf based suites and implementations.

Do you think you'd be able to do that in the next 3-5 months?

I'm confident that we can do that, yes. Depends a little bit on what the scope is; our cryptosuite implementation also works as a drop-in replacement for other suites used with your library, which, as you mentioned should be fairly trivial to do.
If we should rather provide a solution that works standalone without relying on any of your code, we're pretty close to that too - we've got a working implementation of at least basic signing and verification.

Do you think you'd be able to do that in the next 3-5 months?

I'm confident that we can do that, yes.

Great!

our cryptosuite implementation also works as a drop-in replacement for other suites used with your library, which, as you mentioned should be fairly trivial to do.

I believe that as long as the cryptosuite is implemented by you, it should be fine... however,

If we should rather provide a solution that works standalone without relying on any of your code, we're pretty close to that too - we've got a working implementation of at least basic signing and verification.

This option would be better. I know it's a taller ask, so if you're able to do it, wonderful -- that's fully defensible in the WG as a completely independent implementation. If not, there might be some individuals that object because it's not a completely independent implementation (which is debatable, but they'd certainly debate the "it doesn't count unless is a full top-to-bottom implementation).
Any contribution along the lines above will help us get over the finish line, so thank you!

Any contribution along the lines above will help us get over the finish line, so thank you!

@msporny What we would have to align on though is how to handle the injection of the document context on the proof (I'm talking about this line: https://github.com/digitalbazaar/data-integrity/blob/674d5c142b95aa6825349b3bd4e3e31f86154858/lib/DataIntegrityProof.js#L371), which the eddsa-jcs-2022 specs says should not be done. Do you plan to change the library's approach to creating proof data so that it accommodates cryptosuites that do not do this or do this differently? Or should we reimplement createVerifyData on our jcs suite, which from what I can see replaces the default behaviour?

@msporny What we would have to align on though is how to handle the injection of the document context on the proof (I'm talking about this line:
https://github.com/digitalbazaar/data-integrity/blob/674d5c142b95aa6825349b3bd4e3e31f86154858/lib/DataIntegrityProof.js#L371
), which the eddsa-jcs-2022 specs says should not be done.

At this point, we should presume the eddsa-jcs-2022 spec is wrong and we should fix it.

Do you plan to change the library's approach to creating proof data so that it accommodates cryptosuites that do not do this or do this differently? Or should we reimplement createVerifyData on our jcs suite, which from what I can see replaces the default behaviour?

I expect it's the latter, but don't have time in the next week or two to do a deep dive. I'm requesting an opinion from @dlongley and @davidlehn in this thread, which we might want to document here as well:
https://github.com/w3c/vc-data-integrity/issues/225
I'm heads-down trying to get VC v2.0 to the W3C Candidate Recommendation (CR) phase (we're behind schedule by ~2-3 months there). After that is done, I have to circle back around to get BitStringStatusList to CR, and then I'll circle back to the Data Integrity suites to finish them up so we can go through a 2nd CR phase (we'll be forced to do this due to the issue above and because of requests for changes from Google).
That doesn't mean that implementers (such as you, @dlongley, @davidlehn, @HelgeKrueger, and @silverpill) shouldn't figure out the right way to do this and try it out in your implementations. Please do that so that when I get around to writing the text, there is consensus among the implementers on what should be done here.

eddsa-jcs-2022 is a recommended cryptosuite in FEP-8b32, which (to my knowledge) is supported by Fediverse projects Mitra, Bovine and Vervis. One more project is in the process of implementing it (Streams). Two or three others have expressed interest in this proposal.
I don't have data on usage in the wild. At least one implementation (Mitra) already produces and distributes signed objects containing eddsa-jcs-2022 proofs.

Perhaps the existing implementations could be updated to allow both older proofs using eddsa-jcs-2022 that do not include @context in proof (which is what would become non-spec-compliant) but also support proofs that have @context in them (which is what we currently think the spec would change to require).

We will absolutely need your help to prove that, though. If Mitra, Bovine, Vervis, Streams, and the others don't submit endpoints or Docker images proving that they pass the test suites, there are some in the group that will attempt to cut the feature.

@msporny If you're referring to EdDSA Cryptosuite test suite, I think it is unlikely that any of us will make an effort to pass it. The mentioned projects are social web servers, there's no reason for them to implement VC API.
If WG will try to remove eddsa-jcs-2022, we'll fork the vc-di-eddsa spec or consider using a different standard.

Can you see any other way of updating eddsa-jcs-2022 to fix this concern?

I have no control over existing deployments. A breaking change at this point would lead to a lot of coordination problems, so it  should be avoided. Also, injecting @context could be too complicated for implementations that work with plain JSON (the majority of Fediverse software).

Perhaps the existing implementations could be updated to allow both older proofs using eddsa-jcs-2022 that do not include @context in proof (which is what would become non-spec-compliant) but also support eddsa-jcs-2022 proofs that have @context in them (which is what we currently think the spec would change to require).

I can see that adding a conditional rule to the spec could fix the issue while not breaking existing applications. E.g., if we say that when creating a proof, the document SHOULD be added to the proof. However, during issuance, we require updating the document context only if the proof has a @context property.
A remark on the concern by @silverpill:

Also, injecting @context could be too complicated for implementations that work with plain JSON (the majority of Fediverse software).

The data integrity specifications mandate context injection, so if they are implementing them to the letter, this should already be happening. If not, it can't be guaranteed anyway that other implementation would be able to verify the proofs created. It also sounds more complicated than it is; in the end, it is just operating on a set of strings. Context injection for example is adding a fixed string if it's not already in the set, and what we are proposing as a solution is comparing sets of strings and overriding the set if necessary.

@msporny If you're referring to EdDSA Cryptosuite test suite, I think it is unlikely that any of us will make an effort to pass it. The mentioned projects are social web servers, there's no reason for them to implement VC API.

That's really unfortunate, I wish you would have conveyed that before we put in the effort to get it spec'd and adopted by the group. If our primary group of implementers aren't going to make an effort to demonstrate that they're conformant with the specification the WG will almost certainly remove it from the specification.
At this point, we're going to have to depend on a different group of implementers to integrate w/ the test suite and demonstrate conformance (which is a requirement for all global standards produced by the W3C -- we have to demonstrate two independent interoperable implementations).
Integrating w/ the test suite isn't difficult, you just put up an endpoint that can digitally sign a payload and verify a payload. Are you saying that the fediverse community won't even put in that level of effort? I guess the same questions goes out to @HelgeKrueger and @rflechtner? At this point, I believe Digital Bazaar will submit an implementation for testing via the test suite... at a minimum, we need at least one more independent implementation.

The data integrity specifications mandate context injection, so if they are implementing them to the letter, this should already be happening. If not, it can't be guaranteed anyway that other implementation would be able to verify the proofs created. It also sounds more complicated than it is; in the end, it is just operating on a set of strings. Context injection for example is adding a fixed string if it's not already in the set, and what we are proposing as a solution is comparing sets of strings and overriding the set if necessary.

@rflechtner Is it against the spec to sign JSON objects without the @context property? I didn't know about that.
In the following example, should object have @context in order to have a valid proof?
{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://w3id.org/security/data-integrity/v2"
  ],
  "object": {
    "content": "test",
    "proof": {
      "type": "DataIntegrityProof",
      "cryptosuite": "eddsa-jcs-2022",
      ...
    }
  }
}

Integrating w/ the test suite isn't difficult, you just put up an endpoint that can digitally sign a payload and verify a payload. Are you saying that the fediverse community won't even put in that level of effort?

@msporny If it's really that simple I think somebody could do it. When do you expect the group to reach a decision on eddsa-jcs-2022 cryptosuite?
Also, why the test suite requires VC API? Data Integrity standard has many use cases beyond Verifiable Credentials, I'm surprised that such requirement has been introduced.

@msporny Note: I haven't read through the details about what is being modified with eddsa-jcs-2022. I'm just responding about the test suite.

Integrating w/ the test suite isn't difficult, you just put up an endpoint that can digitally sign a payload and verify a payload. Are you saying that the fediverse community won't even put in that level of effort? I guess the same questions goes out to @HelgeKrueger and @rflechtner?

I'm planning to implement the test suite. However, when I first looked at https://github.com/w3c/vc-di-eddsa-test-suite/, I was confused, so it got low priority. The explanation is easier to understand now. Just to clarify what is needed.

Provide something that can be queried on the web at https://test.example
There is a controller document similar to this at https://test.example/controller that provides the verificationMethod https://test.example/controller#key.
I implement a method https://test.example/issue that signs arbitrary documents as the above controller. So the request contains an unsigned document as request body, the response is then the signed document.
I implement a verify method https://test.example/verify that verifies keys, that retrieves the keys from a controller document.
I add the following manifest at https://test.example/manifest then update https://github.com/w3c/vc-test-suite-implementations/tree/main/implementations

{
  "name": "My Company",
  "implementation": "My implementation",
  "issuers": [{
    "id": "",
    "endpoint": "https://test.example/issue",
    "method": "POST",
    "tags": ["eddsa-jcs-2022"]
  }],
  "verifiers": [{
    "id": "",
    "endpoint": "https://test.example/verify",
    "method": "POST",
    "tags": ["eddsa-jcs-2022"]
  }]
}

Answering first to @silverpill:

@rflechtner Is it against the spec to sign JSON objects without the @context property? I didn't know about that.

Yes and no. Signer implementations are expected to add the @context property if it does not yet exist, so while you can sign a document that does not have it, the resulting secured document MUST have a @context.
See https://www.w3.org/TR/vc-data-integrity/#context-injection

In the following example, should object have @context in order to have a valid proof?

That's a more subtle issue, because it depends on what the 'secured document' is here. Are you signing over the whole structure, or only over the contents of object?
The way I see it, if you are doing the former, this may be a valid implementation, although it would be first time I see that the proof property is added to a nested object. I can't tell if the proof being a top-level property is a hard requirement, what is your take on this @dlongley & @msporny? I can say though that many implementations will assume the proof to be present in the root of the document, so this would hurt interoperability.
The latter implementation however would be more problematic, because it would mean that the @context values are not secured and can be modified after signing, changing the semantics of the document. For that reason, and because the context is not part of what I would consider the secured document, this implementation would be deficient in my eyes.

I guess the same questions goes out to @HelgeKrueger and @rflechtner?

As discussed, we're happy to provide an integration with the test suite at the very least for our cryptosuite implementation, using @digitalbazaar/data-integrity as glue code. I can't say how likely it is that we get a completely independent implementation to work with the test suite at this point, I'd have to take a deeper look at the assumptions made there first. For example, controller documents in our system are DL-based DID documents; and while they can be resolved through http gateways like https://dev.uniresolver.io I suspect this could give us a bit of a headache.

@HelgeKrueger wrote:

The explanation is easier to understand now. Just to clarify what is needed.

Yes, that's more or less accurate. You don't need to do the controller document if you use a did:key, which I expect you will... a did:key is just a byte header with the raw public key bytes of a compressed ECDSA or EdDSA key appended to it and base58 encoded. We have code that can generate one for you if necessary, so that part should be simpler than what you wrote above.
@BigBlueHat, @JSAssassin, @aljones15 can help you through the details when the time comes to integrate... they're the ones that are putting together the test suite.

I'm planning to implement the test suite. However, when I first looked at https://github.com/w3c/vc-di-eddsa-test-suite/, I was confused, so it got low priority.

@BigBlueHat, @JSAssassin, @aljones15, please note the sub-par experience that @HelgeKrueger had wrt. integrating w/ the test suites. I agree that the documentation on how to do it is still quite difficult for implementers to understand, and it sounds like we've already lost one implementer because we've made it seem more difficult than it is to implement... and given how precious each implementer/implementation is for the standards ratification process, we need to improve things from where they are right now.

@HelgeKrueger wrote:

I'm planning to implement the test suite.

@rflechtner wrote:

As discussed, we're happy to provide an integration with the test suite ... for our cryptosuite implementation

Excellent, then we're back in business! That means there will most likely be 3 independent implementations, and all we need is two.

@rflechtner wrote:

The way I see it, if you are doing the former, this may be a valid implementation, although it would be first time I see that the proof property is added to a nested object. I can't tell if the proof being a top-level property is a hard requirement, what is your take on this @dlongley & @msporny?

proof is not required to be a top-level property, but as you said, then only the sub-object is secured. That's a perfectly fine thing to do, but as a warning, that might lead developers to think that the entire document is secured when it is definitely not. Those kinds of misalignments lead to security failures, usually really bad ones, so perhaps the specification should say something about this.
We don't want to insist that proof is always at the top level because there are legitimate reasons for it not to be, such as an API result in JSON from a "System A" that can be trusted (because the API result came from "System A" over TLS), but the result contains information that another system, System B, asserted. The information from System B, which would be nested in System A's response, would need to be digitally signed to be trusted (to ensure that System A isn't lying about something System B said).
You could also argue that System A should digitally sign its payload as well, which would include a signature over System B's payload (so you could trust the outer payload AND the inner payload).
Talking about that sort of thing should probably be placed in the Security Considerations section, as talking about application-layer details a bit "out of scope" for the specification... but it's worth mentioning. Would one of you please raise an issue if you think this is worth noting in the specification?

I can say though that many implementations will assume the proof to be present in the root of the document, so this would hurt interoperability.

Yes, this is true. The algorithms are written such that if the proof isn't at the top-level of the object passed into verification, that verification will fail.

That's a more subtle issue, because it depends on what the 'secured document' is here. Are you signing over the whole structure, or only over the contents of object?

@rflechtner In the example above I'm signing only the contents of object. Top-level structure remains unsecured.

The latter implementation however would be more problematic, because it would mean that the @context values are not secured and can be modified after signing, changing the semantics of the document. For that reason, and because the context is not part of what I would consider the secured document, this implementation would be deficient in my eyes.

I would accept this risk. The majority of consumers are expected to be JSON processors, not aware of JSON-LD.
In addition to that example, my software needs to support the following use cases:

Both top-level structure and embedded object are signed.
Signing a JSON document without @context property.

https://www.w3.org/TR/vc-data-integrity/#context-injection says

If an @context property is not provided in a document that is being secured or verified, or the Data Integrity terms used in the document are not mapped by existing values in the @context property, implementations MUST inject or add an @context property with a value of https://w3id.org/security/data-integrity/v2.

but it is not entirely clear when and how to inject it. If context injection is a required part of the proof generation, it probably should be specified as an optional (?) step in https://www.w3.org/TR/vc-di-eddsa/#algorithms

@BigBlueHat, @JSAssassin, @aljones15, @msporny

Yes, that's more or less accurate. You don't need to do the controller document if you use a did:key, which I expect you will... a did:key is just a byte header with the raw public key bytes of a compressed ECDSA or EdDSA key appended to it and base58 encoded. We have code that can generate one for you if necessary, so that part should be simpler than what you wrote above.

I think the use did:key is an excellent hint to simplify implementing the test suite.
Assuming did:key, one can both avoid the controller document and the parsing logic for controller documents. This means one is back to only testing the signing/verifying algorithms (as far as I can tell now).
My experience from the Fediverse is: Resolving controller documents is hard. In the Fediverse the controller is called an actor and the public key is in the now deprecated publicKey property. This tool, I build, helps ensure that people do it correctly.

@HelgeKrueger wrote:

I think the use did:key is an excellent hint to simplify implementing the test suite. Assuming did:key, one can both avoid the controller document and the parsing logic for controller documents.

Yes, exactly. That is why we only require did:key for the test suites.

This means one is back to only testing the signing/verifying algorithms (as far as I can tell now).

Yes, correct. We're trying to balance at least two things:

Making it as easy as possible for implementers to integrate with the test suite.
Ensuring that the test suite is run on a regular basis (e.g., at least weekly) against all the implementations that we know about.

My experience from the Fediverse is: Resolving controller documents is hard. In the Fediverse the controller is called an actor and the public key is in the now deprecated publicKey property. This tool, I build, helps ensure that people do it correctly.

Excellent! Thank you for writing that tool. We are attempting to write similar testing tools for the W3C Verifiable Credentials ecosystem, and our hope is that some of these tools w/ overlap w/ Fediverse needs.
As a related aside, this link is 404'ing, and I was hoping to see a list of conforming implementations there: https://funfedi.dev/support_tables/verify/ ... We define "Controller Documents" in https://www.w3.org/TR/vc-data-integrity/#controller-documents now, and we're thinking of moving the concept of "Actor/Controller Documents into their own standard at W3C: https://w3c.github.io/vc-controller-document/ (but may not be able to finish that work off during this round of the official Verifiable Credentials Working Group).

@rflechtner In the example above I'm signing only the contents of object. Top-level structure remains unsecured.

I think we all agree on one thing here: the 'secured document', that is, the structure that is secured by a proof, is the one that has the proof property. I also see that there are valid reasons to nest this secured document within a larger structure and have no qualms with that.
The implication as I read it is that the @context property in the top level is irrelevant here, because a @context property MUST be added, explicitly or implicitly, before signing and verification. By that I mean: Even if it is not present on the secured document, the specification calls for you to add it during the signing process, so that what's actually being signed looks like this:
{
    "@context": [
      "https://w3id.org/security/data-integrity/v2"
    ],
    "content": "test",
    "proof": {
      "type": "DataIntegrityProof",
      "cryptosuite": "eddsa-jcs-2022",
      ...
     }
}

In turn, if you sent the document from your example to me, I would be adding the @context property to the secured document during verification, so that I could only successfully verify your proof if you had implicitly applied the transformation to the document above during signing.

I would accept this risk. The majority of consumers are expected to be JSON processors, not aware of JSON-LD.

The VC data integrity specifications have been designed with interoperability between jcs & rdfc based proof methods in mind;  as @dlongley stated initially, this was an explicit design goal.
We miss this goal if consumers of a signed piece of data cannot be sure that what they are presented with matches that which the issuer had originally signed. Be aware that changing the mapping provided by the context file can completely transform the meaning of this data in the eyes of a linked data processing application, so some mechanism needs to be in place to prevent this.
While this mechanism might not be particularly important for your use case, I think it is necessary to make this specification a success, as not having it would introduce nasty gotchas to this method of authenticating third party supplied data. And as outlined above, choosing not to implement it would mean that no other implementation would be able to verify the proofs you generated, which defeats the purpose of agreeing on a specification.

In addition to that example, my software needs to support the following use cases:

Both top-level structure and embedded object are signed.
Signing a JSON document without @context property.

The first should be no issue as far as I can see; simply sign the embedded object first, and the top-level structure second. Since we've established that a proof property concerns the object in which it is embedded, the mapping of which proof pertains to which secured document is unambiguous.
Signing a JSON document without a @context property is possible as long the implementation adds a @context property containing [ "https://w3id.org/security/data-integrity/v2" ] in the process of signing. Ideally, this is done so that the modified document is returned by this implementation, as the property only existing for signing and verification algorithms, but never actually being visible to processing applications would be another confusing feature in the system.

If context injection is a required part of the proof generation, it probably should be specified as an optional (?) step in https://www.w3.org/TR/vc-di-eddsa/#algorithms

I agree that how to handle context injection is underdeveloped and needs to be made much more explicit in both the VC data integrity and the vc-di-eddsa specifications. That is why I opened issue https://github.com/w3c/vc-data-integrity/issues/225, but I can open another one that specifically asks to add language that clarifies when context injection should happen (e.g., do we really perform it on verification, or should we just error if the expected context isn't present?), and open a third on https://github.com/w3c/vc-di-eddsa/ asking to explicitly point out that this step needs to be performed.

Thank you for this explanation @rflechtner !
I incorrectly assumed that injection is not needed if top-level structure already has @context.

Signing a JSON document without a @context property is possible as long the implementation adds a @context property containing [ "https://w3id.org/security/data-integrity/v2" ] in the process of signing.

Did you mean containing "https://w3id.org/security/data-integrity/v2", or it actually must be an array?

That is why I opened issue https://github.com/w3c/vc-data-integrity/issues/225, but I can open another one that specifically asks to add language that clarifies when context injection should happen (e.g., do we really perform it on verification, or should we just error if the expected context isn't present?), and open a third on https://github.com/w3c/vc-di-eddsa/ asking to explicitly point out that this step needs to be performed.

Yes, I'd appreciate if you open these additional issues

Did you mean containing "https://w3id.org/security/data-integrity/v2", or it actually must be an array?

That's one of the things that really should be properly defined in the specifications, especially if we treat adding the context on verification as allowable. Otherwise implementations may do this differently, resulting interoperability issues. I added this to the necessary clarifications in https://github.com/w3c/vc-data-integrity/issues/231

PRs w3c/vc-di-eddsa#79, w3c/vc-di-eddsa#80, w3c/vc-di-ecdsa#61, and w3c/vc-di-ecdsa#62 were raised and merged to address this issue (discussion happened in those issues).
This DB data-integrity library (really the cryptosuites) now need to implement this feature. I expect the libraries will fail the test suite until they properly implement the feature. I'm going to leave this open for now until we have a working set of cryptosuites that test successfully against the W3C test suite.

@HelgeKrueger sorry for the late response here, have you managed to get into the test suite yet? We are planning on updating the Data Integrity tests very soon. Are you part of the test suite office hours? We do have support for implementers. Anyways thanks for the feedback and I will try to further digest this thread.


:truck: chore: update svelte&kit, and use esm-env to detect if on browser

See https://github.com/sveltejs/kit/issues/5879#issuecomment-1344927680 for the reason, this allows us to get in compile time the value of BROWSER safely without any extra need from the user, allowing for omitting unneeded code during minification when the user build his project.
In addition, also upgrade the dependencies of the project for svelte&kit (and the new @sveltejs/package for packaging).

@orefalo please merge if you agree, didn't want to upgrade versions by myself

looks good Tal, as usual.
I wonder if the issue on sveltejs/kit#5879 (comment) shouldn't be raised as a bug on sveltekit

Look, it was fixed in https://github.com/sveltejs/kit/issues/8033.
The idea is to restrict the usage of $app path from libraries and warn them, and for the envrionment, we can use the universal package that detect if on browser for every environment.

actually, there is an issue with the last merge...
the REPL breaks with -> cannot find "/index.js" in<EMAIL_ADDRESS>https://svelte.dev/repl/1e96cbd4bcd148e3b85a4c8ca76d7309?version=3.48.0

OK, please keep this version there for this week so the author of esm-env(Ben, part of the Svelte team) will see this issue.

actually, there is an issue with the last merge...
the REPL breaks with -> cannot find "/index.js" in<EMAIL_ADDRESS>https://svelte.dev/repl/1e96cbd4bcd148e3b85a4c8ca76d7309?version=3.48.0

OK, please keep this version there for this week so the author of esm-env(Ben, part of the Svelte team) will see this issue.

Update: It seems that the solution there won't be in the near future for now, so I'm working now on an alternative to esm-env that will have a fallback handling for Svelte REPL, and will apply this to this package.


Train on custom data-bounding box wrong position

Hi- I could successfully train the model on my data set along with all COCO classes. I had added "Divider" as new class. As per tensorboard visualization,  bounding box is fitting well for all COCO 80 classes but for the new class 'Divider' it is not fitting well. Always, the bounding box is fitted on the top of the object 'Divider'.  Would request some suggestion and help to fix this issue.

hello,
are you sure of the coordinate you give into your coco dataset format ? If I remember well, contrary to PASCAL VOC which is xmin ymin xmax and ymax coordinate, coco format is xmin/ymin and h/w


Dp structures

New possibility to export/import clusters structures.

Sorry it's taken so long to get to this! I've finally had a chance to sit down with this and I think we need some more info in the structures data we're getting back, but I also realise we didn't really spec this out at all before starting out on it.
Let's try to spec this out better before we take it any further.

OK so I had a chance to put together a mockup of what I was looking for here. Below is a JSON snippet with an example of what I'm after.
    "clusters": [
        {
            "name": "default",
            "generation": 4,
            "riak_config": null,
            "advanced_config": null,
            "nodes": [
                {
                    "name": "riak-ts-default-1",
                    "status": "started",
                    "container_path": "data",
                    "persistence_id": "4d98228b-4049-4af3-a677-2798ea105c07"
                },
                {
                    "name": "riak-ts-default-2",
                    "status": "started",
                    "container_path": "data",
                    "persistence_id": "8b263a66-054f-4df0-97e6-746b632aba15"
                },
                {
                    "name": "riak-ts-default-3",
                    "status": "started",
                    "container_path": "data",
                    "persistence_id": "06346eab-abc7-40ad-8d68-1977db3b390d"
                }
            ]
        }
    ]

This is a subset of the information available from rms_cluster_manager:get_cluster("default") and rms_node_manager:get_node("riak-ts-default-1"). Data such as persistence_id would need to be optional and maybe we don't even include it yet.
For now, we exclude port numbers because we don't have a way to re-use ports.
Armed with this exact data (even without the persistence_id), we have enough to recreate the cluster in a fresh scheduler.

.clusters.riak_config and .clusters.advanced_config would just be the string format of the config file set within the scheduler, if it has been set.

I feel like this should be in a separate module, which we can then expose via HTTP from rms_wm_resource for the CLI. That module can then become the entrypoint for the next steps that build on this, where we'll start building out the machinery that takes this config back in, and applies it to create clusters.

LGTM :shipit: 👍


Feedback on First Run Experience

I'm set up with a development version of Sulu now. Great job. It seems awesome!
I had two issues getting started:

PostgreSQL support needs some love. #516
I forgot to set the SYMFONY_ENV so had a mysterious getTheme on a non-object error trying to run locally #518 — a decent error message would help there.

My only other observation beyond these is that it's a bit of a pain to have to set up Nginx or Apache (+ FastCGI, on the documented approach) just to try out the project.
It would be great if there was a bundled router.php so that users could just clone, install dependencies, run the build script, and so on before using the development server to get up and running.

Ah, this is what all of this was about :smile: Actually there are already router scripts located at app/config/router_admin.php and app/config/router_website.php, so it can easily be run with app/console server:run -r app/config/router_admin.php and accessing localhost:8000 afterwards. I guess this information is missing in the documentation (or at least at the wrong place, there is a mention of this at http://docs.sulu.io/en/latest/developer/behat/getting-started.html#running-the-tests). /cc @joemoe

... there are already router scripts...

Ah-ha! "Testing" was my next reading topic. :-)
A quick app/console server:run -r app/config/router_admin.php gives me No route found for "GET /". (But now I know where to look I can experiment there.)
Thanks for the help getting off the ground.

No problem. But it would really be interesting to add these to the getting started documentation, for people who really just want to try it out as fast as possible.

+1

https://github.com/sulu-io/sulu/issues/2078

This linked issue seems to be solved and since introducing sulu-minimal php development server works without problems


recurse directories in DirectoryIterator

Added functionality to recurse subdirectories when using flow_from_directory, optionally also follow symlinks.
I find it useful since my image data is on a separate device from my code, organised in subdirectories for a reason. I think it might be useful for others too.

Can you elaborate on how the class categories are derived in the recursive case?

Hello @isaacgerg.
Sub-directories don't map to sub-classes in my code. The only change in the code is to traverse the subdirectories inside the class directories, rather than only pick the files in the specified directory. In case where there are no sub-directories, I believe the code works as it used to.
Imagine a scenario where the cat/dog training dataset from kaggle is supplemented with a large number of cat and dog photos from other sources:
/cats
/cats/cats-from-kaggle
/cats/photos-of-wild-cats
/cats/external-disk -> /mnt/external-disk/cat-photos
/cats/network-volume -> /mnt/mounted-network-volume/cat-photos
/dogs
/dogs/dogs-from-kaggle
/dogs/dog-show-archive
/dogs/external-disk -> /mnt/external-disk/dog-photos
/dogs/network-volume -> /mnt/mounted-network-volume/dog-photos

With subfolders and links, data sources can be added or removed without actually moving or copying files. This can be especially useful with:

shared network volumes, used also by other users
datasets with more categories than needed for the task
very large collections of images that are chunked into subdirectories

I hope that helps.

@lunardog This sounds good.  Just wanted to make sure I understood.

Is this functionality unit-tested, by the way? If it's not, it would be good at add tests.

Thanks for reminding me, @fchollet. I added a test and found a bug in the process - the iterator collected 6 images, but only 4 file names. Fixed now, should be correct.
By the way, I added some test images (public domain, from Pixabay) to tests/integration_tests/images. Is that a good way to go about it?

The tests failed due to a timeout downloading Theano from github.
Cloning git://github.com/Theano/Theano.git to /tmp/pip-q22Mqh-build
fatal: unable to connect to github.com:
github.com[0: <IP_ADDRESS>]: errno=Connection timed out
github.com[1: <IP_ADDRESS>]: errno=Connection timed out
Command "git clone -q git://github.com/Theano/Theano.git /tmp/pip-q22Mqh-build" failed with error code 128 in None
The command "pip install git+git://github.com/Theano/Theano.git" failed and exited with 1 during .
Your build has been stopped.

Rather than adding an integration test, please add a unit test for flow_from_directory in tests/keras/processing/test_image.py.

@fchollet

I put the test in processing, not integration_Tests
I added sorting
I kept the underscore in follow_links to match Keras API

Sorry about all the oops commits. Should I squash and make a new PR?

Made a new, properly squashed PR. Closing.


#2742 [Part 5] Allow editing attachment points by context menu

This PR is one of the small PRs to resolve #2742  for easy review.
This PR is open for reviews but please do not merge it.
What is done in this PR?

Allow editing R-Group attachment points by context menu

Allow removing R-Group attachment points by context menu

Check list

[ ] unit-tests written
[ ] e2e-tests written
[ ] documentation updated
[ ] PR name follows the pattern #1234 – issue name
[x] branch name doesn't contain '#'
[x] PR is linked with the issue
[x] base branch (master or release/xx) is correct
[ ] task status changed to "Code review"

Merged with https://github.com/epam/ketcher/pull/2978


Don't hardwire version numbers in examples

Are you sure you still want to hardwire version numbers into your examples there on http://openlayers.org/en/v3.12.1/doc/quickstart.html ?
Google doesn't do that anymore in their examples.
https://developers.google.com/maps/tutorials/fundamentals/adding-a-google-map
Compare src="https://maps.googleapis.com/maps/api/js" vs. src="http://openlayers.org/en/v3.12.1/build/ol.js".

In fact, this penchant for adding version numbers results in major fragmentation for any Google search results:
openlayers.org/en/v3.4.0/examples/kml.html
openlayers.org/en/v3.2.1/examples/
openlayers.org/en/v3.8.2/examples/simple.html

https://github.com/openlayers/openlayers.github.io/pull/62


[BUG] Bot Detector always fails to activate due to 'Invalid command line arguments'

Bot Detector always fails to activate when launching TF2, it provides the error below:

This occurs despite any or no command line arguments. Neither Steam nor TF2 are set to run in Administrator mode.
TF2 Console gives this error:
Invalid rich presence string: " -usercon -high +developer 1 +alias developer +contimes 0 +alias contimes +ip <IP_ADDRESS> +alias ip +sv_rcon_whitelist_address <IP_ADDRESS> +alias sv_rcon_whitelist_address +sv_quota_stringcmdspersecond 1000000 +alias sv_quota_stringcmdspersecond +rcon_password oYbkYnLw4zqODykT +alias rcon_password +hostport 53907 +alias hostport +alias cl_reload_localization_files +net_start +con_timestamp 1 +alias con_timestamp -condebug -conclearlog"
For some reason I'm unable to upload my log file.
Microsoft Windows 10 Home 10.0.19042 Build 19042
TF2 BotDetector version <IP_ADDRESS>8
I haven't played in several months but previously had no issues.

2021-06-26_13-57-35.log.txt

#331
update to nightly


Add a zkASM architecture

This is a very initial take on this if only to be able to specify zkasm-unknown-unknown target in tooling that uses target-lexicon. Some of the details are not exactly 100% correct in the current state of the machine (e.g. pointer width is 256 bits), but this will potentially change soon…

@sunfishcode would it be possible to get a review/merge here?

zkasm doesn't appear to be a Rust or LLVM target name. We can potentially add it, but since I'm not familiar with zkasm at all: how confident are you that "zkasm" will remain the name of this architecture, and that no other architecture will call itself "zkasm", for the forseeable future?

zkasm doesn't appear to be a Rust or LLVM target name.

This is expected – we’re just now are experimenting with implementing a backend for this architecture in cranelift. Even then it might very well end up as an external backend, than something that's upstream. For the time being we’ve hijacked sparc to refer to our target, but that's quite inconvenient too.

We can potentially add it, but since I'm not familiar with zkasm at all: how confident are you that "zkasm" will remain the name of this architecture, and that no other architecture will call itself "zkasm", for the forseeable future?

Unfortunately I have little I can guarantee about either of these questions. The zkASM project is driven by the Polygon team (which I have a contact with but am not a part of,) and since zkASM is not a trademark to the best of my knowledge I or Polygon have little control over what other groups come up with and how they name their projects. That said, a quick search on a search engine for this name will bring Polygon’s project up in the first page of the search results, so that’s as good as reservation can get for the time being?
For what it is worth, I’m perfectly happy with this target not being set in stone in perpetuity. I anticipate the benefit of doing so to be very low and I would imagine it wouldn’t be too onerous to repurpose or remove the architecture string from target-lexicon if a need to do so arises.

Yeah, target-lexicon is kind of in a somewhat awkward position, being used by cranelift which doesn't need to be limited to stable targets, but also having become more broadly used by libraries that want a stable API.
Would it make sense to add zkarch under a feature flag, like #[cfg(feature = "zkarch")], which cranelift could enable, and which could be documented like this?
[features]
...
# Enable unstable zkarch architecture support.
zkarch = []

Yeah, I think we can do that no problem.

Done. I ended up picking arch_zkasm here for the feature name, with a thought that if there were ever further future extensions of this sort, it would make a lot of sense to have them behind arch_*, os_* or similar features as well.

Looks good, thanks!


vscodium: <IP_ADDRESS>72 -> <IP_ADDRESS>96

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for vscodium is: Open source source code editor developed by Microsoft for Windows,
Linux and macOS (VS Code without MS branding/telemetry/licensing)
meta.homepage for vscodium is: https://github.com/VSCodium/vscodium
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found <IP_ADDRESS>96 in filename of file in /nix/store/mwwpi3h33iwwbfzfs6isvv29c4k1s12f-vscodium-<IP_ADDRESS>96

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)

3 package rebuild(s)

First fifty rebuilds by attrpath
vscode-langservers-extracted
vscodium
vscodium-fhs

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/mwwpi3h33iwwbfzfs6isvv29c4k1s12f-vscodium-<IP_ADDRESS>96 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A vscodium https://github.com/r-ryantm/nixpkgs/archive/62e096cab68d926b427410a3a0abe674b8e15195.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/62e096cab68d926b427410a3a0abe674b8e15195#vscodium

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/mwwpi3h33iwwbfzfs6isvv29c4k1s12f-vscodium-<IP_ADDRESS>96
ls -la /nix/store/mwwpi3h33iwwbfzfs6isvv29c4k1s12f-vscodium-<IP_ADDRESS>96/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review --extra-nixpkgs-config '{ allowInsecurePredicate = x: true; }' run on x86_64-linux 1

3 packages built:

vscode-langservers-extracted
vscodium
vscodium-fhs

Maintainer pings
cc @Synthetica9 @bobby285271 @ludovicopiero for testing.

Would be great to backport this into 23.11
see also:

https://github.com/NixOS/nixpkgs/issues/299004
https://github.com/ryantm/nixpkgs-update/issues/65


[Ignore] Test issue created to check replacement for fabric bot

This is a test issue.

Closing this issue.


reverted changes to python trivy workflows

@danieldaviestelicent I'd appreciate some pointers to builds that were broken by this change that needed this revert because I haven't been able to find any obvious ones but I'll likely looking in all the wrong places
It'd be much better to understand what broke and why so we can fix it as otherwise your builds are going to be continued to be plagued by Trivy DB download failures.
For Java builds there's already a marked reduction in these errors once the cache is primed for each repository

It's on a private repo so will move this converation to an internal ticket tomorrow


Sign patch id instead of commit hash

also adds github webhook triggering via tag update hack

Signed-off-by: Tyler Levine<EMAIL_ADDRESS>

This breaks tests because signing the genesis commit is poorly supported. Will fix later.

Signing the genesis commit is an interesting edge case in your scheme, but hey, tests did their job :D

Kinda-dumb question: why is this PR targetting mac-compatibility rather than master?

@tylerlevine Thanks a lot for the PR, it's a really cool idea.
I left you a bunch of actionable feedback; sorry it took so long before I could make the time to review.  :3

I think we need to allow the future ability to disallow out of order merges as that can have security consequences. To do this we would need the current master head commit hash to be both inside and outside the signature for reference so future git-signatures can flag if a diff was applied earlier or later than intended.
Other than that and kellers suggestions I am +1

Kinda-dumb question: why is this PR targetting mac-compatibility rather than master?

Only because this branch depended on changes in the mac-compatibility branch when it was created, and then that branch was merged into master. I guess github isn't smart enough to update the PR target if that ref is merged somewhere. I'm expecting that this PR will land in master now.

I think we need to allow the future ability to disallow out of order merges as that can have security consequences. To do this we would need the current master head commit hash to be both inside and outside the signature for reference so future git-signatures can flag if a diff was applied earlier or later than intended.
Other than that and kellers suggestions I am +1

I've already started working on that. Essentially this changes the meaning of the review signature from acceptance of a certain patch, to acceptance of a certain patch relative to another commit. If the author tries to land an accepted patch on a commit other than the one approved by the reviewer, the review signature would be invalid. I think that's pretty cool.

@lrvick Yes, we should enforce that the merges stack up properly; that possibly involves putting the base commit hash into the signature, in addition to the patch id?

@KellerFuchs Yeah pretty much just need to sign the patch-id/master-head combo but leave them in the clear so then a future flag on "git signatures verify" will have the information it needs to be opt in/out of that level of strictness. It does mean you can't really have more than one ready-to-merge PR at a time so people may choose to disable this level of strictness on less sensitive repos.
Tradeoffs.

Oh, of course it doesn't show after rebase, even though the diff is the same:
$ ./bin/git-signatures show
 Public Key ID    | Status     | Trust     | Date                         | Signer Name
=======================================================================================================================
error: no note found for object 0d0f7ac917b3bfae9b1f5ee308866494d9206d19.

Oh, of course it doesn't show after rebase, even though the diff is the same:
$ ./bin/git-signatures show
 Public Key ID    | Status     | Trust     | Date                         | Signer Name
=======================================================================================================================
error: no note found for object 0d0f7ac917b3bfae9b1f5ee308866494d9206d19.

Try rerunning git signatures init so that notes.rewriteRef gets set properly. Also I don't have the pubkey you used, here is the one I have for you (which is also present in .gitsigners):
pub   rsa4096/0527B4EFFB4A3AEB 2017-10-31 [C] [expires: 2020-10-30]
      AE2D535ABD2E5B42CE1E97110527B4EFFB4A3AEB
uid                 [ultimate] Keller Fuchs
uid                 [ultimate]<EMAIL_ADDRESS>sub   rsa4096/7AE119B545B40B2A 2017-10-31 [S] [expires: 2018-10-31]

Pubkey used to sign was instead FA1FC72F261DC619

$ ./bin/git-signatures init
$ ./bin/git-signatures show
 Public Key ID    | Status     | Trust     | Date                         | Signer Name
=======================================================================================================================
error: no note found for object 0d0f7ac917b3bfae9b1f5ee308866494d9206d19.

Re: signing key, indeed somehow only my rsa signing key got added...

So I have been thinking a lot on this, and realized there is already yet one more thing we will want to potentially attest to in a future: The compiled binary hash this given patch-id produces.
So far the things that make sense to sign are:

patch-id (always)
target head (optional)
compiled binary hash (optional)

There may yet be others we have not considered yet.
For this reason since we are making a breaking change to the signing format with this commit, let's tweak it to be future proof by using key/value pairs.
JSON sounds the easiest for this, but that will require we depend on jq (parsing json in bash in hell). Jq is becoming very common though so maybe this is acceptable until someone is willing to rewrite this whole thing as a Rust binary or somthing.
Anyway I was thinking a signature could then just be a single line of json containing a body and a signature for that body
{body:{ patchid:"somepatchid",bin-hash:"somebinhash",target-head:"targetheadhere"},signature:"owGbwMvMwMHY577wjenCUFnG00FJDNFzp3pWFySWJGdkplgpFefnpkI5SjpJmXm6GYnFGRBhIA/E
UdIpSSxKTy3RzUhNBGqAcEDsjNSiVKVark4mYxYGRg4GWTFFlvRQ+71LsrdL7dXj5oZZysoEspGB
i1MAJsIzkf8P37uGc1wx5SJH0kK6uGU+Lbu2WVtq9brA62/f5arGnBG4ct37pEa9++LuTZMPuXKs
9L5o1t7YvqRX25qJR3RLdpnRp6fTli7R1u6Qi/sqFjKv5nfbz+1MnB8LE8Wdt3Q1ac2ekLr/4Ydc
1tzimohla6VqA3YdmnvA9NGes/9Pr1+82nzKjIshU5L0VXI8bl6UXdp/vmNNHcfS1FXc+85N7mxj
3NH++IjWVguPZ8Vrv7zb5/EouuFHgk3z8eNCrtf+BVbP+NAic7h88c6K7UcTpp7Y/lasRkWrJDVI
YUv2026zsqjDKYdCAj1KXKq/2x1O2mssy5+sP5nr+Ap7UWm34/v3aTocYkvm3eZ8OfdIjNbz31cv
eNy78T3H7/4inVrZZ78nz9f4LWvx0H7p9mu3pwXFWudxdv6q5L8Z+65t+VnJA/O37uGwL+d+bjrl
rvk9ruVHDXSn91WdfjaPWWP1XwbBaw1WwtN85orfOt590HLKdluHr2E7Dj/onfdjVYnvaknr/Ohr
8afT53TY8iV38GRNvjfT1s0rbGqq2AOlRX1OLpLL9NaEmnSWtH7+7ZszzWNBeH1YUZKJTeXhfYl3
3t/lknoVeNmr9JHGjCKh9LUmhVsXcFYHLtnwKWTvBUed2WHpR1RZTZt+8Uxruhi7r6py4qnnt/te
Xc3N1HRerfDkQfXWtbGV5zNSPjzmD5nSeWTZ1u8/ZcWSNs8+AQA="}
I know I am feature creeping to hell right now so we may want to break this out as a follow up. I just want to find a way to avoid more breaking changes as we add more optional metadata to sign as it relates to a given set of code changes.

Also: mac-comptibility was already merged. I think this needs to target master.

@lrvick I'm not sure it makes sense to stuff this in the same git note as the source review info.

Also, yes, please make a separate issue.

@KellerFuchs If this metadata does not go in the same git note, then it is not all signed together. For projects that yeild a binary signing the binary with the commit hash review attests this code yeilds this hash. For non binary projects this would not be required.
This sort of feature is totally out of scope, but I am making the case that we now have 2 different bits of data that need to be signed by each reviewer and more in the future, thus it probably makes sense to use a key/value structure leaving us room to expand in the future.

So I have been thinking a lot on this, and realized there is already yet one more thing we will want to potentially attest to in future revisons: The compiled binary hash this given patch-id produces.
So far the things that make sense to sign are:

patch-id (always)
target head (optional)
compiled binary hash (optional)

There may yet be others we have not considered yet.
For this reason since we are making a breaking change to the signing format with this commit, let's tweak it to be future proof by using key/value pairs.
JSON sounds the easiest for this, but that will require we depend on jq (parsing json in bash in hell). Jq is becoming very common though so maybe this is acceptable until someone is willing to rewrite this whole thing as a Rust binary or somthing.
Anyway I was thinking a signature could then just be a single line of json containing a body and a signature for that body
{body:{ patchid:"somepatchid",bin-hash:"somebinhash",target-head:"targetheadhere"},signature:"owGbwMvMwMHY577wjenCUFnG00FJDNFzp3pWFySWJGdkplgpFefnpkI5SjpJmXm6GYnFGRBhIA/E
UdIpSSxKTy3RzUhNBGqAcEDsjNSiVKVark4mYxYGRg4GWTFFlvRQ+71LsrdL7dXj5oZZysoEspGB
i1MAJsIzkf8P37uGc1wx5SJH0kK6uGU+Lbu2WVtq9brA62/f5arGnBG4ct37pEa9++LuTZMPuXKs
9L5o1t7YvqRX25qJR3RLdpnRp6fTli7R1u6Qi/sqFjKv5nfbz+1MnB8LE8Wdt3Q1ac2ekLr/4Ydc
1tzimohla6VqA3YdmnvA9NGes/9Pr1+82nzKjIshU5L0VXI8bl6UXdp/vmNNHcfS1FXc+85N7mxj
3NH++IjWVguPZ8Vrv7zb5/EouuFHgk3z8eNCrtf+BVbP+NAic7h88c6K7UcTpp7Y/lasRkWrJDVI
YUv2026zsqjDKYdCAj1KXKq/2x1O2mssy5+sP5nr+Ap7UWm34/v3aTocYkvm3eZ8OfdIjNbz31cv
eNy78T3H7/4inVrZZ78nz9f4LWvx0H7p9mu3pwXFWudxdv6q5L8Z+65t+VnJA/O37uGwL+d+bjrl
rvk9ruVHDXSn91WdfjaPWWP1XwbBaw1WwtN85orfOt590HLKdluHr2E7Dj/onfdjVYnvaknr/Ohr
8afT53TY8iV38GRNvjfT1s0rbGqq2AOlRX1OLpLL9NaEmnSWtH7+7ZszzWNBeH1YUZKJTeXhfYl3
3t/lknoVeNmr9JHGjCKh9LUmhVsXcFYHLtnwKWTvBUed2WHpR1RZTZt+8Uxruhi7r6py4qnnt/te
Xc3N1HRerfDkQfXWtbGV5zNSPjzmD5nSeWTZ1u8/ZcWSNs8+AQA="}
I know I am feature creeping to hell right now so we may want to break this out as a follow up. I just want to find a way to avoid more breaking changes as we add more optional metadata to sign as it relates to a given set of code changes.

I'm not sure how useful that is unless your build system is deterministic. Can we break this out into a separate issue for further discussion?

$ ./bin/git-signatures init
$ ./bin/git-signatures show
 Public Key ID    | Status     | Trust     | Date                         | Signer Name
=======================================================================================================================
error: no note found for object 0d0f7ac917b3bfae9b1f5ee308866494d9206d19.

Re: signing key, indeed somehow only my rsa signing key got added...

You'll have to redo the rebase (the ref in notes.rewriteRef only gets moved when doing rebases or amends).

So I have been thinking a lot on this, and realized there is already yet one more thing we will want to potentially attest to in future revisons: The compiled binary hash this given patch-id produces.
So far the things that make sense to sign are:

patch-id (always)
target head (optional)
compiled binary hash (optional)

There may yet be others we have not considered yet.
For this reason since we are making a breaking change to the signing format with this commit, let's tweak it to be future proof by using key/value pairs.
JSON sounds the easiest for this, but that will require we depend on jq (parsing json in bash in hell). Jq is becoming very common though so maybe this is acceptable until someone is willing to rewrite this whole thing as a Rust binary or somthing.
Anyway I was thinking a signature could then just be a single line of json containing a body and a signature for that body
{body:{ patchid:"somepatchid",bin-hash:"somebinhash",target-head:"targetheadhere"},signature:"owGbwMvMwMHY577wjenCUFnG00FJDNFzp3pWFySWJGdkplgpFefnpkI5SjpJmXm6GYnFGRBhIA/E
UdIpSSxKTy3RzUhNBGqAcEDsjNSiVKVark4mYxYGRg4GWTFFlvRQ+71LsrdL7dXj5oZZysoEspGB
i1MAJsIzkf8P37uGc1wx5SJH0kK6uGU+Lbu2WVtq9brA62/f5arGnBG4ct37pEa9++LuTZMPuXKs
9L5o1t7YvqRX25qJR3RLdpnRp6fTli7R1u6Qi/sqFjKv5nfbz+1MnB8LE8Wdt3Q1ac2ekLr/4Ydc
1tzimohla6VqA3YdmnvA9NGes/9Pr1+82nzKjIshU5L0VXI8bl6UXdp/vmNNHcfS1FXc+85N7mxj
3NH++IjWVguPZ8Vrv7zb5/EouuFHgk3z8eNCrtf+BVbP+NAic7h88c6K7UcTpp7Y/lasRkWrJDVI
YUv2026zsqjDKYdCAj1KXKq/2x1O2mssy5+sP5nr+Ap7UWm34/v3aTocYkvm3eZ8OfdIjNbz31cv
eNy78T3H7/4inVrZZ78nz9f4LWvx0H7p9mu3pwXFWudxdv6q5L8Z+65t+VnJA/O37uGwL+d+bjrl
rvk9ruVHDXSn91WdfjaPWWP1XwbBaw1WwtN85orfOt590HLKdluHr2E7Dj/onfdjVYnvaknr/Ohr
8afT53TY8iV38GRNvjfT1s0rbGqq2AOlRX1OLpLL9NaEmnSWtH7+7ZszzWNBeH1YUZKJTeXhfYl3
3t/lknoVeNmr9JHGjCKh9LUmhVsXcFYHLtnwKWTvBUed2WHpR1RZTZt+8Uxruhi7r6py4qnnt/te
Xc3N1HRerfDkQfXWtbGV5zNSPjzmD5nSeWTZ1u8/ZcWSNs8+AQA="}
I know I am feature creeping to hell right now so we may want to break this out as a follow up. I just want to find a way to avoid more breaking changes as we add more optional metadata to sign as it relates to a given set of code changes.

I'm not sure how useful that is unless your build system is deterministic. Can we break this out into a separate issue for further discussion?

Deterministic builds are becoming very much the standard for open source or security focused projects. You can't reliably assert that a given binary came from a given code from more than one location otherwise. But yeah we can let this merge and discuss a payload refactor to include generic metadata like this after.
I think this project is early enough that we can justify breaking changes easily enough.

@lrvick Please make a separate issue.

CI keeps failing with BADSIG. Is there some some missing deps in the CI configs?

It seems strange that it's fine on travis but failing on gitlab. Any idea what's wrong? I can validate the signatures locally.

I think this is an acceptable alternative rafasc@f39eee0833f32f5d7b08100ebba02dfcf05fa089

@rafasc You had dropped off IRC before I could thank you so thanks!
Sounds like minimum version of v2.11.0 should be safe here then, yeah?

@lrvick feel free to cherry-pick that commit.
I guess with that the minimum version becomes v2.1.0.
Which is the version where --stable was introduced. You could go further back, but you would need to drop --stable and its properties.

This PR was targeted against "mac-compatibility" instead of master. I merged to both branches to make this show as merged.


Multiple instances of "all X goes through this method"

This is a protodoc issue.
See writeLogEntries for an example.

Still appears to be an issue - reference link: https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/google-cloud-logging/src/main/java/com/google/cloud/logging/spi/v2/LoggingServiceV2Client.java#L292

Provisionally marking 'blocking GA'.

ProtoDoc needs an update to fix.


Conflicts with SN Utils when Next Experience is enabled

There seems to be a small conflict in Xplore 4.12 with the SN Utils browser extension:
After updating to 4.12, the SN Utils provided Application and Update set pickers open Xplore instead of the relevant picker list.
To reproduce:

In SN Utils settings, enable setting "Show (Next UI) or Enhance (UI16) Application and Updateset picker at page load."
Observe that now clicking on either the Application or Update set pickers that SN Utils provides, the Xplore tab is opened instead of the relevant picker list.

Tested on: Chrome 104.0.5112.102 (Windows 10), SN Utils <IP_ADDRESS>, Tokyo with Next Experience enabled

Thanks for pinging me on Slack Ingimar.
I'll add a fix in SN Utils for this.

Thanks guys!