Openssl 1.0 Deprecate Issue.

Hi,
It seems homebrew have remove openssl 1.0.0 and I found you library is using xmlsec which refer to openssl 1.0.0

File "/usr/local/lib/python3.7/site-packages/onelogin/saml2/auth.py", line 14, in
import xmlsec
File "/Applications/PyCharm.app/Contents/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
ImportError: dlopen(/usr/local/lib/python3.7/site-packages/xmlsec.cpython-37m-darwin.so, 2): Library not loaded: /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib
Referenced from: /usr/local/lib/python3.7/site-packages/xmlsec.cpython-37m-darwin.so
Reason: image not found

@DavidAtWorking A short term solution is to use the 1.0.2 branch.  Ex.  brew switch openssl 1.0.2t

@darakian @DavidAtWorking do you still experience this issue with the latest release of xmlsec? Trying to reproduce it on 10.15.4. This is what I have installed via Homebrew:
 % brew info openssl
<EMAIL_ADDRESS>stable 1.1.1g (bottled) [keg-only]
Cryptography and SSL/TLS Toolkit
https://openssl.org/
<EMAIL_ADDRESS>(8,059 files, 18MB)
  Poured from bottle on 2020-05-19 at 22:09:22
From<EMAIL_ADDRESS>...
 % brew info libxml2
libxml2: stable 2.9.10 (bottled), HEAD [keg-only]
GNOME XML library
http://xmlsoft.org/
/usr/local/Cellar/libxml2/2.9.10_1 (280 files, 10.5MB)
  Poured from bottle on 2020-05-22 at 00:20:24
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/libxml2.rb
...
 % brew info libxmlsec1
libxmlsec1: stable 1.2.30 (bottled)
XML security library
https://www.aleksey.com/xmlsec/
/usr/local/Cellar/libxmlsec1/1.2.30 (220 files, 5.6MB) *
  Poured from bottle on 2020-05-22 at 00:20:34
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/libxmlsec1.rb
...
 % brew info pkg-config
pkg-config: stable 0.29.2 (bottled)
Manage compile and link flags for libraries
https://freedesktop.org/wiki/Software/pkg-config/
/usr/local/Cellar/pkg-config/0.29.2_3 (11 files, 623.7KB) *
  Poured from bottle on 2020-05-22 at 00:20:38
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/pkg-config.rb

Installing xmlsec:
$ python3 -V
Python 3.6.8
$ python3 -m venv test
$ source test/bin/activate
(test) $ pip install xmlsec
...
Successfully installed lxml-4.5.1 xmlsec-1.3.8
(test) $ otool -L test/lib/python3.6/site-packages/xmlsec.cpython-36m-darwin.so
test/lib/python3.6/site-packages/xmlsec.cpython-36m-darwin.so:
	/usr/local/opt/libxmlsec1/lib/libxmlsec1-openssl.1.dylib (compatibility version 4.0.0, current version 4.30.0)
	/usr/local/opt/libxmlsec1/lib/libxmlsec1.1.dylib (compatibility version 4.0.0, current version 4.30.0)
<EMAIL_ADDRESS>(compatibility version 1.1.0, current version 1.1.0)
	/usr/lib/libxslt.1.dylib (compatibility version 3.0.0, current version 3.26.0)
	/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.0.0)
	/usr/lib/libicucore.A.dylib (compatibility version 1.0.0, current version 64.2.0)

@hoefling All is working well in my project
I've updated lxml to 4.5.1 which pulls xmlsec 1.3.8 and everything works with openssl 1.1.1g
Many thanks :)

The update in my poetry lock file looks like

There are more with the tar at the bottom. Not sure if that's on you or on lxml.

@darakian

All is working well in my project

glad to hear that!

all the wheels seem to be marked as windows wheel despite me not using windows OS.

This is as expected - poetry stores the hashes of all wheels as the lock file is not platform specific. The list of wheels is also correct - I haven't released any wheels except for windows yet. On Linux and MacOS, I'm not sure how to handle the OpenSSL dependency, I have described my reasoning here.

This is as expected - poetry stores the hashes of all wheels as the lock file is not platform specific.

Makes sense. Thank you again for the work :)


Update badge URL for 1.31 image-builder jobs

Issue #, if available:
#2399
Description of changes:
This PR updates the badge URL for 1.31 image-builder jobs and removes the 1.26 image-builder job badge URLs.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve

/approve

/lgtm


Unable to use hspec dependency

Just adding hspec to the cabal file is enough to trigger https://github.com/input-output-hk/haskell.nix/issues/1177
As can be seen in CI,
Configuring library for hspec-core-2.9.4..
Error:
    The following packages are broken because other packages they depend on are missing. These broken packages must be rebuilt before they can be used.
installed package ghc-9.2.1 is broken due to missing package exceptions-0.10.4

exceptions is already listed in nonReinstallablePkgs: https://github.com/Plutonomicon/plutarch/blob/19b5e9b90d9df9492e494be730b2169abcdca386/flake.nix#L177
.. yet it doesn't work, contradicting the suggestion from https://github.com/input-output-hk/haskell.nix/issues/1177

Umm, PEBKAC! I hadn't noticed that modules was defined inside tools.haskell-language-server.

--- a/flake.nix
+++ b/flake.nix
@@ -154,11 +154,8 @@

       ghcVersion = "ghc921";

-      tools.fourmolu = { };
-      tools.haskell-language-server = {
-        modules = [{
-          # https://github.com/input-output-hk/haskell.nix/issues/1177
-          nonReinstallablePkgs = [
+      # https://github.com/input-output-hk/haskell.nix/issues/1177
+      nonReinstallablePkgs = [
             "rts"
             "ghc-heap"
             "ghc-prim"
@@ -201,7 +198,12 @@
             "xhtml"
             "terminfo"
           ];
-        }];
+
+      tools.fourmolu = { };
+      tools.haskell-language-server = {
+        modules = [
+          { inherit nonReinstallablePkgs; }
+          ];
         compiler-nix-name = ghcVersion;
         # For some reason it doesn't use the latest version automatically.
         index-state =
@@ -222,6 +224,7 @@
       };

       haskellModule = system: {
+        inherit nonReinstallablePkgs;  # Needed only so we can use hspec; https://github.com/Plutonomicon/plutarch/issues/409
         packages = {
           basement.src = "${inputs.foundation}/basement";
           basement.components.library.postUnpack = "\n";


Sourcekit-lsp go-to-def not working for modules built from diff toolchain version

Swift version
swift 5
Platform
Linux 5.19
Editor
Visual studio with swift-tools-version 5.9
Does the issue reproduce with Swift 6?
I didn’t try
Description
Creating this issue as I was asked here
Hello, this is using VSCode + sourcekit-lsp
I have a swift library that is built using toolchain sdk version of 13.4.
In this code, I have import modules, import A and import B
The code builds, and I am also able to find the go-to-defs for items in A. However, I am unable to find anything for items in B when using go-to-defs. I've took a look at the LSP logging, and can verify that both output directory of the lib modules are included in the key.compilerargs with the -XCC flag
B uses sdk version of 15.2, A uses version 13.0
Is there something with the sdk versions that causes sourcekit-lsp to be able to not link the symbols correctly? Is this a known issue? Or am I totally on the wrong track here and the issue is elsewhere?
Thanks in advance.
Steps to Reproduce
Unfortunately I don't really have a simple way to reproduce this, as I can't share this project.
Logging
Unfortunately contains internal information, but if you'd like specific information I can do my best to send it

Synced to Apple’s issue tracker as rdar://133965202

Is this a SwiftPM project, a project with compile_commands.json or a build server?
Did you build your project when you’re seeing this issue? SourceKit bundled with Swift 5.9 doesn’t support background indexing and we need a build to update the index.

B uses sdk version of 15.2, A uses version 13.0

Are they just built with different SDKs or also different toolchains / Swift compilers?

Is this a SwiftPM project, a project with compile_commands.json or a build server?

build server

Did you build your project when you’re seeing this issue? SourceKit bundled with Swift 5.9 doesn’t support background indexing and we need a build to update the index.

Yes, I am able to build it. What's weird is that for the module where the go-to-defs don't work, clicking on the import B go-to-def will take me to a .swiftinterface in /tmp/sourcekit-lsp/GeneratedInterfaces/. However, clicking on the actual symbols from B will lead to no-definition-found

Are they just built with different SDKs or also different toolchains / Swift compilers?

Different SDK Versions as far as I am aware. They all use the same build server.
Thanks for looking into

I assume that we are missing index data, which causes cross-module jump-to-definition not to work. Jumping to definition on the import itself uses a slightly different code path because we don’t need to figure out in which module a type was defined.
Could you ensure that you:

Set -index-store-path in the compiler invocations that build your project
Make sure that we have the index store and index database path by either

Return the path to the index store as indexStorePath from your build server and returning indexDatabasePath to any directory in which SourceKit-LSP can build indexstore-db (https://github.com/swiftlang/sourcekit-lsp/blob/a3bb2d77d163a6c3abd647d9d26bcf0c5ba2f975/Sources/BuildSystemIntegration/BuildServerBuildSystem.swift#L175-L181)
Alternatively, you can set these options as -index-store-path and -index-db-path when launching sourcekit-lsp.

Hey thanks for your response

Set -index-store-path in the compiler invocations that build your project

I'm not sure if I understand this correctly. Should I be overriding it to my current projects path?

It’s hard to tell without knowing what your build server does. Do you have an example of build settings that are provided by your build server?

For the build setting, are you referring to the project, or the sourcekit-lsp?
Just as a general info:
So I dug a little bit deeper into the build etc
We use the buck build system
The sourcekit-lsp is built via MAKE that has a specific .mk file to specify configuration, looks like its built using swift bunch of commands and configurations
It does not have an index-store-path in the set of configurations from what I can tell, but I assume it just uses some sort of default set path if thats the case.
The other modules (aka the imports / project) are built using buck
The reason why I was under the impression that the index store is included is because I have
the following from the sourcekit logging when opening the problem project file
sourcekit: [2:handleRequest-before:49.4170] {
  key.request: source.request.editor.open,
  key.name: "/some/directory/problemProject.swift",
  key.enablesyntaxmap: 0,
  key.enablesubstructure: 0,
  key.enablediagnostics: 0,
  key.syntactic_only: 1,
  key.compilerargs: [
  ...

`"-Xcc", "buck-out/.swift-vscode/blah/blah-blah/B/swift-extended_symlink_tree"`,
`"-Xcc", "buck-out/.swift-vscode/blah/blah-blah/A/exported_symlink_tree"`,
...`

With that being said, I examined both directories and they both had their respective `A-Swift.h` and `B-Swift.h` files in those places

For the build setting, are you referring to the project, or the sourcekit-lsp?

I am referring to your project’s build settings.

It does not have an index-store-path in the set of configurations from what I can tell, but I assume it just uses some sort of default set path if thats the case.

You need to pass a -index-store-path so that we can build an index of your project during a build, which can then be used by SourceKit-LSP to provide cross-module functionality. Opening the file in sourcekitd doesn’t use the index and provides mostly functionality that’s contained within a single module. sourcekitd needs to co-operate with the index to provide cross-module functionality.

I will look into being able to do that and report back when I can.
That does bring me another question however.
Clicking on a symbol from B will still take me to the generated interface file of .swiftinterface (Even without generating it first via clicking on import B go to def)
Are there any guesses as to why symbols from B will go to generated-interface, but say no definition found for A symbols, even if go-to-def for that import will take me to a .swiftinterface file as well? I'm trying to follow through in the sourcekit-lsp code but can't seem to understand why such case might happen

I haven’t fully understood that part yet. It is a little hard to reason about without a reproducing example project. But I’m pretty sure that the lack of index data is the root cause here.

Looks like buck model and index store is very incompatible and they are currently working on coming up with a solution for it, so I guess at the time being the cross-module jump functionality is kind of thrown out the window. I do apologize that I'm unable to just provide everything (e.g. reproducible example project) so that you can also easily see whats going on.
I'd still like to discover why the go-to-def works for some symbols but not others to at least direct uesrs to .swiftinterface files. I see https://github.com/swiftlang/sourcekit-lsp/pull/747 seems to have added the functionality for it, but I guess it must be failing for some while succeeding for others. I guess I'll have to keep digging, but yeah, my initial thought was something with the toolchain version since they are both in the   key.compilerargs: ....

To figure out why it works in one module but not the other, I would use SourceKit-LSP from a Swift 6.0 toolchain (because it has way more logging than 5.10), enable extended logging and check if there’s anything interesting in the log, as shown here.
If that doesn’t show anything, I would try building a debug version of SourceKit-LSP and debugging it, especially the SourceKitLSPServer.definition function.


[Enhancement] Add module status to each of Egeria's services' component description

Is there an existing issue for this?

[X] I have searched the existing issues

Please describe the new behavior that that will improve Egeria
Each module in Egeria has a component description that is registered with the audit log so it can be associated with each log record that it produces.  This description includes:

Component Id - integer
Component Name
Component Description
Component's wiki page URL

Lists of component descriptions are maintained as Enums in the egeria code - for example:

Access Services
OMRS Components and connector types

In addition to registering with the audit log, these component descriptions are used to support platform services that list registered services.  They also appear in the audit log report showing the hierarchy of components running in a server.
This enhancement proposes to add a status code to this description that has one of 4 values:

In development
Tech Preview
Supported
Deprecated

This then becomes our official status for each component.  It would provide a way to create reports of the status of all of our components for TSC review - or if an organization is going into production and they want to understand the risks in their deployment.
We would still publish the status of a component in the webpage.  IE https://odpi.github.io/egeria-docs/services/omrs/
Alternatives
Today it is necessary to follow the web URL link to the component's page to read its status from the top of the page.  This makes it hard to automate the generation of reports showing the status of components in a particular deployment environment.
Any Further Information?
No response
Would you be prepared to be assigned this issue to work on?

[X] I can work on this

I am thinking that a 5th status of "sample" would be a good addition.  This would distinguish between the supported runtime and a sample.

Discussion on the community call suggested using "Stable" instead of Released/Supported


[Bug Report] el-tree after drag/drop nodesMap is degraded!

Element UI version
2.3.9
OS/Browsers version
win10 Chrome Version 66.0.3359.139 (Official Build) (64-bit)
Vue version
2.5.16
Reproduction Link
https://jsfiddle.net/abernov/g90mjn9f/
Steps to reproduce

open jsfiddle.net link
click button "show node label" see alert "Level two 2-1"
drag-drop node "Level two 2-1"
click button "show node label" see alert "The BUG"

What is Expected?
see alert "Level two 2-1"
The object object storage/nodesMap defined in the file tree-store.js will not change
What is actually happening?
After each drag-drop nodes disappear from the root of the object nodesMap defined in the file
tree-store.js
Which leads to the inability of most functions .

Can't reproduce.

open jsfiddle.net link click "Run"
click button "show node label" see alert "Level two 2-1"
drag node "Level two 2-1" then drop it to node "Level two 1-1"
click button "show node label" see alert "The BUG"


Refactor allowlist reading/sending to enable verifier-side signature checks

This PR refactors how allowlists are read and processed by the tenant and verifier, enabling verifier-side signature checks.
Now, instead of all allowlist processing being done tenant-side, the tenant sends the raw allowlist artifact along with any provided validation artifacts (checksum, signature, signing key) as a bundle to the verifier. (Note that the existing tenant-side signature check is preserved.)
The verifier will then unbundle everything and perform its own signature validation step before performing the final processing step and storing the formatted allowlist in the database as before.
This enhances security by thwarting potential man-in-the-middle attacks - prior to this, Keylime's security model relied on a trusted tenant; however, if a malicious tenant sent a bad payload (ex. one that did not pass signature checks), the verifier would blindly accept it. The new verifier-side signature check makes this attack impossible.
Verifier-side signature validation can be controlled via a new configuration option in keylime.conf. By default, signatures are not enforced, but setting enforce_allow_list_signatures to True will require that check.
Signed-off-by: Mark Bestavros<EMAIL_ADDRESS>

Test failures suggests that it is no longer possible to register an agent using previously added named allowlist.
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Add keylime agent using the named allowlist
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 18:20:56 ] :: [  BEGIN   ] :: Running 'lime_keylime_tenant -t <IP_ADDRESS> -u d432fbb3-d2f1-4a97-9ef7-75bd81c00000 --allowlist-name list2 --exclude excludelist.txt -f allowlist.txt -c add'
Reading configuration from ['/etc/keylime.conf']
2022-05-18 18:20:57.389 - keylime.tpm - INFO - TPM2-TOOLS Version: 5.2
2022-05-18 18:20:57.392 - keylime.tenant - INFO - Setting up client TLS in /var/lib/keylime/cv_ca
2022-05-18 18:20:57.392 - keylime.registrar_client - WARNING - TLS is enabled.
2022-05-18 18:20:57.392 - keylime.registrar_client - INFO - Setting up client TLS...
2022-05-18 18:20:57.425 - keylime.tenant - INFO - TPM PCR Mask from policy is 0x408000
<Response [200]>
2022-05-18 18:20:57.547 - keylime.tpm - WARNING - PCR #0 in quote not found in tpm_policy, skipping.
2022-05-18 18:20:57.548 - keylime.tenant - WARNING - DANGER: EK cert checking is disabled and no additional checks on EKs have been specified with ek_check_script option. Keylime is not secure!!
2022-05-18 18:20:57.548 - keylime.tenant - INFO - Quote from <IP_ADDRESS> validated
2022-05-18 18:20:57.585 - keylime.tenant - ERROR - Response code 400: Allowlist data not provided with request!
2022-05-18 18:20:57.585 - keylime.tenant - ERROR - POST command response: 400 Unexpected response from Cloud Verifier: {"code": 400, "status": "Allowlist data not provided with request!", "results": {}}
:: [ 18:20:57 ] :: [   PASS   ] :: Command 'lime_keylime_tenant -t <IP_ADDRESS> -u d432fbb3-d2f1-4a97-9ef7-75bd81c00000 --allowlist-name list2 --exclude excludelist.txt -f allowlist.txt -c add' (Expected 0, got 0)

Also, keylime_tenant probably should not exit with 0 in such a situation but that is a different issue I guess.

@mbestavros thank you for this PR. I'll try to finish https://github.com/keylime/enhancements/pull/71, so that everything IMA related is actually part of that bundle.
For the bundle format I would add also a type and an alg field as described in https://github.com/keylime/enhancements/pull/65#issuecomment-1085843137 and rename allowlist to `ima_policy'. This makes it more flexible for us in the future.
Having the key part of the bundle has some consequences (more below). Do you think we need some kind of trusted keyring on the verifier?

This enhances security by thwarting potential man-in-the-middle attacks - prior to this, Keylime's security model relied on a trusted tenant; however, if a malicious tenant sent a bad payload (ex. one that did not pass signature checks), the verifier would blindly accept it. The new verifier-side signature check makes this attack impossible.

Not really. In the current design the key is still in the bundle. An attacker can create their own malicious bundle and provide a their own valid key with it. I my opinion if an attacker has API access to the verifier it is mostly game over.
What is solves is that we can provide a known good IMA policy for e.g. Fedora 35 which can be downloaded from the internet signed with a key from Fedora and Keylime validates if that key is the same as we expect. Moving it into the verifier makes it usable also for people that don't use the tenant (like me).

Test failures suggests that it is no longer possible to register an agent using previously added named allowlist.
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::   Add keylime agent using the named allowlist
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [ 18:20:56 ] :: [  BEGIN   ] :: Running 'lime_keylime_tenant -t <IP_ADDRESS> -u d432fbb3-d2f1-4a97-9ef7-75bd81c00000 --allowlist-name list2 --exclude excludelist.txt -f allowlist.txt -c add'
Reading configuration from ['/etc/keylime.conf']
2022-05-18 18:20:57.389 - keylime.tpm - INFO - TPM2-TOOLS Version: 5.2
2022-05-18 18:20:57.392 - keylime.tenant - INFO - Setting up client TLS in /var/lib/keylime/cv_ca
2022-05-18 18:20:57.392 - keylime.registrar_client - WARNING - TLS is enabled.
2022-05-18 18:20:57.392 - keylime.registrar_client - INFO - Setting up client TLS...
2022-05-18 18:20:57.425 - keylime.tenant - INFO - TPM PCR Mask from policy is 0x408000
<Response [200]>
2022-05-18 18:20:57.547 - keylime.tpm - WARNING - PCR #0 in quote not found in tpm_policy, skipping.
2022-05-18 18:20:57.548 - keylime.tenant - WARNING - DANGER: EK cert checking is disabled and no additional checks on EKs have been specified with ek_check_script option. Keylime is not secure!!
2022-05-18 18:20:57.548 - keylime.tenant - INFO - Quote from <IP_ADDRESS> validated
2022-05-18 18:20:57.585 - keylime.tenant - ERROR - Response code 400: Allowlist data not provided with request!
2022-05-18 18:20:57.585 - keylime.tenant - ERROR - POST command response: 400 Unexpected response from Cloud Verifier: {"code": 400, "status": "Allowlist data not provided with request!", "results": {}}
:: [ 18:20:57 ] :: [   PASS   ] :: Command 'lime_keylime_tenant -t <IP_ADDRESS> -u d432fbb3-d2f1-4a97-9ef7-75bd81c00000 --allowlist-name list2 --exclude excludelist.txt -f allowlist.txt -c add' (Expected 0, got 0)

Also, keylime_tenant probably should not exit with 0 in such a situation but that is a different issue I guess.

Thanks for pointing the broken test out @kkaarreell!
I've got some interesting findings after digging into the tenant/verifier code - while it is true that my changes broke that test, I don't know if the test itself is valid at present given the existing codebase, because the allowlist CRD API (which is exposed through the tenant as the addallowlist, showallowlist, and deleteallowlist commands) isn't yet fully implemented on the verifier side. Specifically, this means that even though the endpoints (post, get, delete) exist and work from the tenant's perspective, they don't actually store the allowlist itself - only the tpm_policy that the tenant sends alongside the allowlist. (side note, this is why the other tests for showallowlist and deleteallowlist work - they are getting and deleting something from the verifier's database, it's just not the full allowlist.)
In short, adding an agent using named allowlists isn't actually supported at present to begin with (because named allowlists themselves aren't supported yet). So why did the test break - perhaps more importantly, why did it pass in the first place if it's not actually working?
It turns out that, at present, the tenant will just throw out the allowlist_name argument when executing the add command for initializing an agent and send an empty allowlist to the verifier. Prior to this PR, allowlists were processed entirely on tenant, so the verifier would happily take the empty allowlist and everything else would work - even though the allowlist it was using didn't actually have anything in it. That's changed with my PR - since allowlist processing now happens on the verifier, I made it so that it would fail if no allowlist was provided. This is where I could use the input of other Keylime contributors (@THS-on @mpeters) - do we want the verifier to return an error if no allowlist data is provided, or should we assume an empty allowlist, as before?
(Side note, my next task after this PR is completing the implementation of the allowlist CRD API - so hopefully this test isn't invalid for much longer 😄)

MITM is still very much a concern, and this doesn't fix that.

Maybe I'm missing something, but I disagree with this. The connection is protected using a mTLS connection. You can only MITM a connection between the verifier if you already have a key and certificate trusted by the verifier. If you have that then you can do what ever you want at the verifier, so there is no reason for MITM a connection.

I also want to clarify that I didn't intend the addition of "bundles" in this PR to be a part of the new IMA policy changes, more just a way to send additional artifacts alongside the allowlist in its current format (specifically, associated signing materials). My understanding was the IMA policy evolution would be a larger effort around the allowlist format itself, which would then render this obsolete - but in the meantime, this is a good stopgap measure.

Ok, I then propose the following steps:

Get this merged with the temporary bundle format
Implement the new IMA policy format with the temporary bundle format
Move everything to a well specified bundle format

Maybe I'm missing something, but I disagree with MITM is a concern. The connection is protected using a mTLS connection. You can only MITM a connection between the verifier if you already have a key and certificate trusted by the verifier. If you have that then you can do what ever you want at the verifier, so there is no reason for MITM a connection.

@THS-on Right, apologies my wording was not precise. I meant that this change wouldn't impact MITM security, for better or worse. It's a security improvement on the tenant side, but doesn't impact transit. My mistake 😅

Ok, I then propose the following steps:

Get this merged with the temporary bundle format
Implement the new IMA policy format with the temporary bundle format
Move everything to a well specified bundle format

Sounds like a plan to me!

Some updates: I've made various fixes based on testing and adjusted test_restful.py with better test allowlist data, and tidied up lints and formatting. All tests pass now.
@mpeters @THS-on any additional feedback at this stage, or does this look good to go?

@THS-on @mpeters You've both raised a good point about API levels and backwards compat. I'm happy to write some code to maintain backwards compat, but if we do I'd like a sanity check to make sure the bundle format I'm using makes sense for the foreseeable future (perhaps including the IMA policy format overhaul).
Keylime at present
payload sent to the verifier:
"allowlist": <processed allowlist>

stored in the verifier's database:
"allowlist": <processed allowlist>

After this PR
payload sent to the verifier: (note that the exclusion list needs to be sent separately because they aren't combined until allowlist processing, which now happens on the verifier after transit)
"ima_policy": {
    "ima_policy": <raw allowlist>,
    "key": <pubkey>,
    "sig": <signature>,
    "checksum": <checksum>,
    "excllist": <exclusion list>
}

stored in the verifier's database:
"allowlist": <processed allowlist>

The future
payload sent to the verifier:
"ima_policy": {
    "ima_policy": <raw IMA policy in new format?>,
    "key": <pubkey>,
    "sig": <signature>,
    "checksum": <checksum>
}

stored in the verifier's database:
"ima_policy": {
    "ima_policy": <raw IMA policy in new format?>,
    "key": <pubkey>,
    "sig": <signature>,
    "checksum": <checksum>
}

Does this make sense?

@THS-on @mpeters You've both raised a good point about API levels and backwards compat. I'm happy to write some code to maintain backwards compat, but if we do I'd like a sanity check to make sure the bundle format I'm using makes sense for the foreseeable future (perhaps including the IMA policy format overhaul).

I'm not sure we can make guarantees that the format you're using for IMA policy will last for the "foreseeable future". These things evolve as people use them and that other enhancement still needs to be implemented. But that shouldn't really matter for this PR.

After this PR
payload sent to the verifier: (note that the exclusion list needs to be sent separately because they aren't combined until allowlist processing, which now happens on the verifier after transit)

Why is that? The old way combined the exclude into the allowlist and called them both the allowlist. Are you not bundling the exclude list in with the ima_policy bundle?

"ima_policy": {
    "ima_policy": <raw allowlist>,
    "key": <pubkey>,
    "sig": <signature>,
    "checksum": <checksum>,
    "excllist": <exclusion list>
}

stored in the verifier's database:
"allowlist": <processed allowlist>

The future
payload sent to the verifier:
"ima_policy": {
    "ima_policy": <raw IMA policy in new format?>,
    "key": <pubkey>,
    "sig": <signature>,
    "checksum": <checksum>
}

stored in the verifier's database:
"ima_policy": {
    "ima_policy": <raw IMA policy in new format?>,
    "key": <pubkey>,
    "sig": <signature>,
    "checksum": <checksum>
}

This would include the exclude list in both of these correct?

@THS-on Awesome, thanks for following up. I'll be pushing up another draft of the PR relatively soon with all the changes discussed here.
@mpeters to be clear, I'm not proposing this bundle structure as a first draft of the new IMA policy format - as you say, that's happening through the enhancements process.
I'm just proposing a format to carry the raw, user-provided artifact (currently the raw allowlist file plus exclusion list as separate entries, but in the future it would be the combined, new-format IMA policy as a single entry) from the tenant over to the verifier. I'm wondering if the format I've proposed makes sense for that specific purpose, and won't need to be changed once the IMA policy evolution happens.

@THS-on @mpeters I've pushed an update that should hopefully address most of the feedback we've discussed.
Most notably, I've tweaked a few things about the bundle transmission - the bundle is now sent under the ima_policy_bundle key in the payload, instead of just ima_policy before. This leaves the ima_policy key available in the future for when the IMA policy format evolution is complete and (potentially) the temporary bundle format would no longer be needed.
I've also tweaked how the CV handles allowlists. Instead of requiring the new bundle format, the verifier will maintain backwards-compatible behavior as before if an allowlist is sent using the legacy allowlist key in the POST payload.
I've also made some miscellaneous small fixes and naming updates based on your comments. Would appreciate another look whenever you both have the chance - thank you!

@mpeters @THS-on My latest push includes some minor fixes for naming as pointed out by Michael. Also, the tenant will no longer attempt to verify signing materials for allowlists provided as a list instead of a file (Michael pointed out this could be unreliable).
@THS-on have you had a chance to test this?

Codecov Report

Merging #994 (62062fc) into master (304865f) will increase coverage by 2.17%.
The diff coverage is 72.97%.

Flag
Coverage Δ

packit-e2e
68.20% <72.97%> (+2.11%)
:arrow_up:

testsuite
50.00% <40.54%> (+1.97%)
:arrow_up:

unittests
13.67% <10.81%> (+0.43%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

keylime/ima/ima.py
71.06% <69.81%> (+0.60%)
:arrow_up:

keylime/cloud_verifier_tornado.py
68.01% <73.33%> (+0.06%)
:arrow_up:

keylime/tenant.py
69.41% <100.00%> (+0.27%)
:arrow_up:

keylime/signing.py
65.30% <0.00%> (-24.49%)
:arrow_down:

keylime/tornado_requests.py
68.42% <0.00%> (-5.27%)
:arrow_down:

keylime/cloud_verifier_common.py
81.20% <0.00%> (-0.68%)
:arrow_down:

keylime/cmd/webapp.py

keylime/tenant_webapp.py


NSFS | S3 | Versioning: AccessDenied on GetObject in a version-enabled bucket despite bucket policy allowing

Environment info

NooBaa Version: 5.15.4
Platform: CentOS 8, deployed using https://github.com/IBM/StorageScaleVagrant

Actual behavior
GetObject returns AccessDenied although bucket policy allows access.
        policy_document = json.dumps(
        {
            "Version": "2012-10-17",
            "Statement": [{
                    "Effect": "Allow",
                    "Principal": { "AWS": "{}".format(get_alt_user_id()) },
                    "Action": [ "s3:GetObject", "s3:GetObjectVersion", "s3:ListBucketVersions" ],
                    "Resource": "arn:aws:s3:::{}/{}".format(bucket_name, key)
                }
            ]
        })

        client.put_bucket_policy(Bucket=bucket_name, Policy=policy_document)

>       response = alt_client.get_object(Bucket=bucket_name, Key=key)

s3tests_boto3/functional/test_s3.py:1759:
...
E           botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

Expected behavior
GetObject should pass as policy has granted access to alternate user - here: CES S3 user with same gid as primary user, but different uid.
Steps to reproduce
Run test_s3cever11 with Ceph s3-tests suite. Two users with the same gid are configured:
def test_s3cever11():
    client = get_client()
    bucket_name = get_new_bucket(client=client)

    # The following operation uses another client instance -> following operations will fail
#    check_configure_versioning_retry(bucket_name, "Enabled", "Enabled")
    _ces_safe_check_configure_versioning_retry(client, bucket_name, "Enabled", "Enabled")

    key = "test"
    version_ids = []
    contents = []
    num_versions = 3
    (version_ids, contents) = create_multiple_versions(client, bucket_name, key, num_versions)

    response = client.list_object_versions(Bucket=bucket_name)
    assert len(response['Versions']) == 3

    resource1 = "arn:aws:s3:::" + bucket_name
    resource2 = "arn:aws:s3:::" + bucket_name + "/*"
    policy_document = json.dumps(
    {
        "Version": "2012-10-17",
        "Statement": [{
        "Effect": "Allow",
        "Principal": {"AWS": "*"},
        "Action": "s3:ListBucketVersions",
        "Resource": [
            "{}".format(resource1),
            "{}".format(resource2)
          ]
        }]
     })

    client.put_bucket_policy(Bucket=bucket_name, Policy=policy_document)

    alt_client = get_alt_client()
    response = alt_client.list_object_versions(Bucket=bucket_name)
    assert response['ResponseMetadata']['HTTPStatusCode'] == 200

    response_code = ""
    try:
        response = alt_client.get_object(Bucket=bucket_name, Key=key, VersionId=version_ids[0])
    except ClientError as e:
        response_code = e.response['Error']['Code']

    assert response_code == "AccessDenied"

    # s3:versionid condition key is not supported by Noobaa, so skipping that policy
    policy_document = json.dumps(
    {
        "Version": "2012-10-17",
        "Statement": [{
                "Effect": "Allow",
                "Principal": { "AWS": "{}".format(get_alt_user_id()) },
                "Action": [ "s3:GetObject", "s3:GetObjectVersion" ],
                "Resource": "arn:aws:s3:::{}/{}".format(bucket_name, key)
            }
        ]
    })

    print(policy_document)
    client.put_bucket_policy(Bucket=bucket_name, Policy=policy_document)

    response = alt_client.get_object(Bucket=bucket_name, Key=key)
    body = _get_body(response)
    assert body == contents[2]

    response = alt_client.get_object(Bucket=bucket_name, Key=key, VersionId=version_ids[0])
    body = _get_body(response)
    assert body == contents[0]
    response = alt_client.get_object(Bucket=bucket_name, Key=key, VersionId=version_ids[1])
    body = _get_body(response)
    assert body == contents[1]

More information - Screenshots / Logs / Other output
Interestingly when I execute a similar CLI sequence, I can't reproduce the error symptom:
s3 mb s3://shared2
s3api put-bucket-versioning --bucket shared2 --versioning-configuration '{"Status": "Enabled"}'
s3 cp ../custom-tests/foobar.txt s3://shared2/test
s3 cp ../custom-tests/foobarbaz.txt s3://shared2/test
# following fails as expected
s3u2 cp s3://shared2/test test
s3api put-bucket-policy --bucket shared2 --policy '{"Version": "2012-10-17","Statement": [{"Effect": "Allow","Principal": {"AWS": "s3user2"},"Action":["s3:GetObject", "s3:GetObjectVersion" ],"Resource": "arn:aws:s3:::shared2/test"}]}'
# following succeeds as expected
s3u2 cp s3://shared2/test test
s3u2api get-object --bucket shared2 --key test --version-id mtime-d47smoigl4ao-ino-kqo x

Log with "all" log level attached.
noobaa_20240916_1.log.gz

@hseipp  In the logs, I can see that:

Sep 16 14:19:13 m1.example.com node[26226]: Sep-16 14:19:13.556 [nsfs/26301]    [L1] core.endpoint.s3.s3_bucket_policy_utils:: bucket_policy:  Action  fit? s3:ListBucketVersions s3:GetObjectVersion
Sep 16 14:19:13 m1.example.com node[26226]: Sep-16 14:19:13.556 [nsfs/26301]    [L1] core.endpoint.s3.s3_bucket_policy_utils:: bucket_policy:  Principal  fit? SENSITIVE-7846cdd4c2b90527 s3user2
Sep 16 14:19:13 m1.example.com node[26226]: Sep-16 14:19:13.556 [nsfs/26301]    [L1] core.endpoint.s3.s3_bucket_policy_utils:: bucket_policy:  Resource  fit? /^arn:aws:s3:::s3tests-cwd1j5qytc5i90iqt1wf3-1$/ arn:aws:s3:::s3tests-cwd1j5qytc5i90iqt1wf3-1/test
Sep 16 14:19:13 m1.example.com node[26226]: Sep-16 14:19:13.556 [nsfs/26301]    [L1] core.endpoint.s3.s3_bucket_policy_utils:: bucket_policy:  Resource  fit? /^arn:aws:s3:::s3tests-cwd1j5qytc5i90iqt1wf3-1/.*$/ arn:aws:s3:::s3tests-cwd1j5qytc5i90iqt1wf3-1/test
Sep 16 14:19:13 m1.example.com node[26226]: Sep-16 14:19:13.556 [nsfs/26301]    [L1] core.endpoint.s3.s3_bucket_policy_utils:: bucket_policy: is_statements_fit false true true true
Sep 16 14:19:13 m1.example.com node[26226]: Sep-16 14:19:13.556 [nsfs/26301] [ERROR] core.endpoint.s3.s3_rest:: S3 ERROR AccessDeniedAccess Denied/s3tests-cwd1j5qytc5i90iqt1wf3-1/test?versionId=mtime-d47rvncf29kw-ino-d8gm153cob7-cwgief-18a4 GET /s3tests-cwd1j5qytc5i90iqt1wf3-1/test?versionId=mtime-d47rvncf29kw-ino-d8g {"host":"localhost:6443","accept-encoding":"identity","user-agent":"Boto3/1.35.4 md/Botocore#1.35.4 ua/2.0 os/linux#4.18.0-348.7.1.el8_5.x86_64 md/arch#x86_64 lang/python#3.8.8 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.35.4","x-amz-date":"20240916T141913Z","x-amz-content-sha256":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","authorization":"AWS4-HMAC-SHA256 Credential=84zUfVLtmuPXVgqLt2qE/20240916/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=57d241b9e60c4990c9f62ea68ed76c89c9df099bc1267c9beb24c21bb9fd8d26","amz-sdk-invocation-id":"97c400f0-9e91-48d4-8556-3f718e0813e9","amz-sdk-request":"attempt=1"} Error: Access Denied

We iterate on all the actions and print if they fit, in this log, I can see only one check - ListBucketVersions and I would expect to see also s3:GetObject and 3:GetObjectVersion in the printing:
https://github.com/noobaa/noobaa-core/blob/b28bd87e37e825456e4f7b6c79366a2fb123b3bd/src/endpoint/s3/s3_bucket_policy_utils.js#L152-L162
Could you please add to your script (that appears under the "Actual behavior" title) printing of the bucket policy as it was saved (print the get_bucket_policy)? and also the printing of "AWS": "{}".format(get_alt_user_id()) just to be sure the alternative account is set as needed in the configuration and attach the collected logs.

Please note that in the meantime my test system got upgraded to noobaa-core 5.17.0-20240904.
I can still re-produce the issue.

E           botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

plus the following output for the print statements:
{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"AWS": "s3user2"}, "Action": ["s3:GetObject", "s3:GetObjectVersion"], "Resource": "arn:aws:s3:::s3tests-o2061vmb4d278hurumavk-1/test"}]}
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["s3user2"]},"Action":["s3:GetObject","s3:GetObjectVersion"],"Resource":"arn:aws:s3:::s3tests-o2061vmb4d278hurumavk-1/test"}]}

Relevant sections of my s3tests.conf:
[s3 main]
# main display_name set in vstart.sh
display_name = s3user

# main user_idname set in vstart.sh
user_id = s3user1
...
[s3 alt]
# alt display_name set in vstart.sh
display_name = s3user2
## alt email set in vstart.sh
email =<EMAIL_ADDRESS>
# alt user_id set in vstart.sh
user_id = s3user2
...

Noobaa config:

$ sudo cat /ibm/cesShared/ces/s3-config/accounts/s3user1.json | jq '.name, .email'
"s3user1"
"s3user1"
$ sudo cat /ibm/cesShared/ces/s3-config/accounts/s3user2.json | jq '.name, .email'
"s3user2"
"s3user2"

The corresponding "all" debug level log is attached
[noobaa_20240918_1.log.gz](https://github.com/user-attachments/files/17040550/noobaa_20240918_1.log.gz)

I'm closing the issue, if needed @hseipp feel free to re-open.
As mentioned, I opened another issue for the bucket cache #8391.
cc: @nadavMiz, @romayalon

I will open a different issue for the bucket_namespace_cache updates between forks. @hseipp would you please validate that with the time.sleep(65) it works for you?
cc: @nadavMiz @romayalon

I can confirm that with the time.sleep(65) delay the test passes on my test environment.

Hi @hseipp,
I want to let you know that we merged a code change in PR #8527.
Could you please test it again on the master branch? (on RPM that includes the code changes)
In case it didn't solve the issue, please attach all the details:

RPM version and date
Partial code snippet (to see that it is with the code change)
The steps that you did
Logs with high debug level

I'm adding a label for validation, please update in a comment the status.


Nlog.Windows.Forms License

Hello. I want to ask why NLog and Nlog.Windows.Forms have different licenses?
NLog: https://raw.githubusercontent.com/NLog/NLog/master/LICENSE.txt
NLog.Windows.Forms: GNU GENERAL PUBLIC LICENSE: https://raw.githubusercontent.com/NLog/NLog.Windows.Forms/master/LICENSE
Perhaps this is a mistake?

Yes it is. Gpl is evil IMO
See https://github.com/NLog/NLog.Windows.Forms/issues/2

Think this has been resolved with https://github.com/NLog/NLog.Windows.Forms/pull/26

:) thx!


PSR-0 autoloading issue

See #13
I am opening a new issue as the referenced issue was closed as completed incorrectly.
Per the request in that issue here is the error output from composer:

Class DCarbone\PHPFHIRGenerated\DSTU2\FHIRElement\FHIRMeasmnt_Principle located in C:/sites/jte/vendor/dcarbone/php-fhir-generated/src\DCarbone\PHPFHIRGenerated\DSTU2\FHIRElement\FHIRMeasmnt_Principle.php does not comply with psr-0 autoloading standard. Skipping.
Class DCarbone\PHPFHIRGenerated\DSTU2\FHIRStringPrimitive\FHIRMeasmnt_PrincipleList located in C:/sites/jte/vendor/dcarbone/php-fhir-generated/src\DCarbone\PHPFHIRGenerated\DSTU2\FHIRStringPrimitive\FHIRMeasmnt_PrincipleList.php does not comply with psr-0 autoloading standard. Skipping.
Class DCarbone\PHPFHIRGenerated\DSTU2\PHPFHIRTests\FHIRElement\FHIRMeasmnt_PrincipleTest located in C:/sites/jte/vendor/dcarbone/php-fhir-generated/src\DCarbone\PHPFHIRGenerated\DSTU2\PHPFHIRTests\FHIRElement\FHIRMeasmnt_PrincipleTest.php does not comply with psr-0 autoloading standard. Skipping.
Class DCarbone\PHPFHIRGenerated\DSTU2\PHPFHIRTests\FHIRStringPrimitive\FHIRMeasmnt_PrincipleListTest located in C:/sites/jte/vendor/dcarbone/php-fhir-generated/src\DCarbone\PHPFHIRGenerated\DSTU2\PHPFHIRTests\FHIRStringPrimitive\FHIRMeasmnt_PrincipleListTest.php does not comply with psr-0 autoloading standard. Skipping.

Quoted from the link I provided in #13 about the issue with the underscores in the name:

Underscores in Namespaces and Class Names
\namespace\package\Class_Name => /path/to/project/lib/vendor/namespace/package/Class/Name.php
\namespace\package_name\Class_Name => /path/to/project/lib/vendor/namespace/package_name/Class/Name.php
The standards we set here should be the lowest common denominator for painless autoloader interoperability. You can test that you are following these standards by utilizing this sample SplClassLoader implementation which is able to load PHP 5.3 classes.

This repository does not implement PSR-0, it implements PSR-4: https://www.php-fig.org/psr/psr-4/

I have updated this repo to use psr-4.


Add multi-instances support

Please consider this MR as I want to monitor multiple logstash instances, which is a common case for a large ELK cluster.

Sorry, but it is wrong way. If you want to collect metrics of several instances of logstash, you should run dedicated instance of exporter for every instance of logstash, scrape all of them and aggregate metrics on the prometheus side.


[ISSUE #1979] Refactor integration test for rate-limiter plugin

Ref #1979 .
Refactor integration test for rate-limiter plugin

Codecov Report

Merging #2046 (7f2f392) into master (1fb1d28) will decrease coverage by 0.36%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #2046      +/-   ##
============================================
- Coverage     60.78%   60.41%   -0.37%
+ Complexity     4025     4011      -14
============================================
  Files           602      604       +2
  Lines         17715    17753      +38
  Branches       1687     1692       +5
============================================
- Hits          10768    10726      -42
- Misses         6073     6156      +83
+ Partials        874      871       -3

Impacted Files
Coverage Δ

...ache/shenyu/plugin/grpc/cache/GrpcClientCache.java
70.58% <0.00%> (-23.53%)
:arrow_down:

...enyu/client/tars/TarsServiceBeanPostProcessor.java
26.47% <0.00%> (-17.65%)
:arrow_down:

.../apache/shenyu/web/handler/GlobalErrorHandler.java
85.71% <0.00%> (-14.29%)
:arrow_down:

...a/org/apache/shenyu/common/utils/VersionUtils.java
67.85% <0.00%> (-10.72%)
:arrow_down:

...che/shenyu/sync/data/http/HttpSyncDataService.java
71.84% <0.00%> (-9.71%)
:arrow_down:

...he/shenyu/plugin/grpc/client/ShenyuGrpcClient.java
8.69% <0.00%> (-4.35%)
:arrow_down:

...henyu/plugin/grpc/resolver/ShenyuNameResolver.java
60.00% <0.00%> (-4.22%)
:arrow_down:

...henyu/admin/service/impl/UpstreamCheckService.java
22.76% <0.00%> (-4.07%)
:arrow_down:

...u/plugin/modify/response/ModifyResponsePlugin.java
14.70% <0.00%> (-3.61%)
:arrow_down:

...henyu/plugin/httpclient/NettyHttpClientPlugin.java
58.00% <0.00%> (-2.00%)
:arrow_down:

... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1fb1d28...7f2f392. Read the comment docs.


Valid git repository not found

Hi, I am using an indirect git repository. This means my git folder itself is not stored in the obsidian folder but outside. This worked well in the past, and also works with the git client and SourceTree (atlassian product).
It means there is a textfile in your folder called .git (not a folder) in which a redirect link is located. The contents of that file is:
gitdir: C:/Users/ZZZ/Documents/repos/Obsidian.git

This worked in the past, and is broken now, I assume there is a check before push if the .git folder is there? It does not have to be, it can also be a file.
It would be great if this can be fixed.

So the Obsidian.git folder is your actually git folder? And that's your root repo?

Fixed in new release

So the Obsidian.git folder is your actually git folder? And that's your root repo?

Yes but I am using the git repository directory option to specify my git files outside the obsidian directory since I am syncing the obsidian files to Google Drive and I don't want to sync all git files too. It is a nifty option to keep your working folder clean.

Thanks for the tip. Had exact that problem too.


Notification : add an option to automatically add a close button

This would replace the manual adding of b.Delete(...any) with e.OnClick(b.JSRemoveThisNotification)


Windows update didn't work and I can't install the new version

Please pardon my ignorance...
I've been running 18xx-maker on my local Windows PC for a while.  The recent release seems to have removed the version I had installed, without installing the new version.
I've tried downloading 18xx-maker-setup-1.0.0-beta-46.exe and running it by hand, but the installer pops up a message saying that Windows can't find 18xx maker.exe and asking me to locate it, which I can't do because the file no longer exists.
I've tried running the uninstall option and restarting Windows, but that didn't help.
How am I meant to install the new release?

Looks like there is a bug in electron builder. Going to try a different version to see if it works.
https://github.com/electron-userland/electron-builder/issues/5461

Latest version on our version of node still doesn't work. I'll try to update node / react / electron-builder ASAP to see if latest versions work but I don't know if I'll get to that today or not.

Attempting a node and react-scripts update: https://github.com/18xx-maker/18xx-maker/pull/485

Have it passing tests but some images on the site are broken. I will be fixing those later and then we'll get this out to see if it fixes the windows install.

Publishing a test with a much newer version of stuff that works on the latest node, we'll see if the windows build works.

Windows build still doesn't work, have to try more stuff later.

omg I think I solved it by building for windows ia32... please check beta 53

That works!  Thanks.


Implement ANALYZE on target variables

Problem
For a bayeslite generator G, MML supports running a generic "transition" operator:
ANALYZE G FOR <s> SECONDS WAIT
We are now at a stage where this per-generator ANALYZE is too coarse-grained. In a high-dimensional population pop, a user may only want to analyze a subset of the modeled variables. For a composite GPM with sub-cgpms consisting of gpmcc and others i.e. linear_regression, the runtime characteristics of ANALYZE will vary wildly -- gpmcc might need several hours, while linear_regression just needs a few minutes -- and it does not make sense to intertwine the transition operators in one big ANALYZE statement. Here is a working example:
CREATE POPULATION pop FROM pop.csv(
    a NUMERICAL,
    b NUMERICAL,
    ...
    y NUMERICAL,
    z NUMERICAL
)

CREATE GENERATOR G FOR pop USING cgpm(
    a LOGNORMAL,
    b EXPONENTIAL,
    -- Default components model for c through z.

    -- Override gpmcc for variable y.
    MODEL y GIVEN x, z USING linear_regression;
)

Currently, the cgpm_metamodel exercises an enormous hack over here where the hooked cgpms are transition for 1 iteration at composition time, and then never again!
Solution
Extend ANALYZE to support taking in an optional set of target variables. As it stands gpmcc accepts a set of target variables to transition against. For a composite cgpm, the runtime will look up the cgpms that model those variables, and use cgpm.transition(N, targets) . All this logic will be implemented by Engine and thus transparent to bayeslite (for model independence).
Here are some possible ways we can extend ANALYZE to gracefully deal with this situation:
Option 1
ANALYZE G
    TARGETING <vars, ...>
    FOR <n> ITERATION

ANALYZE G
    TARGETING *
    FOR <n> ITERATION

Option 2
ANALYZE <vars, ...>
    FROM G
    FOR <n> ITERATION;

ANALYZE *
    FROM G
    FOR <n> ITERATION;

Options 3+
Please weigh in!
The question is whether * should apply to all variables, or only those modeled by gpmcc. It could be a poor UI to have the user specify 25 variables modeled by gpmcc, leaving only 1 out (i.e. a through z except for y in the example above) if they do not want to transition the composed cgpms. One option could be
ANALYZE *
    FROM G
    [IGNORE | EXCEPT FOR | SKIP | etc] <vars, ...>
    FOR <n> ITERATION;

Another issue is coordinating the time for analysis for variables that come from different CGPMs, i.e. when a user says FOR <s> SECONDS. One solution is to disallow timed analysis when the target variables that are modeled by different cgpms. Timed analysis in lovecat is sub-optimal anyway since it is not parallelized at all, but gpmcc can do multiprocessed timed analysis.

First reaction: I anticipate that we will want to develop a richer and possibly per-metamodel language for programming analysis, much like we already have for expressing generators.  For example, it is not hard to imagine that we will want to be able to embed VentureScript for some things.
ANALYZE satellites_lovcat FOR 360 MINUTES (
    VARIABLES (x, y, z),
    HYPERPARAMETERS,
    ...
);
ANALYZE satellites_cgpm FOR 360 MINUTES (
    VARIABLES (x, y, z),
    VARIABLES (apogee, perigee, period)
        VENTURESCRIPT { mh(default, one, 25) },
    ...
);

On *: I can't imagine why you would want to make * apply to all variables including those that aren't modelled and hence can't be analyzed, which would render it unusable for ANALYZE.  It seems intuitively obvious to me that if there is any * notation it should apply only to modelled variables -- all variables that could possibly make sense in the context.
Timed analysis: Obviously we should have a serious story for timed analysis that is not lock-step deparallelized like lovecat is, by pushing the timing into the metamodel code.  The only reason we did not do that for lovecat was vkm's consistent exhortations to touch lovecat as little as possible.

@riastradh-probcomp yes, we talked about "more advanced" analysis languages for metamodels. I am bound to push that for the future, until we have a more concrete proposal for the MML syntaxes and use cases of custom inference programs.
I was reading the BayesDB paper, and the phrase that appears there for ANALYZE is
ANALYZE [variable subset OF] population FOR timelimit

In summary, I agree with you, but suggest we first do the variable subset, which we understand, is related to our immediate/medium goals, and we can hit the ground running directly using cgpm-lib (which takes in target variables, and I am adding the ability for State to transition hooked cgpms as well).

The reason I suggest the foundation of a more generic language is not that I think we should go beyond a simple set of target variables now, but rather that the work done to get
ANALYZE population FOR duration (
    VARIABLES (x, y, z)
)

working and factored generically enough for future use (a) is not terribly difficult and (b) would not be wasted for, or get in the way of, future extensions.

Related https://github.com/probcomp/bayeslite/issues/571

Fixed by e4138e1 which resolved  https://github.com/probcomp/bayeslite/issues/571


EDITOR-#13 Markers in Wavesurfer & Edit and Add button turn to Cancel after clicked

What is this PR for?

Add markers in wavesurfer to show frames' position.
EDIT and ADD buttons would toggle to Cancel. Click Cancel would reset currentStatus to origin.

What type of PR is it?
Feature
Todos
None
What is the Github issue?
#13
How should this be tested?

Add frame arbitrary, new markers would be showed in wavesurfer.
click EDIT or ADD buttons, make some changes, and click Cancel button.

Screenshots (if appropriate)

Questions:

Do the license files need updating? No
Are there breaking changes for older versions? No
Does this need new documentation? No

Besides, can you upload the control record in ./dance_json/control_merge_v6.json and show all the markers?
I want to know how it will look like with hundreds of markers.

@Kenchu123 I've already added some comments and deleted useless ones.

zoom = 1

zoom = 10

@caffrey928. I have seen the screenshot. I think the markers are too tight that maybe one will want to turn off the markers. Maybe you can add a switch or flag for turning off the marker display?

I've already add a switch.
Also, I change the color of markers and comment out the label of markers.


Added support for Enabling/Disabling Contacts

Thanks for the contribution, we're in the process of changing the API endpoints but we'll get this into the project if we can.

Thanks @cglrkn / @halitokumus :)


Fix post-deph lighttile

Do RenderPostDepthTile() once after depth pre-pass of the first view that doesn't have any nested views. Fixes the issue in #1212.

I can't comment on the blockiness, but without this (or the revert) I see a significant increase in dynamic light intensity and both this and the revert both return it to the same level.
I suppose LGTM?

Do you think it's odd that all three of us have 3 kinda separate issues?

Works for me, LGTM.

Do you think it's odd that all three of us have 3 kinda separate issues?

Not really, I suppose OpenGL buffers can have undefined contents.

Not really, I suppose OpenGL buffers can have undefined contents.

Yes, glBufferData() with a nullptr makes the contents of the buffer undefined.


Added Docker and Docker-Compose Support

Fixes

Fixes #94 by @pratikdaigavane
Description
Now we can start the development environment in just one line: docker-compose up.
Checklist

[x] My pull request has a descriptive title (not a vague title like Update index.md).
[x] My pull request targets the default branch of the repository (main or master).
[x] My commit messages follow best practices.
[x] My code follows the established code style of the repository.
[ ] I added tests for the changes I made (if applicable).
[x] I added or updated documentation (if applicable).
[x] I tried running the project locally and verified that there are no
visible errors.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

@krysal, please review the PR and do let me know if any changes are required!

@krysal, please review the PR and do let me know if any changes are required!

Thanks for showing initiative in this task @pratikdaigavane. I get an error when I run docker-compose up:
ERROR: The Compose file './docker-compose.yml' is invalid because:
Invalid top-level property "x-logging". Valid top-level sections for this Compose file are: version, services, networks, volumes, and extensions starting with "x-".

You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/
services.app.depends_on contains an invalid type, it should be an array

Docker version 19.03.14
docker-compose version 1.24.1

Please check you docker-compose.yml file, I think the default logging config is fine for development.

Thanks for showing initiative in this task @pratikdaigavane. I get an error when I run docker-compose up:
ERROR: The Compose file './docker-compose.yml' is invalid because:
Invalid top-level property "x-logging". Valid top-level sections for this Compose file are: version, services, networks, volumes, and extensions starting with "x-".

You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/
services.app.depends_on contains an invalid type, it should be an array

Docker version 19.03.14
docker-compose version 1.24.1

Please check you docker-compose.yml file, I think the default logging config is fine for development.


Remove unused functions: ComponentExist, PushedComponent

Signed-off-by: Parthvi Vala<EMAIL_ADDRESS>
What type of PR is this:
/kind cleanup

What does this PR do / why we need it:
This PR removes unused functions/methods that I encountered while working on #5923.
Which issue(s) this PR fixes:

Fixes #
PR acceptance criteria:

[ ] Unit test

[ ] Integration test

[ ] Documentation

How to test changes / Special notes to the reviewer:

/lgtm
/approve
/override Kubernetes Integration Tests/Kubernetes Integration Tests
failed to do request: Head "https://registry.stage.devfile.io/v2/devfile-catalog/python/manifests/1.0.0": read tcp <IP_ADDRESS>:46890-><IP_ADDRESS>:443: read: connection reset

/override ci/prow/unit-and-validate-test
Unit and validation tests pass on IBM Cloud


fix-JWT-refresh-tokens-unable-to-refresh-themselves

This change is needed because JwtTokenStore has no implementation for storeAccessToken and storeRefreshToken.
Which is fine, because JWT doesn't require persistence. But a refresh token is valid if

Digital signature is verified
There's an approval in the DB for all the scopes and the client_id, stored in the JWT itself.

When you refresh a token, the DefaultTokenServices removes all refresh tokens and access tokens. The only reasonable
way to 'remove' a JWT is to remove it's approval in the approval store. Since once removed it's never inserted again (user has to grant an approval again)
the next time you use a refresh token it will not be validated because of 2) and will fail.
A simple fix for that is providing an implementation for store(Access/Refresh)Token which just inserts an approval to the DB.

@gharizanov92 Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@gharizanov92 Thank you for signing the Contributor License Agreement!

Thanks for the PR @gharizanov92. There are a few failing tests that need to be resolved.
If you still feel this issues needs to be addressed, please fix the failing tests and add test(s) for the proposed changes.

@gharizanov92 I'm closing this PR due to lack of feedback from review.
If you still feel this is an issue, please submit a new PR and address the review feedback.


npm err when doing a dbus build

What I'm doing is upgrading the wifi connect code to support resin 2.0.   Following https://docs.resin.io/runtime/migrate-to-2.0/. Copied this project code into my project and merged the dockerfile (like before using resin 1.x).   Thought it would be an drop in upgrade but it wasn't. Reporting another error when npm is doing dbus build.
[Build]    Step 13 : RUN JOBS=MAX npm install --unsafe-perm --production     && npm cache clean
[Build]     ---> Running in be7cf738c160
[Build]    npm WARN deprecated<EMAIL_ADDRESS>..psst! While Bower is maintained, we recommend Yarn and Webpack for *new* front-end projects! Yarn's advantage is security and reliability, and Webpack's is support for both CommonJS and AMD projects. Currently there's no migration path but we hope you'll help us figure out one.
[Info]     Still working...
[Build]
[Build]    ><EMAIL_ADDRESS>install /app/pinballiot/node_modules/dbus
[Build]    > node-gyp configure build
[Build]    make: Entering directory '/app/pinballiot/node_modules/dbus/build'
[Build]      CXX(target) Release/obj.target/dbus/src/dbus.o
[Build]      CXX(target) Release/obj.target/dbus/src/connection.o
[Build]      CXX(target) Release/obj.target/dbus/src/signal.o
[Build]      CXX(target) Release/obj.target/dbus/src/encoder.o
[Build]      CXX(target) Release/obj.target/dbus/src/decoder.o
[Build]      CXX(target) Release/obj.target/dbus/src/introspect.o
[Build]      CXX(target) Release/obj.target/dbus/src/object_handler.o
[Build]      SOLINK_MODULE(target) Release/obj.target/dbus.node
[Build]      COPY Release/dbus.node
[Build]    make: Leaving directory '/app/pinballiot/node_modules/dbus/build'
[Build]    npm notice created a lockfile as package-lock.json. You should commit this file.
[Build]    added 52 packages in 72.581s
[Build]    npm ERR! As of npm@5, the npm cache self-heals from corruption issues and data extracted from the cache is guaranteed to be valid. If you want to make sure everything is consistent, use 'npm cache verify' instead.
[Build]    npm ERR!
[Build]    npm ERR! If you're sure you want to delete the entire cache, rerun this command with --force.
[Build]
[Build]    npm ERR! A complete log of this run can be found in:
[Build]    npm ERR!     /root/.npm/_logs/2017-06-14T04_16_41_794Z-debug.log
[Build]    Removing intermediate container be7cf738c160
[Error]    Build failed: The command '/bin/sh -c JOBS=MAX npm install --unsafe-perm --production     && npm cache clean' returned a non-zero code: 1

Alberts-MacBook-Pro:pinballiot alwong$ OBS=MAX npm install --unsafe-perm --production     && npm cache clean --force
npm WARN deprecated<EMAIL_ADDRESS>..psst! While Bower is maintained, we recommend Yarn and Webpack for *new* front-end projects! Yarn's advantage is security and reliability, and Webpack's is support for both CommonJS and AMD projects. Currently there's no migration path but we hope you'll help us figure out one.

><EMAIL_ADDRESS>install /Users/alwong/sandbox/pinballiot/node_modules/dbus
> node-gyp configure build

  CXX(target) Release/obj.target/dbus/src/dbus.o
../src/dbus.cc:6:10: fatal error: 'dbus/dbus.h' file not found
#include <dbus/dbus.h>
         ^
1 error generated.
make: *** [Release/obj.target/dbus/src/dbus.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:276:23)
gyp ERR! stack     at emitTwo (events.js:106:13)
gyp ERR! stack     at ChildProcess.emit (events.js:191:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12)
gyp ERR!  Darwin 16.6.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "--unsafe-perm" "--production"
npm ERR! node v6.10.1
npm ERR! npm  v3.10.10
npm ERR! code ELIFECYCLE

npm ERR<EMAIL_ADDRESS>install: `node-gyp configure build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script 'node-gyp configure build'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the dbus package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-gyp configure build
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs dbus
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls dbus
npm ERR! There is likely additional logging output above.

upgrading to latest stable
sudo npm cache clean -f
sudo npm install -g n
sudo n stable

../src/dbus.cc:6:10: fatal error: 'dbus/dbus.h' file not found
#include <dbus/dbus.h>
         ^
1 error generated.
1 error generated.
make: *** [Release/obj.target/dbus/src/object_handler.o] Error 1
make: *** Waiting for unfinished jobs....
make: *** [Release/obj.target/dbus/src/introspect.o] Error 1
1 error generated.
1 error generated.
1 error generated.
make: *** [Release/obj.target/dbus/src/connection.o] Error 1
make: *** [Release/obj.target/dbus/src/decoder.o] Error 1
make: *** [Release/obj.target/dbus/src/signal.o] Error 1
1 error generated.
make: *** [Release/obj.target/dbus/src/dbus.o] Error 1
1 error generated.
make: *** [Release/obj.target/dbus/src/encoder.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:285:23)
gyp ERR! stack     at emitTwo (events.js:125:13)
gyp ERR! stack     at ChildProcess.emit (events.js:213:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:197:12)
gyp ERR! System Darwin 16.6.0
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "build"
gyp ERR! cwd /Users/alwong/sandbox/pinballiot/node_modules/dbus
gyp ERR! node -v v8.0.0
gyp ERR! node-gyp -v v3.6.1
gyp ERR! not ok
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>install: `node-gyp configure build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

 There is likely additional logging output above.
826 verbose exit [ 1, true ]

Rolled back to a clean project without wifi code. Tested clean.  Manually added the new wifi code.   Works just fine.

I'm getting the same error.  Tried to create a new application and still can't build. What do you mean by project without wifi code?

Fixed it by adding to Dockerfile template:
RUN apt-get update
&& apt-get install -y python
python-dev
python-dbus
python-pip
build-essential


Cleanup of temporary files for client requests after request processing

Describe the feature you'd like to add to nginx
As far as I can see in the documentation, the option to clean up temporary files only exists for directive client_body_in_file_only set to clean. 1, but we don't have that setting configured.
Large request bodies are stored in the path configured with client_body_temp_path, but I don't see an option to automatically delete these after the request has been handled. 2
I don't think client_body_in_file_only should be used for this, but it could make sense to add a separate directive for this, or even set this as the default, right? Or am I missing a setting?
Describe the problem this feature solves
We're running into issues where users upload large files. The request bodies are stored in a temporary file, but that temporary file never gets cleaned up, resulting in us running out of disk space. (To be clear, we don't have client_body_in_file_only configured.)
There are some older discussions on several forums where this is solved with tmpwatch or some other custom solution:
https://stackoverflow.com/questions/21484788/nginx-automated-tmp-cleanup
https://serverfault.com/questions/570052/nginx-temporary-files-cleanup-automatically
But some other sources say this should be the default behaviour.
Additional context
We do have fastcg_request_buffering enabled, that might be related?

All nginx temporary files should be deleted automatically after request completion, unless it's explicitly disabled. This may not work properly if:

the request takes a long time to complete; all this time the file will be on disk
nginx worker is killed, in which case the file will never be deleted

Indeed fastcgi_request_buffering directive may result in creation of temporary files on disk and they should be automatically deleted, as stated above.
Please make sure nginx worker was not killed and the request which created the file has finished.
In case it's all ok, please enable debug logging and find all lines in error.log related to a particular request body file that was created and never deleted. Those lines need to be analyzed.

Thanks for the extensive response and explanation of expected behaviour! I've been trying to reproduce the behavior we see here locally, but I'm unable to reproduce this currently. I'll close this issue here and contact our hosting provider. Thanks again for the help!


Make sure current working API is documented up to date

So other groups know how our stuff works and how to connect
(note: remember to do friend/{user1}/{user2} works for if friends)

all done i believe!


my findings from ZITADEL testing

[ ] Sidepanel collaps arrow overlapping with scroll

[ ] dropdown org chooser jumps - (I have a videoclip if needed (can't upload here))

[ ] password policies not known to user at initial setup

[ ] password policies error too cryptic

[ ] password not equal only disables button (user don't know why) and don't get an error if not changing focus

[ ] user: table no margin between table rows (Email/Status) (also labels)

[ ] username length unlimited: intended? (possible UI/displaying issues)

[ ] space only in names intended?

[ ] roles: if I set a new role, it's not instantly in the list after finishing setup. (only after reload page)

[ ] authorization: unclear, cryptic error message (reason unknown):

[ ] deactivate (project) no instant feedback to user on UI (besides fly-in message). button doesn't change and if clicked again gets a cryptic error message

[ ] same with add manager to project.

-[ ] add manager: if clicking on button while non-valid email in the input, it gets deleted (confusing interaction)
(I have a videoclip if needed (can't upload here))

[ ] hover effects on many interaction elements are not present (for example edit user)

[ ] phone number can be only letters (without numbers)

[ ] get cryptic 'undefined' error message if done

[ ] after click on 'initialisierung abschliessen' (mail) i get an error. (mobile)

[ ]after clicking on 'verify email' i get to the browser and have to go back to the mailing program to copy code. not very user friendly

[ ] and it doesn't seem to work after i entered it (EventdtGaa error)

[ ] dashboard: user administration leads only to my own user, not the list of the project users

[ ] deleting password policy button doesn't do anything and don't give feedback why not

[ ] domain with only one character valid. intended?

[ ] sidepanel information global. intended? I expected them to be only from the active ORG

[ ] light design should also have a primary color (not grey!) same blue as in dark should be viable.

[ ] user toasts saved message (reason unclear)

[ ] email allows unsupported special characters (like brackets)


Avoid exceeding maximum safe Javascript integer

Some JSON libraries cannot parse integers greater than 2^53-1.
So clamp "no limit" 2^64 sigop/sighash limits to 2^53-1 before
returning them.
A couple of implementation notes:
Changing the constant returned by the consensus/params.h methods
from uint64_t::max to 2^53-1 would be another fix, but that felt
like a layering violation: the consensus code shouldn't care about
details of representing numbers in JSON.
Also, I didn't define JSON_NUMBER_LIMIT in a .h file because it
isn't clear where it belongs (util.h? one of the json/ .h?) and
because at this point it makes sense to try to create patches
that change fewer files.
Fixes issue# 45.
After review, should be pulled into the only_bigblocks branch, also.

LGTM
I asked the bug reporter to test it out. The fix seems like a no-brainer but it'd be nice to ensure there aren't other weird compatibility issues there.
In future it'd be nice to offer something better than JSON for the RPC layer e.g. protobufs. JSON just seems riddled with obscure quirks.

utACK.  I see, "sigoplimit" would have been a problem post-fork.

Tested by running two -regtest instances, having one mine a block (so getblocktemplate doesn't complain about being in initial-block-download state), and then observing getblocktemplate output:
gavin$ ./bitcoin-cli -regtest -datadir=$HOME/src/bitcoin/src/.rt2 getblocktemplate
{
    "capabilities" : [
        "proposal"
    ],
    "version" : 536870919,
    "previousblockhash" : "2f7d4ea63ebfd6eec89b3985da6c06cd715b89abc28a79451c8e624148afc9f8",
    "transactions" : [
    ],
    "coinbaseaux" : {
        "flags" : ""
    },
    "coinbasevalue" :<PHONE_NUMBER>,
    "longpollid" : "2f7d4ea63ebfd6eec89b3985da6c06cd715b89abc28a79451c8e624148afc9f82",
    "target" : "7fffff0000000000000000000000000000000000000000000000000000000000",
    "mintime" :<PHONE_NUMBER>,
    "mutable" : [
        "time",
        "transactions",
        "prevblock"
    ],
    "noncerange" : "00000000ffffffff",
    "sizelimit" : 1000000,
    "sigoplimit" : 20000,
    "accuratesigoplimit" :<PHONE_NUMBER>740991,
    "sighashlimit" :<PHONE_NUMBER>740991,
    "curtime" :<PHONE_NUMBER>,
    "bits" : "207fffff",
    "height" : 4
}

Didn't work here. The application throws the following warning during compilation:
rpcmining.cpp:592:39: warning: left shift count >= width of type
     uint64_t JSON_NUMBER_LIMIT = (1L<<53)-1;

The numbers were still reported as the largest unsigned 64-bit integer instead of 2^53.
Thankfully, the fix was as easy as changing the (1L<<53)-1 to (1LL<<53)-1 (long long instead of long).
After that the max values are reported correctly as<PHONE_NUMBER>740991. cgminer also connects to the node without any issues after the fix.

Replaced the shift-and-subtract with explicit<PHONE_NUMBER>740991 constant, which should be the most portable way of doing it (see http://en.cppreference.com/w/cpp/language/integer_literal -- LL suffix is C++11).

Tried with a constant value and it works correctly.

Perfect, thanks guys.

I wonder if null would be more appropriate here? Or a string "Infinity" or the boolean false (ie, no limit)?

@luke-jr : if you are working in a strongly-typed language, RPC values that can be more than one type or can be missing entirely are painful to deal with. And if they suddenly switch from an integer to a string (as would happen when the fork activates) that's a very good place for a very bad bug to happen if the code interpreting the RPC value isn't written properly.

Fair enough; I can't think of any practical harm to representing a practically-unbounded limit like this.


New leadership election algorithm adapted to gRPC model

Related to: https://github.com/giantswarm/giantswarm/issues/506

related to this: https://github.com/giantswarm/giantswarm/issues/496

In here: https://github.com/giantswarm/fleet/pull/18#issuecomment-185652155
The above graphic represents the current logic, while the bottom one the newest.

@ping @htr I think we can go ahead with this implementation and merge it upstream into our PR.

LGTM


Admin screen to test attribute resolution/release

There are scenarios where CAS admins may need to learn what attributes may be resolved by the attribute resolver machinery for a given user, in order to troubleshoot attribute release, etc. The CAS admin UIs should provide this facility.
Note that this does NOT take into account attributes that are resolved by the authentication engine directly. We're descoping that bit, given it requires credentials to successfully execute authn to retrieve attributes. This issue specifically tracks attribute repositories. Support for the authentication engine may be worked out elsewhere.

Will need @mindblender's help on this one.

You should be able to test/try this in about an hour or so, by using the new SNAPSHOT.


[carbon dashboard] Site down, possible 500 server error

Carbon dashboard is down https://carbon.klimadao.finance/

see #1497

see #1497

It is this one: https://github.com/KlimaDAO/dash-apps/pull/172


Sidebar button crashes browser

Hi, I am using Brave, have installed code, set .env vars, and ran with turbo dev.  Installed extension, clicked icon, logged into google (have Google OAUTH info in .env).  Then, I see the sidebar button highlighted, if I click it, the browser crashes.  There are no errors or logs of the event.  I tried opening the service workers logs, again, nothing, they crash too.  Please advise how to get running on virgin install.  Something seems to be missing in instructions, I am wondering if there is an install step missing.  Looks promising, would like to try it and extend it, but can't get off the ground.  I am happy to connect and share screens to troubleshoot. Thanks.

Hey! Glad to hear to you want to get involved. Are you ok sending me your email address? I can add you to a Slack channel so we can share screenshots etc.


compile error with format string

The CI doesn't seem to check debug info in build. When configured with
../configure CXXFLAGS="-O0 -g -ggdb" --enable-debug CPPFLAGS="-DLOG_LEVEL=LOG_LEVEL_TRACE"

Error:
In file included from ../../src/backend/storage/data_table.cpp:22:0:
../../src/backend/storage/data_table.cpp: In member function 'peloton::ItemPointer peloton::storage::DataTable::GetEmptyTupleSlot(const peloton::storage::Tuple*, bool)':
../../src/backend/common/logger.h:132:43: error: use of deleted function 'std::atomic<long unsigned int>::atomic(const std::atomic<long unsigned int>&)'
   ::fprintf(LOG_OUTPUT_STREAM, __VA_ARGS__);                          \
                                           ^
../../src/backend/storage/data_table.cpp:152:3: note: in expansion of macro 'LOG_TRACE'
   LOG_TRACE("tile group count: %lu, tile group id: %lu, address: %p",
   ^
In file included from ../../src/backend/concurrency/transaction.h:16:0,
                 from ../../src/backend/storage/data_table.h:22,
                 from ../../src/backend/storage/data_table.cpp:17:
/usr/include/c++/4.8/atomic:658:7: error: declared here
       atomic(const atomic&) = delete;
       ^
In file included from ../../src/backend/storage/data_table.cpp:22:0:
../../src/backend/common/logger.h:132:43: error: cannot pass objects of non-trivially-copyable type 'struct std::atomic<long unsigned int>' through '...'
   ::fprintf(LOG_OUTPUT_STREAM, __VA_ARGS__);                          \
                                           ^
../../src/backend/storage/data_table.cpp:152:3: note: in expansion of macro 'LOG_TRACE'
   LOG_TRACE("tile group count: %lu, tile group id: %lu, address: %p",
   ^
../../src/backend/common/logger.h:132:43: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'std::atomic<long unsigned int>' [-Werror=format=]
   ::fprintf(LOG_OUTPUT_STREAM, __VA_ARGS__);                          \
                                           ^
../../src/backend/storage/data_table.cpp:152:3: note: in expansion of macro 'LOG_TRACE'
   LOG_TRACE("tile group count: %lu, tile group id: %lu, address: %p",
   ^

Excellent find. I will update the Jenkins build configuration.

PR #162 updating travis and fixing other string format issue


Support for "select type" construct

The Fortran 2003 feature "select type" which allows you to find the type of a polymorphic entity is not supported through syntax highlighting like "select case" is.

Fixed in #42


Linting error for all pull request automated testing

Describe the issue/behavior that seems buggy
https://github.com/highlightjs/highlight.js/blame/3d5d07e0c93f79df33f18971a93cb83c28f75bf3/tools/lib/makestuff.js#L8
When running any automated PR tests or npm run lint, an error is generated:
<dir>/highlight.js/tools/lib/makestuff.js
  8:19  error  Parsing error: Unexpected token import

This makes all pull requests fail testing.
Sample Code or Instructions to Reproduce
npm run lint
or
open a pull request
Error is in line to of this snippet:
async function clean(directory) {
  let del = await import('del');
  del.deleteSync([directory]);
  fs.mkdirSync(directory, { recursive: true });
}

Expected behavior
All tests should pass
Additional context
There is an issue in the eslint repo with the response:

When using the default parser and sourceType: "module", the word import becomes a keyword token that can only be used in import declarations.

https://github.com/eslint/eslint/issues/11189#issuecomment-446586797

If you know how to fix this a PR would be welcome.  I haven't had a chance to look yet.


create rocket project [backend]

For the backend we shall use the rocket framework

ok so the rocket project is now made.... closing :)


[MRG] Make the URL used to generate launch badges configurable

This lets you set the base URL used for generating badges. This is useful when you have a group of hubs at different hostnames but want badges generated on all of them to point to a common hostname. I think we should use this for operating the GKE and OVH cluster as a federation so that all clusters generate https://mybinder.org/ badges and launch URLs.

This looks great to me - the one thing that's missing is any documentation about this feature 📚 . I guess it would show up here: https://binderhub.readthedocs.io/en/latest/reference/app.html maybe that's enough?

Nods that is where you'd find an entry.
We could write a short how to section on setting up a federation? I think that is the only context in which you'd want to use this feature.

@betatim just to double check are you planning on putting off the docs for this for another PR? (in which case I can open an issue about it so we don't forget)

It should appear automatically in the reference section of the docs. Not sure any other documentation needs writing.
I'd make writing "howto setup a federation" a new issue as that will require a bit of thinking (for example what redirector should people use).

then thanks for the contribution @betatim !


Invoked in extraction

I run the script in MISC to Project Vanilla NeRF to PlenOctree an error happened

and I finally find out only running this line
https://github.com/sxyu/plenoctree/blob/f0b82631199a1aa7dc9ce263c08980a3a7504014/octree/nerf/models.py#L70
import this package will get that error.
Anyone knows what happened?

Probably there exists some conflict between your cuda version and jaxlib. I have met similar error as yours.
For stable jaxlib, I suggest you to use cuda11.1.


Fix crash in - [ARTiledImageView drawRect:], line 112

Version: 2.3.2 (2015.12.01) | net.artsy.artsy
Stacktrace
ARTiledImageView;drawRect:;ARTiledImageView.m;112
ARTiledImageViewWithBreadcrumb;drawRect:;ARZoomView.m;33
Reason
objc_msgSend() selector name: bounds
Link to HockeyApp

https://rink.hockeyapp.net/manage/apps/37029/crash_reasons/73191802

Closing in favour of #773.


Nutanix AHV is incorrectly identified as Hyper-V

Version (k3OS / kernel)
$ k3os --version
k3os version v0.10.0
$ uname --kernel-release --kernel-version
5.0.0-43-generic #47~18.04.1 SMP Wed Apr 1 16:27:01 UTC 2020
$ uname -m
x86_64
Describe the bug
k3os is deployed on Nutanix AHV hypervisor, which is based on KVM/qemu.
But hypervisor is incorrectly identified as "Hyper-V".

To Reproduce
$ sudo virt-what
hyperv
Expected behavior
Hypervisor should be identified as Nutanix AHV, or KVM/qemu.
Additional context
This issue has already been adressed in the upstream version of  virt-what:
http://git.annexia.org/?p=virt-what.git;a=summary

I think we need an upstream release (aka tag) before the Alpine maintainers will take it up.


⚠ ClamAV has degraded performance

In ab71b57, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 197 ms

Resolved: ClamAV performance has improved in e9320e5.


Main page having issues loading NFT metadata

Wanted to report some issues with fetching NFT metadata.

I'm really excited about this SDK.

Try using a thirdweb client id in your ThirdwebManager if you aren't already. You can get one for free here.

Just realized this is our preview build, needs updating thanks for the report!


How do I load multiple different models using 'lmdeploy serve api_server'? Run 'lmdeploy serve api_server' again with new --server-port?

you got it


Overlay Output and Type Map

I trained the same model that you propose in the repository. However, the prediction overlays have different colors than the ground truth (Figure). This is normal? (I ask because the metrics are good) I would also like to know how I can obtain the type maps through the results obtained when running infer.py since the function returns:
'bbox': bounding box coordinates for each nucleus
'centroid': centroid coordinates for each nucleus
'contour': contour coordinates for each nucleus
'type_prob': per class probabilities for each nucleus (default configuration doesn't output this)
'type': prediction of category for each nucleus
Image tiles output a mat file, with keys:
'raw': raw output of network (default configuration doesn't output this)
'inst_map': instance map containing values from 0 to N, where N is the number of nuclei
'inst_type': list of length N containing predictions for each nucleus
Image tiles output a png overlay of nuclear boundaries on top of original RGB image

You can specify the colour of the overlay that you wish to see in this file.

Thank you!!! And how could I get the type map prediction from the output .mat files while running tile.sh?


Bump kernel version to 4.9.8

Purpose

$subject

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/5546719989

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/5546719989
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/5549850461

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/5549850461
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/5550394923

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/5550394923
Status: success

Approving the pull request based on the successful pr build https://github.com/wso2/product-is/actions/runs/5550394923

Approving based on the integration test results.


Automatically add to Kainda Model the user-generated methods and statics defined in the schema

In Mongoose is relatively easy because the schema.statics is the user-defined statics and schema.methods the user-defined instance methods, so it should be easy

For Sequelize though, it may be needed to use this paradigm
class A {
hello() {
console.log("Hello from class A");
}
}
class B extends A {
goodbye() {
console.log("Goodbye from class B");
}
}
const instanceB = new B();
console.log(instanceB.hasOwnProperty("hello")); // false
console.log(instanceB.hasOwnProperty("goodbye")); // true


Custom schema filters missing causing conflicts on push

Link to reproduction
No response
Describe the Bug
I am working on a project that requires the ability to use different schemas within the DB.
I have managed to setup targeting a custom schema successfully but have noticed when attempting to push to the schema via payload an error occurs if any of the tables in the new schema have the same name as a 'public' schema table.
For instance if I have a 'users_organisations' table on public and then point payload to a 'test' schema I cannot push my tables to the 'test' schema without receiving the following error.
error: relation "test.users_organisations" does not exist
    at /{{MY DIRECTORY}}/node_modules/.pnpm/pg-pool@3.6.1_pg@8.11.3/node_modules/pg-pool/index.js:45:11
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async DrizzleORMPgClient.query (/{{MY DIRECTORY}}/node_modules/.pnpm/drizzle-kit@0.20.14-1f2c838/node_modules/drizzle-kit/payload.js:34498:21)
    at async apply (/{{MY DIRECTORY}}/node_modules/.pnpm/drizzle-kit@0.20.14-1f2c838/node_modules/drizzle-kit/payload.js:36648:9)
    at pushDevSchema (/{{MY DIRECTORY}}/node_modules/.pnpm/@payloadcms+db-postgres@3.0.0-alpha.49_@types+react@18.2.64_payload@3.0.0-alpha.49_react@18.2.0/node_modules/@payloadcms/db-postgres/dist/utilities/pushDevSchema.js:35:5)
    at Object.connect (/{{MY DIRECTORY}}/node_modules/.pnpm/@payloadcms+db-postgres@3.0.0-alpha.49_@types+react@18.2.64_payload@3.0.0-alpha.49_react@18.2.0/node_modules/@payloadcms/db-postgres/dist/connect.js:77:5)
    at localOperations (/{{MY DIRECTORY}}/node_modules/.pnpm/payload@3.0.0-alpha.49_@swc+core@1.4.8_@swc+types@0.1.6/node_modules/payload/dist/index.js:239:27)
    at localOperations (/{{MY DIRECTORY}}/node_modules/.pnpm/payload@3.0.0-alpha.49_@swc+core@1.4.8_@swc+types@0.1.6/node_modules/payload/dist/index.js:239:27)
    at resetBaseDB (/{{MY DIRECTORY}}/src/database/reset/resetBaseDb/resetBaseDb.ts:5:18)
    at runAndKill (/{{MY DIRECTORY}}/src/database/runAndKill.ts:6:2) {
  length: 122,
  severity: 'ERROR',
  code: '42P01',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'namespace.c',
  line: '428',
  routine: 'RangeVarGetRelidExtended'
}

After a little while of debugging and testing I realised that if I dropped my public schema so that it did not contain the 'users_organisations' table I could then push to the 'test' schema successfully.
In the drizzle documentation there is mention that you cannot have the same table across multiple schemas however, this can be bypassed using a schemaFilter.
I noticed in the codebase for payload/db-postgres there was no usage of the schemaFilter nor any way to add your own usage of the schemaFilter.
I believe implementing a way to add your own schemaFilter to the postgres db adapter would allow circumventing of this limitation and resolve it until drizzle develop a more permanent solution to the issue.
Note: This works fine the other way around from my testing. Having a 'test' schema table and then pushing a 'public' schema table with the same name goes through fine.
To Reproduce

Create a payload instance pointing at public and push the schema
Point the payload instance at a custom schema
Attempt to push the payload tables into the custom schema

Payload Version
3.0.0-alpha.49
Adapters and Plugins
@payloadcms/db-postgres

I'm glad you pointed this out! I ran into some odd behavior in my testing and couldn't recreate it so I moved on. This explains what I was seeing now, thanks for opening this issue!

@mcornielje090404 I can't decide if this needs to be an additional configurable property or if it is better to simply pass the db adapter schemaName through to drizzle in the schemaFilter property.
It seems reasonable to say that each Payload project can only use one schema and enforce that this way.

@DanRibbens I reckon since we can only have 1 schema at any given point currently just automatically pass it through, would reduce possibility of devs accidentally missing it. Can revise later if changes are made to schema handling in Payload.


neurodata footer

i still found it a bit too salient.
i made it more transparent and the logo smaller.
i think it is better along both dimensions.  thoughts?
(recall: page numbers and branding should be far less salient than the content on the pages).

agreed 👍
-CS


Doc Feedback:

Location : https://is.docs.wso2.com/en/latest/guides/adaptive-auth/configure-adaptive-auth/
I am using openJDK 18, but still adaptive authentication is disabled by default

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Ellipsize narrow description in app bar for message list

In the app bar for the message list, we describe the narrow you're looking at: the stream and topic, or list of DM recipients, etc. That can easily be too much text to fit, and in that case we just run off the end. Here's how that looks in debug mode with its warning stripes:

Instead, we should bound the width to fit and ellipsize the text.
A similar issue applies to the recipient headers, but less severely: there's more room there, and a smaller font size (body-text size instead of a big title size), so the overflow is much less common.

Hey @gnprice I was wondering wether I could work on this if it's open to take up?

This issue no longer reproduces; it looks like we just neglected to close this issue thread.
(Specifically, both the app bar and the recipient headers ellipsize properly when the topic name is long.)
@ShivamMenda Please take a look at the Zulip project's guidance for picking an issue to work on. In this repo we don't have the "help wanted" label because this codebase isn't yet fully set up to welcome contributors; but that just makes the other steps all the more important.