CERT/CC SSVC metrics for CVE using ADP

This is a follow-up after discussions in CVE QWG meeting on the topic of being able to publish as an Authorized Data Provider (ADP) into CVE's current JSON schema.  CERT/CC Stakeholder Specific Vulnerability Categorization (SSVC) project attempts to provides vulnerability metrics in the form of decision trees for different vulnerability management communities.
More information about SSVC can be found SSVC Overview.  In practice, SSVC code, examples and customization information are available in GitHub repository (https://github.com/CERTCC/SSVC.
CERT/CC would like to publish such metrics in adherence to the CVE-5 JSON schema. We have a sample ADP enhanced CVE record that is available at https://democert.org/ssvc/cve-5/CVE-2022-0012-adp.json.  This record validates properly for the current CVE-5.0 JSON schema.
The ADP container data from the example is also included here for convenience.  Let us know how we can provide such data into CVE to support enrichment of the CVE JSON records.
	"adp": [{
            "providerMetadata": {
                "dateUpdated": "2022-02-09T18:45:53Z",
                "orgId": "e9c1279f-00f6-4ef7-9217-f89ffe703ec0",
                "shortName": "cert_cc"
            },
	    "datePublic": "2022-01-27T00:00:00",
	    "metrics": [{
		"other": {
		    "type": "ssvc",
		    "content": {
			"role": "Coordinator",
			"id": "CVE-2022-0012",
			"version": "2.0",
			"generator": "Dryad SSVC Calculator 5.1.1",
			"computed": "SSVCv2/E:N/A:Y/T:P/P:M/B:M/M:L/D:T/2022-02-09T18:45:53Z/",
			"timestamp": "2022-02-09T18:45:53Z",
			"options": [
			    {
				"Exploitation": "none"
			    },
			    {
				"Automatable": "yes"
			    },
			    {
				"Technical Impact": "partial"
			    },
			    {
				"Mission Prevalence": "Minimal"
			    },
			    {
				"Public Well-being Impact": "Minimal"
			    },
			    {
				"Mission & Well-being": "low"
			    },
			    {
				"Decision": "Track"
			    }
			],
			"$schema": "https://democert.org/ssvc/SSVC_Computed_v2.02.schema.json",
			"decision_tree_url": "https://democert.org/ssvc/CISA-Coordinator-v2.0.3.json"
		    }
		}
	    }]

	}]

Thanks
Vijay
Additional stakeholders highlighted:
@zmanion @david-waltermire-nist @chandanbn

I don't think that
"$schema": "https://democert.org/ssvc/SSVC_Computed_v2.02.schema.json",

will be accepted by CVE Services during an ADP container submission. CVE Services uses Amazon DocumentDB to store JSON documents, and doesn't allow a $ character in that context (even though the schema allows it):

https://docs.aws.amazon.com/documentdb/latest/developerguide/functional-differences.html#functional-differences.field-name-restrictions
Amazon DocumentDB also does not support the $ prefix in field names. The following field names that begin with the $ prefix have been whitelisted and can be successfully used in Amazon DocumentDB: $id, $ref and $db.

$schema is not accepted by the implementation. See https://github.com/CVEProject/cve-schema/issues/145 for other information about what happens within the CVE Services code, and what error a client would see.

Hello @ElectricNroff
Thanks for your quick response. Happy to modify he $schema to be reference_schema (as below), avoid any $ references so it poses less trouble. I was using $schema from JSON doc recommendations. not really married to that notation. Let me know if there are any other concerns to address.
	"adp": [{
            "providerMetadata": {
                "dateUpdated": "2022-02-09T18:45:53Z",
                "orgId": "e9c1279f-00f6-4ef7-9217-f89ffe703ec0",
                "shortName": "cert_cc"
            },
	    "datePublic": "2022-01-27T00:00:00",
	    "metrics": [{
		"other": {
		    "type": "ssvc",
		    "content": {
			"role": "Coordinator",
			"id": "CVE-2022-0012",
			"version": "2.0",
			"generator": "Dryad SSVC Calculator 5.1.1",
			"computed": "SSVCv2/E:N/A:Y/T:P/P:M/B:M/M:L/D:T/2022-02-09T18:45:53Z/",
			"timestamp": "2022-02-09T18:45:53Z",
			"options": [
			    {
				"Exploitation": "none"
			    },
			    {
				"Automatable": "yes"
			    },
			    {
				"Technical Impact": "partial"
			    },
			    {
				"Mission Prevalence": "Minimal"
			    },
			    {
				"Public Well-being Impact": "Minimal"
			    },
			    {
				"Mission & Well-being": "low"
			    },
			    {
				"Decision": "Track"
			    }
			],
			"refrence_schema": "https://democert.org/ssvc/SSVC_Computed_v2.02.schema.json",
			"decision_tree_url": "https://democert.org/ssvc/CISA-Coordinator-v2.0.3.json"
		    }
		}
	    }]

	}]

Just capturing these notes from QWG meeting on 2022/03/10.  Currently the CVE Services 2.1 which is about to launch and be ready soon will not support publishing of ADP containers. It is planned somewhere in the fall time for CVE services next revision to accept JSON ADP containers.  Once we are able publish, CERT/CC can request an update to the CVE 5.1 JSON schema to include a well-formatted SSVC ADP record as a metric.
Vijay

The QWG will need to address this as a new optional feature in v5.1. Marking this milestone.

SPWG is prioritizing ADP, this came up at today's meeting in that certain ADPs, if they are approved to provide content that is not already part of the existing CNA container schema, would have to create custom schema. This is the case for an SSVC ADP. It seems that a references ADP may be the first pilot, however we'll need to develop process for custom schema development and inclusion at some point.

Our ADP container updated setup has been updated as the following. We are also tracking this with discussions under our SSVC - https://github.com/CERTCC/SSVC/discussions/229
{
    "adpContainer": [
        {
            "providerMetadata": {
                "dateUpdated": "2022-02-09T18:45:53Z",
                "orgId": "e9c1279f-00f6-4ef7-9217-f89ffe703ec0",
                "shortName": "cert_cc"
            },
            "datePublic": "2022-01-27T00:00:00",
            "metrics": [
                {
                    "other": {
                        "type": "ssvc",
                        "content": {
                            "role": "CISA-Coordinator",
                            "id": "CVE-2022-45119",
                            "version": "2.0",
                            "generator": "Dryad SSVC Calculator 5.1.7",
                            "computed": "SSVCv2/E:P/A:N/T:T/P:S/B:M/M:M/D:R/2023-06-20T14:33:57Z/",
                            "timestamp": "2023-06-20T14:33:57Z",
                            "options": [
                                {
                                    "Exploitation": "poc"
                                },
                                {
                                    "Automatable": "no"
                                }
                            ],
                            "reference_schema": "https://certcc.github.io/SSVC/ssvc-calc/SSVC_Computed_v2.03.schema.json",
                            "decision_tree_url": "https://certcc.github.io/SSVC/ssvc-calc/CISA-Coordinator-v2.0.3.json"
                        }
                    }
                }
            ]
        }
    ]
}

ADP testing is ready to start and we will be working with CVE AWG on feedback.
Thanks
Vijay

We have a production schema now as version 1.0.1 using schemaVersion templates. This should be ready for the v5.2.0 integration. Please collect the latest from
https://certcc.github.io/SSVC/data/schema/v1/Decision_Point_Group-1-0-1.schema.json

Created pull request #348 to implement SSVC as an imported schema.

Please see #350  is the new PR that covers all the inclusions and fixes.


Add set_frame, pause, and oneshot to AnimatedTexture

Pull requested for godot-proposals #686
I think the ability to control animations in AnimatedTexture, possibly via AnimationPlayer or other things, is a nice feature, it doesn't break any compatibility, and is very easy to write as well. I don't think it can possibly conflict with anything, really.
I needed this for my project, and maybe there are others who need it as well, so I gave it some polishing and created this PR.

Thanks, and congrats for your first merged PR! :tada:

Cherry-picked for 3.2.2.


[libflac] update to 1.4.1

Library name: libflac
New version number: 1.4.1
https://xiph.org/flac/changelog.html
https://github.com/xiph/flac/releases/tag/1.4.1

Hi @JonLiu1993, do you have a planned date for this update?
Thanks.

@HakanDanisik , I am working on updating this port, I will commit a pr today.

https://github.com/microsoft/vcpkg/issues/29588
https://github.com/microsoft/vcpkg/pull/29584


Added BigInteger method for Fermats Factorization

Added BigInteger method for Fermats Factorization and some supporting functions for BigInteger method. Also use BigDecimal for square root calculation.

Thanks, Merged!


feat: use fuel-abi-types 0.5.0

This PR bumps fuel-abi-types to 0.5.0 which internally changes the log_id type from u64 to String.
Checklist

[x] I have done my best to ensure that my PR adheres to the Fuel Labs Code Review Standards.
[x] I have requested a review from the relevant team or maintainers.

:shipit:


MINOR: improve error reporting in DescribeConsumerGroupTest

nit: It's a bit more difficult to find usages of an exception class when we assert against the class' name as a string. What do you think about creating a helper in TestUtils.scala that abstracts asserting the  instanceof of a class and logs a helpful message if the assertion fails?

@stanislavkozlovski: That is a fair point.  It would be good to keep the reference to the class object.  I pushed an update that should do this.

thanks


Create Electron protocol handler to block internet access when offline

As a high-security-needs user, I want Platform.Bible to guarantee that it will never touch the internet so I can be safe while using the product.
As an extension developer, I want Platform.Bible to give me a way to provide saved assets at arbitrary urls when Platform.Bible is offline so I can easily use the same code between some external webapp I made and the webapp adapted to an extension.

Since there are many ways to access the internet from the frontend and we want to be able to police all of them (and since we can't fully enforce a separate CSP in webviews from the renderer overall), let's implement an electron protocol handler that polices every internet request from the frontend.

When online, the protocol handler passes requests through except for blacklisted sites (if they exist)
When offline, the protocol handler does two things:

Asks the extension if it wants to provide a response to the request at the specified url if the extension registered an offline backup function (the extension can register an offline backup function of some sort)
Rejects the request if not registered or if we don't like it for some reason

Created as a result of #176

This will not work naturally if we make a web version, but we can go with option 1 in a web scenario. However, we don't know how we would make an offline backup function work in the online setting with arbitrary urls due to security restrictions on protocol handlers and service workers. Hopefully the number of extensions that really need that as an option will be very minimal.

Note: remember we need to police all endpoints including https and ws


feat(docs) restore enum expansion on docs page

This restores the enum display in the docs lost by #731
To get it to work I'm now parsing the docgen to get the correct values which has it's limitations. docgen is very literal when it comes to programatic propTypes, so I've tried expanding the easy ones but there's a few occurrences that are more difficult to make work (see last screenshot for an example)

Codecov Report

:exclamation: No coverage uploaded for pull request head (enumdocs@3f933a9).

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 672feac...3f933a9. Read the comment docs.

Thanks!


Simplify things when find_script_file() fails

As discussed in #147, the fullname variable is in fact necessary and cannot be replaced by the modified code. Thanks!

Now I understand.  Thanks for the explanation.


KON-230 Rename Suffixes Of Objects In Util Package To Util

This util package also contains the KotlinFileParser object. Should I rename it? Or maybe should I move this to another parser package?


GPU crashes when running Keras/tensorflow-gpu, specifically when clock speed goes to idle at 0 MHz

I'm using Jupyter Notebook to run Keras with a Tensorflow GPU backend. I've done some testing with various dummy models while simultaneously monitoring my GPU usage using MSI Afterburner, GPU-Z, nvidia-smi and Task Manager. My GPU is a GeForce GTX 960M, which has no issues running games. The temperatures are also low when running Keras.
What I've noticed is that the Keras runs fine (e.g. loading or training a model) in the beginning but whenever Keras is not running anything, the GPU naturally wants to idle from 1097 MHz to 0 MHz and as soon as it does that the GPU crashes. I can see that the "GPU is lost" on nvidia-nvsmi. I have to then disable and re-enable my GPU in the Device Manager to get it to work.
Here's a sample script, which I believe was from the official documentation:
import keras
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation
from keras.optimizers import SGD

import numpy as np
x_train = np.random.random((1000, 20))
y_train = keras.utils.to_categorical(np.random.randint(10, size=(1000, 1)), num_classes=50)
x_test = np.random.random((100, 20))
y_test = keras.utils.to_categorical(np.random.randint(10, size=(100, 1)), num_classes=50)

model = Sequential()
# Dense(64) is a fully-connected layer with 64 hidden units.
# in the first layer, you must specify the expected input data shape:
# here, 20-dimensional vectors.
model.add(Dense(200, activation='relu', input_dim=20))
model.add(Dropout(0.5))
model.add(Dense(200, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(50, activation='softmax'))

sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy',
              optimizer=sgd,
              metrics=['accuracy'])

model.fit(x_train, y_train,
          epochs=20,
          batch_size=128)
score = model.evaluate(x_test, y_test, batch_size=128)

The model runs fine and completes the training but after a few seconds my GPU dies. This even happens if I just load a model. I.e. I import keras modules, then use "load_model", and in less than a minute everything crashes as soon as the clock speed drops to 0 MHz.
Does anyone have any idea why this might be happening?

Update: I can temporarily prevent this from happening for very small programs by using the "allow_growth" feature as follows:
import tensorflow as tf
from keras.backend.tensorflow_backend import set_session
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
sess = tf.Session(config=config)
set_session(sess)

However, this only works if the operation is really small that uses only about 0.1 GB of GPU memory such as loading a model or running a really small model. However, if the program is using GPU memory of about 0.3 GB, my graphics card crashes since the memory does not go to 0 GB before the clock speed drops to 0 MHz (lower power state). That is, it takes longer for the 0.3 GB memory to clear compared to 0.1 GB.

I was finally able to figure out the issue thanks to someone from another forum. It was a driver issue. The latest drivers provided by Nvidia are causing the issue unlike the old drivers provided by my laptop manufacturer.
Since I was not able to run tensorflow with my old drivers and do more troubleshooting, what I did was download eDrawings Viewer and open up some random assembly drawings I found online. First I tried with the latest Nvidia drivers, and I see that when I manipulate the models, my card is at P0 state but if I don't do anything and let the software idle, my card goes to a lower power state and crashes my GPU. But when I did the same exercise with my ASUS manufacturer-certified drivers (since this software was compatible even with the older drivers unlike TF), my GPU did NOT crash.
What I also discovered was that eDrawings Viewer does not crash even with the latest Nvidia drivers if I go into the Nvidia Control Panel and select "Prefer Maximum Performance" under Power Management Mode. The card stays at P0 state whenever I have the software open even after idling for minutes. Unfortunately, since python.exe does not have a graphical interface, this option does not work for my case. As a workaround, I can still run tensorflow without getting it to crash by running eDrawings Viewer in the background (or really any program that uses a graphical interface), which keeps my card at the P0 State.

Closing as this is resolved.


Fix typos and prettify markdowns

Found via codespell . and prettier -w .

Thanks!

:partying_face: :partying_face: :partying_face: :partying_face: :partying_face:


HBASE-25656 Backport to branch-2.2: [HBASE-25548 Optionally allow sna…

…pshots to preserve cluster's max filesize config by setting it into table descriptor]

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
2m 34s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s
No case conflicting files found.

+0 :ok:
prototool
0m  0s
prototool was not available.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 2 new or modified test files.

_ branch-2.2 Compile Tests _

+0 :ok:
mvndep
0m 19s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
3m 23s
branch-2.2 passed

+1 :green_heart:
compile
1m 57s
branch-2.2 passed

+1 :green_heart:
checkstyle
4m 38s
branch-2.2 passed

+0 :ok:
refguide
16m 14s
branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.

+1 :green_heart:
shadedjars
4m 17s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
3m 46s
branch-2.2 passed

+0 :ok:
spotbugs
3m  3s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
17m 12s
branch-2.2 passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 16s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
3m  3s
the patch passed

+1 :green_heart:
compile
2m  7s
the patch passed

+1 :green_heart:
cc
2m  7s
the patch passed

+1 :green_heart:
javac
2m  7s
the patch passed

-1 :x:
checkstyle
0m 40s
hbase-client: The patch generated 3 new + 270 unchanged - 0 fixed = 273 total (was 270)

-1 :x:
checkstyle
2m 17s
root: The patch generated 3 new + 298 unchanged - 0 fixed = 301 total (was 298)

-1 :x:
rubocop
0m 10s
The patch generated 3 new + 370 unchanged - 1 fixed = 373 total (was 371)

-1 :x:
whitespace
0m  0s
The patch has 25 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+0 :ok:
refguide
5m 22s
patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.

+1 :green_heart:
shadedjars
3m 55s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
27m 38s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1.

+1 :green_heart:
hbaseprotoc
5m 28s
the patch passed

-1 :x:
javadoc
0m 25s
hbase-client generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)

-1 :x:
javadoc
2m 18s
root generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6)

-1 :x:
findbugs
1m 28s
hbase-client generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

-1 :x:
findbugs
9m 16s
root generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

_ Other Tests _

+1 :green_heart:
unit
138m 52s
root in the patch passed.

+1 :green_heart:
asflicense
2m 40s
The patch does not generate ASF License warnings.

277m 25s

Reason
Tests

FindBugs
module:hbase-client

Useless object stored in variable snapshotProps of method org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.createSnapshotDesc(SnapshotProtos$SnapshotDescription)  At ProtobufUtil.java:snapshotProps of method org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.createSnapshotDesc(SnapshotProtos$SnapshotDescription)  At ProtobufUtil.java:[line 2946]

FindBugs
module:root

Useless object stored in variable snapshotProps of method org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.createSnapshotDesc(SnapshotProtos$SnapshotDescription)  At ProtobufUtil.java:snapshotProps of method org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.createSnapshotDesc(SnapshotProtos$SnapshotDescription)  At ProtobufUtil.java:[line 2946]

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3055

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop refguide

uname
Linux 2a972c14b5e6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-3055/out/precommit/personality/provided.sh

git revision
branch-2.2 / bf6fad3398

Default Java
Oracle Corporation-1.8.0_282-b08

refguide
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/artifact/out/branch-site/book.html

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/artifact/out/diff-checkstyle-hbase-client.txt

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/artifact/out/diff-checkstyle-root.txt

rubocop
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/artifact/out/diff-patch-rubocop.txt

whitespace
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/artifact/out/whitespace-eol.txt

refguide
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/artifact/out/patch-site/book.html

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/artifact/out/diff-javadoc-javadoc-hbase-client.txt

javadoc
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/artifact/out/diff-javadoc-javadoc-root.txt

findbugs
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/artifact/out/new-findbugs-hbase-client.html

findbugs
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/artifact/out/new-findbugs-root.html

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/testReport/

Max. process+thread count
3101 (vs. ulimit of 12500)

modules
C: hbase-protocol-shaded hbase-client hbase-server hbase-shell . U: .

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/1/console

versions
git=2.20.1 maven=3.6.3 findbugs=3.1.11 rubocop=0.81.0

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

LGTM. Its safe to backport because we provide defaults for the API added?

Correct, the parameter is optional, if not specified, we'll not apply it.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 34s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+0 :ok:
prototool
0m  0s
prototool was not available.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 2 new or modified test files.

_ branch-2.2 Compile Tests _

+0 :ok:
mvndep
0m 17s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
3m 13s
branch-2.2 passed

+1 :green_heart:
compile
1m 56s
branch-2.2 passed

+1 :green_heart:
checkstyle
4m 32s
branch-2.2 passed

+0 :ok:
refguide
16m 17s
branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.

+1 :green_heart:
shadedjars
4m  0s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
3m 29s
branch-2.2 passed

+0 :ok:
spotbugs
2m 48s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
15m 21s
branch-2.2 passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 15s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 54s
the patch passed

+1 :green_heart:
compile
1m 55s
the patch passed

+1 :green_heart:
cc
1m 55s
the patch passed

+1 :green_heart:
javac
1m 55s
the patch passed

-1 :x:
checkstyle
0m 43s
hbase-client: The patch generated 1 new + 270 unchanged - 0 fixed = 271 total (was 270)

-1 :x:
checkstyle
2m  8s
root: The patch generated 1 new + 298 unchanged - 0 fixed = 299 total (was 298)

-1 :x:
rubocop
0m 11s
The patch generated 3 new + 370 unchanged - 1 fixed = 373 total (was 371)

-1 :x:
whitespace
0m  0s
The patch has 25 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+0 :ok:
refguide
5m  9s
patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.

+1 :green_heart:
shadedjars
4m  4s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
27m 29s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1.

+1 :green_heart:
hbaseprotoc
5m 27s
the patch passed

+1 :green_heart:
javadoc
3m 48s
the patch passed

+1 :green_heart:
findbugs
16m  7s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
136m 51s
root in the patch passed.

+1 :green_heart:
asflicense
2m 37s
The patch does not generate ASF License warnings.

269m 14s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3055

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop refguide

uname
Linux 9cba0eb71fdb 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-3055/out/precommit/personality/provided.sh

git revision
branch-2.2 / bf6fad3398

Default Java
Oracle Corporation-1.8.0_282-b08

refguide
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/2/artifact/out/branch-site/book.html

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/2/artifact/out/diff-checkstyle-hbase-client.txt

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/2/artifact/out/diff-checkstyle-root.txt

rubocop
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/2/artifact/out/diff-patch-rubocop.txt

whitespace
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/2/artifact/out/whitespace-eol.txt

refguide
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/2/artifact/out/patch-site/book.html

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/2/testReport/

Max. process+thread count
3829 (vs. ulimit of 12500)

modules
C: hbase-protocol-shaded hbase-client hbase-server hbase-shell . U: .

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/2/console

versions
git=2.20.1 maven=3.6.3 findbugs=3.1.11 rubocop=0.81.0

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 33s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+0 :ok:
prototool
0m  0s
prototool was not available.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 2 new or modified test files.

_ branch-2.2 Compile Tests _

+0 :ok:
mvndep
0m 17s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
3m 11s
branch-2.2 passed

+1 :green_heart:
compile
1m 56s
branch-2.2 passed

+1 :green_heart:
checkstyle
4m 26s
branch-2.2 passed

+0 :ok:
refguide
16m 19s
branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.

+1 :green_heart:
shadedjars
4m  2s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
3m 31s
branch-2.2 passed

+0 :ok:
spotbugs
2m 38s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
15m  6s
branch-2.2 passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 16s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 52s
the patch passed

+1 :green_heart:
compile
1m 57s
the patch passed

+1 :green_heart:
cc
1m 57s
the patch passed

+1 :green_heart:
javac
1m 57s
the patch passed

-1 :x:
checkstyle
0m 44s
hbase-client: The patch generated 1 new + 270 unchanged - 0 fixed = 271 total (was 270)

-1 :x:
checkstyle
2m 10s
root: The patch generated 1 new + 298 unchanged - 0 fixed = 299 total (was 298)

-1 :x:
rubocop
0m 10s
The patch generated 3 new + 370 unchanged - 1 fixed = 373 total (was 371)

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+0 :ok:
refguide
5m 17s
patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.

+1 :green_heart:
shadedjars
4m  0s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
25m 42s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2 2.10.0 or 3.1.2 3.2.1.

+1 :green_heart:
hbaseprotoc
5m 20s
the patch passed

+1 :green_heart:
javadoc
3m 42s
the patch passed

+1 :green_heart:
findbugs
18m 13s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
137m 53s
root in the patch passed.

+1 :green_heart:
asflicense
2m 34s
The patch does not generate ASF License warnings.

269m  9s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3055

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop refguide

uname
Linux e39fdb1ecec1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-3055/out/precommit/personality/provided.sh

git revision
branch-2.2 / bf6fad3398

Default Java
Oracle Corporation-1.8.0_282-b08

refguide
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/3/artifact/out/branch-site/book.html

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/3/artifact/out/diff-checkstyle-hbase-client.txt

checkstyle
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/3/artifact/out/diff-checkstyle-root.txt

rubocop
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/3/artifact/out/diff-patch-rubocop.txt

refguide
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/3/artifact/out/patch-site/book.html

Test Results
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/3/testReport/

Max. process+thread count
3938 (vs. ulimit of 12500)

modules
C: hbase-protocol-shaded hbase-client hbase-server hbase-shell . U: .

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3055/3/console

versions
git=2.20.1 maven=3.6.3 findbugs=3.1.11 rubocop=0.81.0

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.


Missing Brazil scene

I started looking at another area with stackstac, and investigating
got a 404 from rasterio on this one
File "/home/ubuntu/anaconda3/envs/richard/lib/python3.8/site-packages/stackstac/rio_reader.py", line 334, in _open
raise RuntimeError(f"retrieval {self.url!r} problem" )
RuntimeError: retrieval 'https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/22M/DU2020/8/S2A_22MDU_20200816_0_L2A/B02.tif' problem
The same thing happened for band 3, so I presume this one is missing?

Another couple
S2A_22MGT_20170610_0_L2A
S2A_22MET_20200816_0_L2A

S2A_22MES_20200816_0_L2A
S2A_22MDB_20200816_0_L2A

S2A_22MFB_20200816_0_L2A

S2A_22MEB_20200816_0_L2A
Seeming like a consistent batch of 08016 missing in this bit of Brazil?

S2A_22MFA_20200816_0_L2A

S2A_22MCB_20200816_0_L2A

S2A_22MEA_20200816_0_L2A

couple more
S2A_22MEV_20200816_0_L2A
S2A_22MCV_20200816_0_L2A

Some more
S2A_22MBU_20200816_0_L2A
S2A_22MCU_20200816_0_L2A
S2A_22MDU_20200816_0_L2A
S2A_22MEU_20200816_0_L2A
S2A_22MCT_20200816_0_L2A
S2A_22MET_20200816_0_L2A
S2A_22MBS_20200816_0_L2A
S2A_22MCT_20200816_0_L2A
S2A_22MCS_20200816_0_L2A
S2A_22MBS_20200816_0_L2A
S2A_22MDS_20200816_0_L2A
S2A_22MET_20200816_0_L2A
So maybe 22Ms on 0816

S2B_23MRP_20200809_0_L2A
S2B_23NLA_20200729_0_L2A
S2A_23MRT_20200807_0_L2A
S2B_23MRM_20200809_0_L2A
S2A_21LZL_20200816_0_L2A
S2A_23NLA_20200803_0_L2A
S2B_23MRN_20200809_0_L2A
S2A_22LBR_20200816_0_L2A
S2B_23MRP_20200730_0_L2A
S2B_23NKA_20200729_0_L2A
S2B_23MRM_20200730_0_L2A
S2A_23MRR_20200804_0_L2A
S2A_21LZK_20200816_0_L2A
S2A_23MRQ_20200804_0_L2A
S2A_23MRS_20200728_0_L2A
S2A_21MZM_20200816_0_L2A
S2A_23MRU_20200807_0_L2A
S2A_23MRS_20200807_0_L2A
S2B_23NLA_20200808_0_L2A
S2B_23MRN_20200730_0_L2A

This has been resolved in the v1 api.


Uncaught (in promise) TypeError: res.json is not a function

"vue-infinite-loading": "^2.2.0",
"vue": "^2.4.2",
Hello,
I used the "Use with filter" example in docs, while the console throw a mistake that ：Uncaught (in promise) TypeError: res.json is not a function
Would you please help with this issue? thanks!
Below is my code:
infiniteHandler($state) {
axios.get(baseUrl + '/car/getCarList', {
params: {
pageNum: 1,
pageSize: 100,
cityId: getCookie('cityId'),
orderflag: "default",
orderType: "asc",
car_brand_id:'',
recharge_mileage:'',
price:'',
car_level:'',
page: this.carlist.length / 20 + 1,
},
}).then(res => res.json()).then((data) => {
if (data.hits.length) {
this.carlist = this.carlist.concat(data.hits);
$state.loaded();
if (this.carlist.length / 20 === 10) {
$state.complete();
}
} else {
$state.complete();
}
});
},
changeFilter() {
this.carlist = [];
this.$nextTick(() => {
this.$refs.infiniteLoading.$emit('$InfiniteLoading:reset');
});
},
},

Sorry, it's my mistake. The res.json() method is the way to transform response data to JSON of the vue-resource, but axios support automatic transform so we don't need that.
You can checkout the new document.


[chapter 11.5.2] I am uncertain about the complete overview of the entire workflow.

I only have a vague understanding of what is discussed in the book, and I lack a clear panoramic view of the entire authentication process. Therefore, I would like to present my current understanding here and kindly ask for any corrections if there are any. (As a beginner, I genuinely feel that this section should include a diagram similar to Figure 11.3).
workflow steps(I guessed):

The first request sent by Angular is an HTTP standard request. (This will trigger the authentication flow)
Subsequently, all requests sent are AJAX requests, which are considered unauthenticated requests.
Then, we instruct Spring Security to respond with an HTTP 401 response.
After receiving the HTTP 401 response, the SPA handles this error and initiates the authentication flow. (In other words, initiate authentication flow is required for every AJAX request).
Since the SPA is responsible for the authentication flow, it is necessary to allow unauthorized requests for static resources at this point.

@ongiant I'll consider adding a dedicated diagram in the next edition of the book, it's a really good suggestion, thank you!
The steps you described look correct. I have only one comment about step 4: after the authentication flow, AJAX requests will be allowed through since they will contain the session cookie that Spring Security to validate the request comes from an authenticated session. That means it's not necessary to initiate the authentication flow for every AJAX request. Only the first time when the first AJAX request gets an HTTP 401 response from Edge Service.
If you have more questions about this flow, feel free to comment here or open a new issue.


browserslist-ga output does not sum to 100%

I'm not sure if this is a bug but it seems wrong that the output of browserslist-ga does not sum to 100%.
This is then confusing when you run a browserslist query with the using the file with the --stats option
Please find for example the output I had for my website, which sums to ~48.5%
browserslist-stats.zip
To get those results, I enabled the analytics api for my project and created an oauth client id, then configured my terminal with it  and ran npx browserslist-ga

Your total ios_saf usage is 0%, which might be related to #37


Range and location not showing.

Hi Dielee, again thank you for your great work on this!
Is there a trick to get it to show range?
Mine is functioning great, but doesn't pick up range or location like your examples.
Is there something else I need to do in the Volvo Developer portal?

Note: Car is a 2023 XC40 Recharge Twin

Please provide some debug logs. Pretty sure this is an Volvo API issue

Sure, nothing obvious in the logs since it is operating well.
Just not collecting the range and location data.
On the Volvo developer site I am using an API application I made.  Do you think I need to "publish" this application to enable location and range data?
Jul 11 15:30:09 volvo2mqtt [106] - INFO: Mqtt update done. Next run in 300 seconds.
INFO:root:Sending mqtt update...
Jul 11 15:35:09 volvo2mqtt [106] - INFO: Sending mqtt update...
INFO:root:Mqtt update done. Next run in 300 seconds.
Jul 11 15:35:11 volvo2mqtt [106] - INFO: Mqtt update done. Next run in 300 seconds.
INFO:root:Sending mqtt update...
Jul 11 15:40:11 volvo2mqtt [106] - INFO: Sending mqtt update...
INFO:root:Mqtt update done. Next run in 300 seconds.
Jul 11 15:40:14 volvo2mqtt [106] - INFO: Mqtt update done. Next run in 300 seconds.
INFO:root:Sending mqtt update...
Jul 11 15:45:14 volvo2mqtt [106] - INFO: Sending mqtt update...
INFO:root:Mqtt update done. Next run in 300 seconds.
Jul 11 15:45:17 volvo2mqtt [106] - INFO: Mqtt update done. Next run in 300 seconds.
INFO:root:Sending mqtt update...
Jul 11 15:50:17 volvo2mqtt [106] - INFO: Sending mqtt update...
INFO:root:Refreshing credentials
Jul 11 15:50:17 volvo2mqtt [106] - INFO: Refreshing credentials
INFO:root:Mqtt update done. Next run in 300 seconds.
Jul 11 15:50:20 volvo2mqtt [106] - INFO: Mqtt update done. Next run in 300 seconds.
INFO:root:Sending mqtt update...
Jul 11 15:55:20 volvo2mqtt [106] - INFO: Sending mqtt update...
INFO:root:Mqtt update done. Next run in 300 seconds.
Jul 11 15:55:23 volvo2mqtt [106] - INFO: Mqtt update done. Next run in 300 seconds.
INFO:root:Sending mqtt update...
Jul 11 16:00:23 volvo2mqtt [106] - INFO: Sending mqtt update...
INFO:root:Mqtt update done. Next run in 300 seconds.
Jul 11 16:00:25 volvo2mqtt [106] - INFO: Mqtt update done. Next run in 300 seconds.
INFO:root:Sending mqtt update...
Jul 11 16:05:25 volvo2mqtt [106] - INFO: Sending mqtt update...
INFO:root:Mqtt update done. Next run in 300 seconds.
Jul 11 16:05:28 volvo2mqtt [106] - INFO: Mqtt update done. Next run in 300 seconds.
INFO:root:Sending mqtt update...
Jul 11 16:10:28 volvo2mqtt [106] - INFO: Sending mqtt update...
INFO:root:Mqtt update done. Next run in 300 seconds.
Jul 11 16:10:32 volvo2mqtt [106] - INFO: Mqtt update done. Next run in 300 seconds.
Jul 11 16:15:32 volvo2mqtt [106] - INFO: Sending mqtt update...
INFO:root:Sending mqtt update...
INFO:root:Mqtt update done. Next run in 300 seconds.
Jul 11 16:15:34 volvo2mqtt [106] - INFO: Mqtt update done. Next run in 300 seconds.

This is not a debug log. Please enable debug and send the log as a file.
I can't do anything if I can't see what's going on there.

Sorry, I still can't find the actual log file. But I have enabled debug mode and this is the new log:
  "timestamp" : "2023-07-11T07:54:18Z"
},
"frontRightTyrePressure" : {
  "name" : "frontRightTyrePressure",
  "value" : "NORMAL",
  "timestamp" : "2023-07-11T07:54:18Z"
},
"frontLeftTyrePressure" : {
  "name" : "frontLeftTyrePressure",
  "value" : "NORMAL",
  "timestamp" : "2023-07-11T07:54:18Z"
}

DEBUG:root:Response status code: 200
}
}
DEBUG:root:{
"status" : 200,
"operationId" : "20adac0a-8a9e-47a3-a09d-6d0d454e2795",
"data" : {
"rearLeftTyrePressure" : {
"name" : "rearLeftTyrePressure",
"value" : "NORMAL",
"timestamp" : "2023-07-11T07:54:18Z"
},
"rearRightTyrePressure" : {
"name" : "rearRightTyrePressure",
"value" : "NORMAL",
"timestamp" : "2023-07-11T07:54:18Z"
},
"frontRightTyrePressure" : {
"name" : "frontRightTyrePressure",
"value" : "NORMAL",
"timestamp" : "2023-07-11T07:54:18Z"
},
"frontLeftTyrePressure" : {
"name" : "frontLeftTyrePressure",
"value" : "NORMAL",
"timestamp" : "2023-07-11T07:54:18Z"
}
}
}
DEBUG:root:Starting GET call against https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/engine-status
Jul 11 19:48:06 volvo2mqtt [106] - DEBUG: Response status code: 200
Jul 11 19:48:06 volvo2mqtt [106] - DEBUG: {
"status" : 200,
"operationId" : "20adac0a-8a9e-47a3-a09d-6d0d454e2795",
"data" : {
"rearLeftTyrePressure" : {
"name" : "rearLeftTyrePressure",
"value" : "NORMAL",
"timestamp" : "2023-07-11T07:54:18Z"
},
"rearRightTyrePressure" : {
"name" : "rearRightTyrePressure",
"value" : "NORMAL",
"timestamp" : "2023-07-11T07:54:18Z"
},
"frontRightTyrePressure" : {
"name" : "frontRightTyrePressure",
"value" : "NORMAL",
"timestamp" : "2023-07-11T07:54:18Z"
},
"frontLeftTyrePressure" : {
"name" : "frontLeftTyrePressure",
"value" : "NORMAL",
"timestamp" : "2023-07-11T07:54:18Z"
}
}
}
Jul 11 19:48:06 volvo2mqtt [106] - DEBUG: Starting GET call against https://api.volvocars.com/connected-vehicle/v2/vehicles/{0}/engine-status
Jul 11 19:48:07 volvo2mqtt [106] - DEBUG: https://api.volvocars.com:443 "GET /connected-vehicle/v2/vehicles/LYVXZEDA6PL009989/engine-status HTTP/1.1" 200 226
DEBUG:urllib3.connectionpool:https://api.volvocars.com:443 "GET /connected-vehicle/v2/vehicles/LYVXZEDA6PL009989/engine-status HTTP/1.1" 200 226
Jul 11 19:48:07 volvo2mqtt [106] - DEBUG: Response status code: 200
DEBUG:root:Response status code: 200
DEBUG:root:{
"status" : 200,
"operationId" : "bd90b28e-eafb-4dc5-a668-c6ceb02c6dea",
"data" : {
"engineRunning" : {
Jul 11 19:48:07 volvo2mqtt [106] - DEBUG: {
"status" : 200,
"operationId" : "bd90b28e-eafb-4dc5-a668-c6ceb02c6dea",
"data" : {
"engineRunning" : {
"name" : "engineRunning",
"value" : "false",
"name" : "engineRunning",
"value" : "false",
"timestamp" : "1970-01-01T00:00:00Z"
}
}
"timestamp" : "1970-01-01T00:00:00Z"
}
}
}
}
DEBUG:root:Updating icon to engine-off-outline for engine_state
Jul 11 19:48:07 volvo2mqtt [106] - DEBUG: Updating icon to engine-off-outline for engine_state
INFO:root:Mqtt update done. Next run in 300 seconds.
Jul 11 19:48:07 volvo2mqtt [106] - INFO: Mqtt update done. Next run in 300 seconds.

As said in the issue template, you can find your full log here \\<Your HA IP>\addons\volvo2mqtt\log\volvo2mqtt.log. I need the debug log from the addon start, after restarting the addon.

I would share the full logs if I could find them.
I have tried to find that location in my HA file editor and just navigating in a web browser (with my HA ip first) and it just says 404: not found.
Any hints with finding the log?
(If not you can just close this issue)

You have to install and configure the smb share addon

You have to install and configure the smb share addon
Sorry, this is the first time I've heard about this to find the Add On folder.

Log file attached. Thank you for your help.
volvo2mqtt.log

Thanks for the log. Unfortunately I can't do anything with the log, because you don't restarted the addon. As said above, I need the startup log, the first logs after the addon started. Stop the addon, delete the log file. Then start the addon and wait about 30 secs and send me the log.
AND, I removed your file because it contains your car vin. Please remove all sensitive data from the file before uploading.

Sorry again, full log after restart here.
I appreciate you removing the other log, my apologies.
Log.txt

Jep, the whole energy API isn't supported by your car. Sorry, there is nothing I can do for you.

No problems, thanks for trying!


Failed to resolve address (https://github.com/rust-lang/crates.io-index)

Cargo is unable to connect to github on OS X (installed via nix, not using a proxy):
$ cargo build --verbose
    Updating registry `https://github.com/rust-lang/crates.io-index`
failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  [12/-1] Failed to resolve address for https: nodename nor servname provided, or not known
$ ping github.com
PING github.com (<IP_ADDRESS>): 56 data bytes
64 bytes from <IP_ADDRESS>: icmp_seq=0 ttl=45 time=117.037 ms
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=45 time=116.741 ms
^C
--- github.com ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 116.741/116.889/117.037/0.148 ms

I have a Ubuntu VM on the same machine where Cargo can connect just fine (also installed via nix).

Looks like a bug in URL parsing somewhere along the line? It looks like something is trying to resolve the hostname https:. Did you perhaps use an external libgit2 build rather than the bundled one?

I'm completely new to rust/cargo. What I did was install it via nix-env -i rustc cargo on OS X (where it fails) and Ubuntu (where it works).
Is this a problem of the nix distribution?

Maybe? I've also no idea how Cargo-in-nix works, so it could also be totally unrelated. Unfortunately without a way to reproduce locally this may be very difficult to debug.

$ cargo build --verbose
could not find `Cargo.toml` in `/home/obadz` or any parent directory

What's the simplest thing I could check out to reproduce?

@obadz  try:
$ git clone https://github.com/serde-rs/json
$ cd json/json
$ cargo build --verbose

Could problems arise from using different Cargo and rustc versions? Nix is using Cargo 0.9.0 with rust 1.8.0?

The official dist (where it works) uses
$ cargo -V
cargo 0.9.0-nightly (8fc3fd8 2016-02-29)
$ rustc -V
rustc 1.8.0 (db2939409 2016-04-11)

saw the same on suse

Not using nix anymore.


add check for uuid-runtime package on OctoPi

My install was missing the uuidgen command, needed by the script.  After it failed repeatedly trying to run uuidgen, I installed uuid-runtime and the new instance was created properly.
Please add a check for uuid-runtime package (uuidgen) on OctoPi
Thanks!!

It does attempt to install uuid-runtime. It is not clear why it fails, but it only fails with OctoPi.


Don't send  elements in calendar-multiget

Don't send  elements in calendar-multiget
This is not allowed by the CalDAV spec, see https://www.rfc-editor.org/rfc/rfc4791#page-63
Fixes nibdo/bloben-app#85

Thanks for the PR, I will take a look later this week.

@natelindev ping, in case this has slipped off your radar


Remove emitSwiftModuleSeparately option

This option has always been off by default and there are no particular tests for it, so I think we should just remove it since it makes maintaining SwiftTargetBuildDescription unnecessarily difficult. From what I can gather, this was supposed to be an optimization for build times that maybe was supposed to be tested, but it ended up never being enabled.

@swift-ci please smoke test

This will also resolve https://github.com/apple/swift-package-manager/issues/4549 and https://github.com/apple/swift-package-manager/issues/6577

I'm also pretty sure this option hasn't fully worked in a while, there are all these odds differences between this mode and the "regular" one when looking at SwiftTargetBuildDescription.

seems reasonable. can git history tell us when/who added it originally? maybe worth checking in and confirming the assumptions

seems reasonable. can git history tell us when/who added it originally? maybe worth checking in and confirming the assumptions

Yes, it was added in https://github.com/apple/swift-package-manager/pull/2391

For reference, I spoke to Doug and the conclusion was that not only this but also the current default mode should eventually go away and we should use the integrated driver. The refactoring that this is a part of should help us to eventually get there.


JHU Docs not listing all available geographic levels

The “available for” section here might need to be updated with ‘nation’ ‘hhs’

Confirmed list of geo levels available for JHU:
https://github.com/cmu-delphi/covidcast-indicators/blob/f50278bba514cfbe123ebbc97038df07d0d6d567/jhu/delphi_jhu/geo.py#L6


Another crash when using a prepared statement from a subclass

I tested the just-released x64 driver on PHP 7.0.4, and while I didn't experience a crash when running the exact script in issue #74, I found that tweaking the script slightly still causes the driver to crash. All I changed was to store the connection resource in a class and pass a parameter to the Statement parent constructor.
Before running the script, create a Users table like this:
CREATE TABLE Users (
    user_id int IDENTITY PRIMARY KEY,
    name varchar(50) NOT NULL
);

INSERT INTO Users (name)
VALUES ('Martin'), ('Emmett');

Then try running this:
$connection = sqlsrv_connect('Computer-Name\SQLEXPRESS', ['Database' => 'MyDatabase']);
$db = new SQL($connection);

$sql = "UPDATE Users SET name = ? WHERE user_id = ?";
$id = $name = null;
$stmt = $db->prepare($sql, [&$name, &$id]);

$names = [
    1 => 'Michael J. Fox',
    2 => 'Christopher A. Lloyd',
];

foreach ($names as $id => $name) {
    $stmt->execute();
}

$stmt->close();

class SQL
{
    private $connection;

    public function __construct($connection)
    {
        $this->connection = $connection;
    }

    public function prepare($sql, array $params)
    {
        if (!$stmt = sqlsrv_prepare($this->connection, $sql, $params)) {
            throw new Exception('Failed to prepare query: ' . print_r(sqlsrv_errors(), true));
        }

        return new Statement($stmt, true);
    }
}

class BaseStatement
{
    protected $usedPrepare;

    public function __construct($usedPrepare)
    {
        $this->usedPrepare = $usedPrepare;
    }
}

class Statement extends BaseStatement
{
    private $stmt;

    public function __construct($stmt, $usedPrepare)
    {
        parent::__construct($usedPrepare); // crash occurs unless I comment out this line
        $this->stmt = $stmt;
    }

    public function execute()
    {
        sqlsrv_execute($this->stmt);
    }

    public function close()
    {
        sqlsrv_free_stmt($this->stmt);
    }
}

The script crashes unless I comment out the parent constructor call.

Thanks for the quick feedback. We will look into this and share our findings with you. Did this release help solve #74?

Yes, the specific script in #74 is no longer crashing for me.

@theodorejb we have attempted to fix this issue with the v4.0.3 release, can you help us validate?

:+1: This does appear to be fixed now. Thanks!


typescript: handle import statements, resolving across modules [phab:D1402]

Created by evmar at 2017-03-02 02:20:02:
When you import from another module, all of the various ways to
name that import should all resolve to the same thing.

salguarnieri commented inline (context lost during migration) at 2017-03-02 17:18:25:
Does the first . need to be escaped? --> /.(d.)?ts$/
Also, does case matter? foo.D.TS seems... ugly but I don't know if it is used or not.

salguarnieri commented inline (context lost during migration) at 2017-03-02 17:18:25:
Can you reword the second sentence of the comment. For some reason, I'm having a hard time reading it.

evmar wrote at 2017-03-02 18:46:28:
review comments

evmar wrote at 2017-03-02 18:52:03:
add a dumb null check

evmar commented inline (context lost during migration) at 2017-03-02 18:52:45:
Yep, good catch on the regex!  I think lower case is sufficient for now; I've never seen the extension capitalized.

evmar commented inline (context lost during migration) at 2017-03-02 18:52:45:
I have reorganized this code, centralizing the approach taken first and using consistent names.  PTAL and let me know I need to clarify further.

evmar wrote at 2017-03-02 18:52:45:
PTAL

salguarnieri commented inline (context lost during migration) at 2017-03-02 23:33:37:
Thanks!

salguarnieri accepted this revision at 2017-03-02 23:33:37

salguarnieri closed via commit 40b225b7eed833d6c7aeb6c041291fd0fac4a4cc at 2017-03-02 23:34:45


Added Permission client, PermissionScheme and Role Client

Changelog:
Clients added:

PermissionsClient
PermissionSchemeClient
RoleClient

Clients updated:

GroupClient
IssueTypeClient

They cover all the stable REST API endpoints listed below:
Permissions

GET /rest/api/2/permissions

PermissionScheme

GET /rest/api/2/permissionscheme
POST /rest/api/2/permissionscheme
GET /rest/api/2/permissionscheme/{schemeId}
PUT /rest/api/2/permissionscheme/{schemeId}
DELETE /rest/api/2/permissionscheme/{schemeId}
GET /rest/api/2/permissionscheme/{schemeId}/permission
POST /rest/api/2/permissionscheme/{schemeId}/permission
GET /rest/api/2/permissionscheme/{schemeId}/permission/{permissionId}
DELETE /rest/api/2/permissionscheme/{schemeId}/permission/{permissionId}

Roles a.k.a ProjectRoles

GET /rest/api/2/role
POST /rest/api/2/role
GET /rest/api/2/role/{id}
PUT /rest/api/2/role/{id}
DELETE /rest/api/2/role/{id}
GET /rest/api/2/role/{id}/actors
POST /rest/api/2/role/{id}/actors
DELETE /rest/api/2/role/{id}/actors

IssueType (Newly added)

POST /rest/api/2/issuetype
DELETE /rest/api/2/issuetype/{issueId}
PUT /rest/api/2/issuetype/{issueId}
GET /rest/api/2/issuetype/{issueId}/alternatives

Group (Newly added)

GET /rest/api/2/group/member

Reference: https://developer.atlassian.com/cloud/jira/platform/rest/

Could you please update the "Supported API Calls" section of the README to reflect the new features you've added?

@kylefarris Updated README


chore(CI): update scripts & use lerna rc.5

[ ] rebuild without cache

Deploy preview ready!
Built with commit a3070c5c61a2cca732665856e404f088d3b2aaa2
https://deploy-preview-321--mcs-lite-ui.netlify.com

Deploy preview ready!
Built with commit a3070c5c61a2cca732665856e404f088d3b2aaa2
https://deploy-preview-321--mcs-lite-mobile-web.netlify.com

Deploy preview ready!
Built with commit a3070c5c61a2cca732665856e404f088d3b2aaa2
https://deploy-preview-321--mcs-lite-introduction.netlify.com


Updating to Foundry 9.251 breaks ability to drag macros from actor to token hotbar

Updating to Foundry 9.251 breaks ability to drag macros to token hotbar.
I have found that pulling from the Macros Directory works, but directly from the actor character sheet to the token hotbar does not.  I can drag to Norc custom hotbar when both are enabled.  Tested in dcc, ose and dnd5e.

Resolved via Foundry build 9.254
see https://foundryvtt.com/releases/9.254


Code Refactoring for TensorFlow 2

Hi,
I did converted Final.ipynb to support TensorFlow2. This PR is attached the conda environment Im using too. Please let me know if this PR have any problems.
Thanks!

tf2 patch merged

Thanks!
Next step I'll train back the both pre-build model. Once done, I'll make another PR.

Awesome! Thanks
On Wed, 21 Jul 2021 at 13:59, Han Sheng @.***> wrote:

Thanks!
Next step I'll train back the both pre-build model. Once done, I'll make
another PR.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/arpan65/Scanned-document-classification-using-deep-learning/pull/6#issuecomment-883997761,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGBURMVI5GZHBA7WTSJI353TY2APDANCNFSM5AXOO5HA
.

--
Regards,
Arpan
Cell: +91-9733617260
Web: https://arpan-das.jimdosite.com/


Did you make it?

I think the source code was stolen.
https://play.google.com/store/apps/details?id=tk.henriqueormonde.CryptoCurr

@TypeNine , thanks for the feedback.

poloLender Pro app is open source under MIT license, hence it can be copied, modified and distributed.

There are a number of Poloniex lending apps on the internet. poloLender Pro has unique features, which make it more profitable than other bots

From the app description and available parameters on the link you mentioned, it can be noticed that lending engine is weak and can reach high profitability


Update machine-learning-extension.md

Updated installation instructions for users encountering TLS/SSL error.

@halerankin : Thanks for your contribution! The author(s) have been notified to review your proposed change.


NullPointerException while caching collection - Rset

Expected behavior
No errors or Null pointer exception when Caching RSet. Collection of class-objects are saved to cache.
Actual behavior
Inconsistently throws NullPointer exception.
java.lang.NullPointerException: null 	at o.redisson.command.RedisExecutor.handleError(RedisExecutor.java:602) 	at o.redisson.command.RedisExecutor.execute(RedisExecutor.java:202) 	at o.r.command.CommandAsyncService.async(CommandAsyncService.java:564) 	at o.r.command.CommandAsyncService.writeAsync(CommandAsyncService.java:522) 	at org.redisson.RedissonSet.addAllAsync(RedissonSet.java:317) 	at org.redisson.RedissonSet.addAll(RedissonSet.java:305)
Steps to reproduce or test case
val cacheRSet = redissonClient.getSet(key, TypedJsonJacksonCodec(CacheRecord::class.java, objectMapper))
cacheRSet.addAll(listOf(Object))
cacheRSet.expire(cacheTtl)
Redis version
ElastiCache engine version 7.1.0
Redisson version
3.26.0
Redisson configuration
useClusterServers() timeout = 30000 isCheckLockSyncedSlaves = false

Unable to reproduce. Below is the test
        RSet<String> cacheRSet = redisson.getSet("key", new TypedJsonJacksonCodec(String.class, new ObjectMapper()));
        cacheRSet.addAll(Arrays.asList("1", "2", "3"));
        cacheRSet.expire(Duration.ofSeconds(1));

Is there any update?

@mrniko the error occurs Inconsistently. Would be difficult to reproduce.
Just an observation, the data type used was a java class vs the string used in the above example. Not sure if it has any impact on the way the redisson client reads the data and throws null pointer ?

Fixed in https://github.com/redisson/redisson/issues/5641


how to get the final test accuracy

I've been reviewing the code, and I noticed that the data is split into train_d and test_d
train_d, test_d = split_dataset(data_root, fam_path, which_k)
I'm curious about which specific dataset was used to calculate the accuracy reported in the paper.
Could you clarify whether the accuracy mentioned in the paper is based on train_d, test_d, or some other evaluation set?
Thanks.

Thank you for your attention to our work. In calculating the evaluation metrics, we employ the standard k-fold cross-validation procedure: the original dataset is divided into k subsets, with each subset serving as the test set for the calculation of relevant metrics, while the remaining k-1 subsets are used as the training set for model training. The final evaluation metrics are obtained by averaging the results across all folds.

HI
May I ask which part of code are calculating the final evaluation metrics?
Thanks.

In main_rice3k.py:
def test_one_epoch(i_epoch, loader, model, criterion, args):
    print(f"test epoch [{i_epoch}/{args.epoch}]")

    correct_results = {}
    total_results = {}
    total_loss = 0.0

    model.eval()
    for step, data in enumerate(loader):
        x = data["input"]
        label = data["label"]

        if args.cuda:
            x = x.cuda()
            label = label.cuda()

        with torch.no_grad():
            output = model(x)
            loss = criterion(output.squeeze(-1), label)
            total_loss += loss.detach().cpu().item()

            predicted_labels = torch.argmax(output, dim=1)
            correct_predictions = (predicted_labels == label).float()
            for class_label in torch.unique(label):
                class_mask = (label == class_label).float()
                class_correct = (correct_predictions * class_mask).sum().item()
                class_total = class_mask.sum().item()

                if class_label.item() not in correct_results:
                    correct_results[class_label.item()] = 0
                if class_label.item() not in total_results:
                    total_results[class_label.item()] = 0

                correct_results[class_label.item()] += class_correct
                total_results[class_label.item()] += class_total

    total_loss /= len(loader)

    total_acc = sum(correct_results.values()) / sum(total_results.values())
    # for key in total_results.keys():
    #     print("class {} accuracy: {:.2f}".format(key, correct_results[key] / total_results[key]))
    #     logger.info("class {} accuracy: {:.2f}".format(key, correct_results[key] / total_results[key]))

    print("total accuracy: {:.2f}".format(total_acc))
    metric_value = total_acc

    print(f"test epoch[{i_epoch}/{args.epoch}] test loss:{total_loss}")

    return {"loss": total_loss, "accuracy": metric_value}

You can see the final evaluation metrics.

Thank you for your reply, I got the information I needed

Thank you for your response, it provided the information I needed.
However, I have a few more questions. Since we are using the test dataset as the final evaluation metric, how can we best avoid overfitting?
During training with the Rice3K model, I achieved an accuracy of 0.6 at epoch 39, but the final accuracy dropped to 0.48. Any advice on this discrepancy would be greatly appreciated.
thanks

Well, overfitting is the most critical issue in these two datasets. Both our approaches are aimed at addressing this issue. This is also an open question for you to explore. A kind piece of advice is to reduce the feature dimensionality.

Thank you for your reply and advice. Have a nice day.


Cannot import the module with require('rdkit')

I wanted to test the library (would be really cool to have RDKit in the browser!), but unfortunately was unable to get it to work. I am using what I assume is the latest version (0.5.1) from npm.
When I execute the following code:
const RDKit = require('rdkit'); // using 'rdkit' because 'RDKit' from the README did not work for me

I am getting the following error:
TypeError: rdkit.wast is not a valid URL.

Callstack:
locateFile rdkit.js:21
    integrateWasmJS rdkit.js:63
    rdk rdkit.js:63
    js rdkit.js:77
    __WEBPACK_AMD_DEFINE_RESULT__ rdkit.js:52
    js rdkit.js:53
    js rdkit.js:59
    js 1.chunk.js:289967
    Webpack 28

Relevant code bit:
  18 | var path = getCurrentPathBrowser();
  19 | var Module = {
  20 |   locateFile(url) {
> 21 |     return new URL(url, path).href;
     | ^  22 |   },
  23 |   onRuntimeInitialized() {
  24 |     resolve();

Any idea? Thanks!

Hi @martin-sicho : we aren't really supporting this any more (and should indicate that in the documentation).
There's a new JS wrapper for using the RDKit in the browser that's part of the core RDKit. There's a description with some background info here:
http://rdkit.blogspot.com/2019/11/introducing-new-rdkit-javascript.html

please use the more supported version https://github.com/rdkit/rdkit/tree/master/Code/MinimalLib

@greglandrum Maybe it worth to update npm package to currect version or create new npm package?

We should probably delete the npm package.
I'd be happy to make uploading the new package to npm part of the standard rdkit release cycle if

it's ok to upload large wasm files to npm
someone explains to me how to do it
Note that I have limited time to invest in figuring this stuff out.


Application gets stuck when Feign client is used with virtual threads

Steps to reproduce:

Checkout https://github.com/stsypanov/concurrency-demo
Run DependencyApplication and then ConcurrencyDemoApplication
When both apps are up run StuckApplicationTest
It will take about 1-2 minutes for the test to complete
Now go to demo-service/application.yml and set spring.threads.virtual.enabled: true (by default it's false).
Restart ConcurrencyDemoApplication
Run StuckApplicationTest again

If you now attach a profile (e.g. YourKit) you'll see there's potential deadlock with this stacktrace
Potential deadlock: frozen threads found

It seems that the following threads have not changed their stack for more than 10 seconds.
 These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks. Calls to execute will reuse previously constructed threads if available. If no existing thread is available, a new thread will be created and added to the pool. Threads that have not been used for sixty seconds are terminated and removed from the cache. Thus, a pool that remains idle for long enough will not consume any resources.

Thanks, @stsypanov. Makes sense.

@OlgaMaciaszek hi, it turned out, that the root cause is a JVM issue related virtual threads https://stackoverflow.com/questions/78790376/spring-boot-application-gets-stuck-when-virtual-threads-are-used-on-java-21/78805574#78805574
However, I still think that migration to Executors.newCachedThreadPool() is still reasonable

As soon as it's not either Feign, or spring-cloud issue I close this.

Thanks a lot @stsypanov. Will take a look at your PR today.


Color picker replace preset colors with chosen colors once picked

Right now the preset colors are shown by default

It would be great if:

The preset transparent color is always visible and in the same position.
The rest of colors are being replaced as soon as the user picks color (first color picked, replace the index[1] of presets and so on.
We could use a circular approach, once we reach length index -1 start with position one.

We could add it to the Canvas context, even serialize it once we save the doc (this could be done in a separate stage).

I created a first draft of this feature. Now we can save custom colors. Currently there is no limit on how many palettes we can save, however the color picker doesn't have a shared state so custom colors will not be shared between all the color pickers.

Good stuff !!!
Let's first add a limit, something like 20 colors, (then once you fill all the gaps, then you start in the zero index position, overwrite).
On the other hand let's, we should have a context with the list of shared colors.


Sitemap.xml file is absent

The website lacks a sitemap file. Sitemaps can help robots index your content more thoroughly and quickly.

I can work on this issue!!

#1008


”Turia” is listed in the drop-down menu

Description:
"Turia" is listed in the drop-down menu alongside its corresponding county and commune, according to the law.
Precondition:
The website is up an running.

Step 1
Type "Turia" in the search bar.

Expected results
The location is listed in the drop-down menu as "Comuna Turia, COVASNA".


fix minor ticks

https://github.com/JuliaPlots/Makie.jl/pull/1968 broke minor ticks + grids:

This should fix it and add a reference image test for it.

Compile Times benchmark
Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt.
using time
master:  9.02 < 9.07 > 9.12, 0.03+-
pr:      9.05 < 9.08 > 9.12, 0.02+-
speedup: 0.99 < 1.00 > 1.00, 0.00+-
median:  +0.08% => invariant

This PR does not change the using time.
ttfp time
master   25.04 < 25.15 > 25.32, 0.11+-
pr       25.02 < 25.10 > 25.20, 0.07+-
speedup: 1.00 < 1.00 > 1.01, 0.00+-
median:  -0.19% => invariant

This PR does not change the ttfp time.

I still have the feeling that there are double updates in there but I'm not motivated right now to go in again and check. Such a time sink


where are the updates

Where are the updates??? Mod dead?

idk man

Unfortunately, yes. The maintainer of V-Hud (gennariarmando) left the repository to focus on other (personal) things. So, it might be possible that in the future, this repository will turn into the public archive, and will be read-only.


Implement iroha-lib separate module for Hyperledger Iroha 1

This is a source code for HL Iroha C++ library.
Currently, the latest HL Iroha 1.5 release (hyperledger/iroha:latest Docker image) is supported.
Possible actions:

transactions
batch transactions
queries

@andprogrammer Code looks better and better, almost ready. For now I see:

Mark resolved comments from me as resolved (sometimes comment would be nice)
Add about two examples of queries
Add possibility of adding optional arguments into queries, e.g. GetAccountTransactions (I mean those params)
Clean-up code before review
Correct style according to C++ style guide.

OK. In the interest of not making things too difficult down the line please do the following:

[ ] Update the description with an estimated time of completion.
[ ] Please rebase your commits on top of current develop. This cannot be done automatically, because you have conflicts which you will have to resolve manually.
[ ] Please make a habit of signing-off your commits. Nothing can be merged, if you are missing a sign-off.
[ ]  Try to observe our commit policy. 56 is way too much. I'm not comfortable squashing 56 commits into 1, so I'd recommend that after the code is reviewed and approved that you squash insignificant commits into significant ones.

@andprogrammer Code looks better and better, almost ready. For now I see:
1. Mark resolved comments from me as resolved (sometimes comment would be nice)

2. Add about two examples of queries

3. Add possibility of adding optional arguments into queries, e.g. [GetAccountTransactions](https://iroha.readthedocs.io/en/develop/develop/api/queries.html#get-pending-transactions) (I mean [those params](https://iroha.readthedocs.io/en/develop/develop/api/queries.html#result-pagination))

4. Clean-up code before review

5. Correct style according to [C++ style guide](https://iroha.readthedocs.io/en/develop/community/index.html#id5).

1-5 Done

@andprogrammer Code looks better and better, almost ready. For now I see:
1. Mark resolved comments from me as resolved (sometimes comment would be nice)

2. Add about two examples of queries

3. Add possibility of adding optional arguments into queries, e.g. [GetAccountTransactions](https://iroha.readthedocs.io/en/develop/develop/api/queries.html#get-pending-transactions) (I mean [those params](https://iroha.readthedocs.io/en/develop/develop/api/queries.html#result-pagination))

4. Clean-up code before review

5. Correct style according to [C++ style guide](https://iroha.readthedocs.io/en/develop/community/index.html#id5).

1-5 Done.

@andgit. In the interest of not making things too difficult down the line please do the following:

[x]  Update the description with an estimated time of completion.  20 Nov 2022
[ ]  Please rebase your commits on top of current develop. This cannot be done automatically, because you have conflicts which you will have to resolve manually.
[ ]  Resolve conflicts CMakeLists.txt
[ ]  Please make a habit of signing-off your commits. Nothing can be merged, if you are missing a sign-off.
[ ]   Try to observe our commit policy. 56 is way too much. I'm not comfortable squashing 56 commits into 1, so I'd recommend that after the code is reviewed and approved that you squash insignificant commits into significant ones.

1-4 Done. Repo/branch rebased with squash option.

@andprogrammer I see that You were able to squash all commits into single one. Please check if Your examples are working, if they are please change from Draft PR into normal PR.

Think about tests - but we need to ask Iroha-Core-Team about them. For me - if examples are working and You are wrapping generated from protobuf files: tests are not mandatory, but we have to ask @appetrosyan?

I'd recommend looking at the coverage first. Iroha 1 in general has a very different approach to testing, so IMO unit tests in this case are unnecessary.

Extra work

refactoring of iroha-cli, which should use the newly introduced library.

I fully agree. However, I'd be pragmatic, it seems like a good task for another intern.

Support for HL Ursa

Yes, but not now. Ursa is going to undergo a significant change, so I'd wait until at least Christmas before starting this work.

Wrap the library to be compatible with C language

C-abi library sounds good, but it's going to take a lot of extra work and meticulous code review. Dynamic linkage with a C++-based binary is non-trivial.

Documentation - how to install the library

Agreed. I'll help with the review.


[DONT REVIEW] Revert "Update JNI build to support nvcomp4 (#2217)"

This reverts commit 0bbefc0f7b568cc3acd3c653677010127ce85b30.

if we are going to revert the change in JNI repo, please also help update https://github.com/NVIDIA/spark-rapids-jni/blob/branch-24.12/ci/submodule-sync.sh#L97-L104 to avoid updating nvcomp pins in submodule-sync script thanks

subsumed by https://github.com/NVIDIA/spark-rapids-jni/pull/2483


Documentation: How are Pandas Categorical features identified, by name or the underlying categorical code?

I can't find an answer to this in the documentation or on the internet.
How are entries in a categorical column identified in LightGBM when passing a Pandas DataFrame with columns encoded as categoricals?
I'm running a ML model based on LightGBM in a production environment, and my data contains categorical variables.
Insted af label-encoding the categorical variables and specifying a list of categorical varaibles (columns) to LightGBM, I'm using Pandas DataFrames with the categorical columns set to be of the categorical dtype (df['some_categorical_column'] = df['some_categorical_column'].astype('category') ).
LightGBM can figure out which columns are categorical by inspecting the DataFrame, but does it assume that the underlying categorical codes are the same as they were when the model were trained, or does it inspect the associated labels for each categorical column?
Since my setup involves restarting the Python interpreter and reloading data, I'm concerned that the mapping of categorical values and their associated codes might change each time I run the code, and that might make LightGBM consider Green to be Red, Yellow to be Blue and so on:
# Mapping for a single categorical column when the LightGBM-model is trained:
Red:     1
Blue:    2
Green:   3
Yellow:  4

# Possible mapping for a single categorical column when the LightGBM-model is running in production:
Green:   1
Blue:    2
Yellow:  3
Red:     4

Environment info
Operating System:
Linux (Ubuntu)
CPU/GPU model:
AMD Threadripper
C++/Python/R version:
Python 3.7.3
LightGBM version or commit hash:
2.3.1
Error message
N/A
Reproducible examples
N/A
Steps to reproduce
N/A

Please refer to the code:
https://github.com/microsoft/LightGBM/blob/704e08300ead3e70625d5431c845141f2e564123/python-package/lightgbm/basic.py#L505
https://github.com/microsoft/LightGBM/blob/704e08300ead3e70625d5431c845141f2e564123/python-package/lightgbm/basic.py#L324-L331
I believe that within the same version pandas categories mapping is deterministic.

By default the categories are ordered alphabetically, which is why in the example below Amphibian is represented by a zero.
http://benalexkeen.com/mapping-categorical-data-in-pandas/

All right, so the categorical data will be converted correctly by virtue of Pandas categorical codes to the data in a sorted fashion.
However, if previously unseen data are introduced during the deployment period of the model, it seems like we'd have to manage the categorical codes manually:
X_train LGB-transformed codes and original Series
(array([2.], dtype=float32), 'D')
(array([0.], dtype=float32), 'A')
(array([1.], dtype=float32), 'C')
(array([1.], dtype=float32), 'C')
(array([3.], dtype=float32), 'E')
(array([3.], dtype=float32), 'E')
(array([0.], dtype=float32), 'A')

X_deploy LGB-transformed codes and original Series
(array([1.], dtype=float32), 'B')
(array([4.], dtype=float32), 'E')
(array([0.], dtype=float32), 'A')
(array([2.], dtype=float32), 'C')
(array([3.], dtype=float32), 'D')
(array([4.], dtype=float32), 'E')
(array([5.], dtype=float32), 'F')

Or put in another way: LightGBM relies on Pandas handling the encoding of the categoricals, and does not consider the actual labels of the Categorical when Label-encoding the data.
Thank for the help, it was invaluable!
I hope this example code will help others with similar questions.

Click to see the code which generated the output from above
#!/usr/bin/env python
# -*- coding: utf8 -*-

import pandas as pd
import lightgbm as lgb

X_train = pd.Series(['D', 'A', 'C', 'C', 'E', 'E', 'A'],
                    name='X', dtype='category').to_frame()
X_deploy = pd.Series(['B', 'E', 'A', 'C', 'D', 'E', 'F'],
                     name='X', dtype='category').to_frame()

X_train_transformed = lgb.basic._data_from_pandas(X_train, None, None, None)
X_deploy_transformed = lgb.basic._data_from_pandas(X_deploy, None, None, None)

# print("X_train", X_train, sep=':\n', end='\n\n')
# print("X_deploy", X_deploy, sep=':\n', end='\n\n')

# print('X_train_transformed', X_train_transformed, sep=':\n', end='\n\n')
# print('X_deploy_transformed', X_deploy_transformed, sep=':\n', end='\n\n')

print('X_train LGB-transformed codes and original Series')
print(*list(zip(X_train_transformed[0], X_train.X.to_list())), sep='\n', end='\n\n')
print('X_deploy LGB-transformed codes and original Series')
print(*list(zip(X_deploy_transformed[0], X_deploy.X.to_list())), sep='\n', end='\n\n')

Sorry to bump this discussion, but i have a few follow up questions.
I'm running into a similar problem. Let's say i have a pandas DataFrame with a categorical column 'Country'.  What happens if there are more entries in the training set compared to the deploy one.
Take for example the following case:
X_train = pd.Series(['Germany','France','Italy','Spain','Belgium'],
                    name='X', dtype='category').to_frame()
X_deploy = pd.Series(['Germany','France','Spain','Belgium'],
                     name='X', dtype='category').to_frame()

# Pandas mapping for each df:

dict( enumerate(X_train['X'].cat.categories ) )
# {0: 'Belgium',
#  1: 'France',
#  2: 'Germany',
#  3: 'Italy',
#  4: 'Spain'}

dict( enumerate(X_deploy['X'].cat.categories ) )
# {0: 'Belgium',
#  1: 'France',
#  2: 'Germany',
#  3: 'Spain'}

Thus if i now train my model, if i save the model to .json and look into the decision it made I'd find something of the type:
"right_child": {
    "split_index": 22,
    "split_feature": 1,
    "split_gain": 0.06720539927482605,
    "threshold": "0||2||3",
    "decision_type": "==",
    "default_left": false,
    "missing_type": "None",
    "internal_value": -0.00810006,
    "internal_weight": 771,
    "internal_count": 771,

I see that it made a decision based on my country feature, specifically if Country is Belgium, Germany or Italy, but it displays the category codes.
Thus, if i now apply this model on my deploy set, with a .predict(), since the code 3 now pertains to Spain instead of Italy, will it wrongly compute the result?
If so how i can i avoid this?
Basically its reversing the question OP posed, and thus talking in the general sense.

@AllanLRH Sorry, just noticed that your code snippet is not fully correct.
Here, the last argument is not None, it will be old categories from the training phase.
X_deploy_transformed = lgb.basic._data_from_pandas(X_deploy, None, None, None)

I think that simplified version of _data_from_pandas() for one column will look like the following.
import numpy as np
import pandas as pd

X_train = pd.Series(['D', 'A', 'C', 'C', 'E', 'E', 'A'],
                    name='X', dtype='category').to_frame()
X_deploy = pd.Series(['B', 'E', 'A', 'C', 'D', 'E', 'F'],
                     name='X', dtype='category').to_frame()

# original mapping in training set
print(*list(zip(X_train['X'].values, X_train['X'].cat.codes)), sep='\n')
# ('D', 2)
# ('A', 0)
# ('C', 1)
# ('C', 1)
# ('E', 3)
# ('E', 3)
# ('A', 0)

# simplified version of _data_from_pandas() for one column
X_train_transformed = X_train.apply(lambda x: x.cat.codes).replace({-1: np.nan}).values

pandas_categorical = list(X_train['X'].cat.categories)
X_deploy['X'] = X_deploy['X'].cat.set_categories(pandas_categorical)
X_deploy_transformed = X_deploy.apply(lambda x: x.cat.codes).replace({-1: np.nan}).values

# mapping in deploy set
print(*list(zip(X_deploy['X'].values, X_deploy['X'].cat.codes)), sep='\n')
# (nan, -1)
# ('E', 3)
# ('A', 0)
# ('C', 1)
# ('D', 2)
# ('E', 3)
# (nan, -1)

# what actually will be passed to LightGBM
print(X_train_transformed)
# [[2]
#  [0]
#  [1]
#  [1]
#  [3]
#  [3]
#  [0]]

print(X_deploy_transformed)
# [[nan]
#  [ 3.]
#  [ 0.]
#  [ 1.]
#  [ 2.]
#  [ 3.]
#  [nan]]


net: ipv6: Drop packet with my source IPv6 address

If a UDP packet with source IPv6 address same as the address of echo
server is received, it causes echo server to recursively send the packet
to itself, resulting in a crash and memory depletion. This commit fixes
the crash by dropping the packet.
Signed-off-by: Ruslan Mstoi<EMAIL_ADDRESS>

Codecov Report

Merging #7504 into master will decrease coverage by 1.44%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #7504      +/-   ##
==========================================
- Coverage   55.26%   53.81%   -1.45%
==========================================
  Files         467      467
  Lines       51649    51652       +3
  Branches     9888     9889       +1
==========================================
- Hits        28543    27796     -747
- Misses      19196    20054     +858
+ Partials     3910     3802     -108

Impacted Files
Coverage Δ

subsys/net/ip/ipv6.c
45.58% <100%> (-0.69%)
:arrow_down:

include/net/websocket.h
0% <0%> (-100%)
:arrow_down:

tests/net/websocket/src/main.c
11.79% <0%> (-72.31%)
:arrow_down:

tests/net/socket/tcp/src/main.c
45.18% <0%> (-54.22%)
:arrow_down:

subsys/net/lib/http/http_server.c
15.47% <0%> (-39.59%)
:arrow_down:

tests/net/websocket/src/server.c
17.85% <0%> (-37.5%)
:arrow_down:

subsys/net/ip/net_stats.h
39.16% <0%> (-26.67%)
:arrow_down:

tests/net/socket/udp/src/main.c
72.8% <0%> (-26.41%)
:arrow_down:

subsys/net/lib/app/server.c
30.3% <0%> (-22.23%)
:arrow_down:

subsys/net/lib/http/http.c
5.66% <0%> (-21.7%)
:arrow_down:

... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 03f9f66...13dec70. Read the comment docs.

Then the problem is apparently with "echo server", and that's where it should be fixed, not in the IP stack. How does loopback is supposed to work otherwise?

The problem is that we currently accept packet that is coming from external network interface (so not loopback) where source address is our own address.

The problem is that we currently accept packet that is coming from external network interface (so not loopback) where source address is our own address.

Then I'd suggest to double-check that the change is made on the code path which is used only for external interfaces, add code comment about this, and update commit message to mention that (and also to remove references to UDP and echo server, as TCP and other applications would be affected too). Also, similar change should be done for IPv4 too (if it's already present, worth mentioning that in the commit message either). Thanks.

(The original CI report was https://app.shippable.com/github/zephyrproject-rtos/zephyr/runs/14842/summary/console)

recheck

@rmstoi:

the problem is not only with echo server, it is with whole IP stack. I just use echo server for testing.  You can reproduce this issue sending UDP packet with source address 2001:db8::1.

It would be nice to better understand where the problem happens (i.e. where infinite loop or recursive call happens), and it's actual nature (I for example don't have an idea how it could work with different source/destination port numbers). I don't have an easy way to forge a packet with an arbitrary address, if you can describe how to reproduce your test, I'd be interested to try it.

Loopback seems to work

As you can see from the CI report, the number of tests are broken (some exactly use loopback capabilities to test networking functionality with CI).

Basically it is a DOS attack, after which IP stack does not recover.

Sure, I'm all for fixing it, but let's try to understand the root issue better and describe it better for the folks who'll be looking at the stack later (so I hope my suggestions for comments/commit message make sense).

recheck

recheck

@pfalcon Thanks!


Weird behaviour with the Fn key in Ubuntu 22.04

I've noticed some strange behavior with the Fn key when using xrdp in Ubuntu 22.04.
When pressing the Fn key, it's interpreted as a Delete key. This can be viewed in View Keyboard Layout; here I'm pressing down the Fn key and the Delete key lights up:

This issue isn't that major, but it's quite odd, and it can cause some occasional confusion when clicking the Fn key.
Xrdp version: 0.9.17-2ubuntu2
Ubuntu version: 22.04
Windows specs:
Edition: Windows 10 Pro
Version: 21H2
OS build: 19044.1620
Experience: Windows Feature Experience Pack 120.2212.4170.0
PC specs:
Model: ThinkPad P14s Gen 2a
BIOS version: R1MET45W 1.15

Hi @AndreasNasman
What do you get for the following command:-
sudo grep -E '(keyboard)|(keymap)' /var/log/xrdp.log

we'll try to figure out what keyboard your client is sending over.
I take it everything is OK on the Windows 10 side with the keyboard?

I use EurKey by default, forgot to change it before doing the test. 🙂 The edited post is with English (US) as the layout and the Fn key behaves as Delete in Ubuntu still.

Can I somehow log what the Fn key is interpreted as on the Ubuntu side?

You can use the xev utility to see what scancode the Fn key is being interpreted as.
As I correct in thinking this key is an extra key to a standard keyboard? I've got a full-size UK keyboard in front of me here, and it has no Fn key. On an Acer laptop I user for testing, there's an Fn key, but it doesn't generate a keypress that xev can see.
What do you want the Fn key to do when it's pressed in Ubuntu?

You can use the xev utility to see what scancode the Fn key is being interpreted as.

Here are the result from running xev and repeatedly pressing the Fn key.
First with Xrdp. As you can see, it's interpreted as a Delete keystroke:

Next, running the VM straight with Hyper-V. Not sure how its interpreted, but it's not a Delete key:

As I correct in thinking this key is an extra key to a standard keyboard? I've got a full-size UK keyboard in front of me here, and it has no Fn key. On an Acer laptop I user for testing, there's an Fn key, but it doesn't generate a keypress that xev can see.

It's the Fn key of a ThinkPad P14s Gen 2a laptop, with a Nordic keyboard layout:

What do you want the Fn key to do when it's pressed in Ubuntu?

Nothing, really; I want similar behavior as connecting to the VM with Hyper-V (see picture above).
Here's an example where the Fn key behaving as Delete is an issue. At work, I have a keyboard that's missing the Home key. To simulate it, I press Fn + ⬅. Say I want to copy some text from Ubuntu to Windows. I highlight the word, copy it, and navigate to Windows with the keyboard. To do that, I would press Ctrl + Alt + Home, although I have no Home key, so the shortcut is then Ctrl + Alt + Fn + ⬅. I forgot to un-highlight the text, and the Delete key event from the Fn key deletes the text in Ubuntu as I'm moved over to Windows.
It's not a major issue by any means, just an annoyance that I don't understand why it's happening.
Xrdp otherwise seems like a more appealing environment, enabling Enhanced mode in Hyper-V. The only issues so far I've found are this one and #2092.

Can you use xev to see what the keycode is for delete?
At a technical level, there's a lot of indirection going on here. The RDP protocol uses virtual scan codes to represent keys. These are converted in xrdp to X11 keycodes in xrdp/lang.c using a hard-coded table. The X server, whichever one we're using then maps those keycodes to an actual input data stream.
Esentially, if you're getting a different keycode for the delete key and the Fn key we can do something at the xrdp level. If they are the same, you'll need to figure out a way to change the virtual scan code which is getting passed over the link.

I see! Thanks for the explanation!
The Delete key seems to give the exact output as pressing the Fn key. Here I spam the Delete key:

If they are the same, you'll need to figure out a way to change the virtual scan code which is getting passed over the link.

I guess this is the next step then? 🙂

Indeed.
That'll be down to something in the Windows settings, or possibly the BIOS. By the time the code gets to us, there's not a lot we can do I'm afraid.

Indeed.
That'll be down to something in the Windows settings, or possibly the BIOS. By the time the code gets to us, there's not a lot we can do I'm afraid.

How is that possible that windows RDP client sends a delete keypress when the 'Fn' key is pressed when connected to an XRDP server, but not when connected to a native Windows server?  Is is possible that the keypress is fundamentally misinterpreted at the server end.? The whole thing is clear as mud to me, because I would expect that the 'Fn' key would be interpreted only on the client hardware/OS and not forwarded as a keypress at all.
  'Fn' keys are found on most laptops without full size keyboards.

@dpzimmer - you make some good points. Thanks for that.
Are you also experiencing this issue? If so, what kind of keyboard do you have and what is the layout? I'll need to try to reproduce this using a soft keyboard I think.

@dpzimmer - you make some good points. Thanks for that.
Are you also experiencing this issue? If so, what kind of keyboard do you have and what is the layout? I'll need to try to reproduce this using a soft keyboard I think.

Hi Matt,
Yes, I am experiencing literally the exact same behavior as Andreas.
My system is "Lenovo ThinkPad T14 Gen 3" running "Windows 11 Pro 22H2".  The keyboard layout is attached.
I am connecting to "Ubuntu 23.04" as well as other slightly less recent Ubuntu versions, all experiencing the same issue.
Thanks,
Don.

I've had a bit more of a think about this; the behaviour you're both seeing must be caused by the xrdp server receiving a scancode when the Fn key is pressed.
It would be very useful indeed if either of you is able to install one of the utilities above on your systems and see if you can get a scancode from the Fn key. If there is one, and if it's being sent over the RDP link I can trace it through the keyboard logic on the server side.

Hi Matt,
Regarding "Try pressing Fn+ ESC and see if the operation of the Fn key changes.", It toggles the FnLock state, but no change in behaviour when pressing the Fn key alone.
Following up with the tools you suggested, and one more online tool (images below), it seems that a keycode of 0xFF (not a valid code) is forwarded, which would match against anything if your are using bitwise OR at the receiving end.

Thanks for the swift reply @dpzimmer
The scancode used by RDP is I believe the "BIOS Key Code". So in this case it would be 99 (0x63). Also the E0 is I believe significant and is also visible in the Keyboard Scan Code Specification. I don't know how this gets communicated over the RDP link.
I'll need to read and experiment further, but I think we've discovered why xrdp behaves differently from a Windows Server.

How is this going? I remember, 20 years ago, lol, all my virtual machine integration worked perfectly :).
Would be great to sort this out. Though I do admit, I totally forgot there was an actual home and end key on my keyboard, still they are very far away, and I code a lot. Would be great to work this out.
One other thing which may have a bearing, is that I have the FN and CTRL key swapped on my lenovo (ill boot it up without that and see if it makes a difference). But looking at these posts I would guess not

This should all be largely addressed in the development version of xrdp, where we've moved to evdev keycodes by default. If it's not addressed, it should be a lot easier to sort out.

Heya, which version should have those changes?
Observing the same problem with xrdp-<IP_ADDRESS> on OpenSUSE Tumbleweed.

The development version has these changes. That's not 0.9.x or 0.10.x.
It' s not released yet, so you'll need to build from source to try it.

Got it, thank you so much for the clarification. Will wait for it to trickle into Tumbleweed then :)


Add more error message

Ticket: []
Feature flag: n/a

Purpose

Summary of Changes

Screenshot(s)

Side Effects

QA Notes

@adlius yml translation errors. Also, don't we need to fix invalidSchemaId and invalidColumnId translations. https://github.com/CenterForOpenScience/ember-osf-web/pull/1316/files#diff-537d69fea1991fe4226fddb2f2decac5c2ce6dcfe1c9ac2d547ec6abde2e239eL1037-L1042


brior1.1 wavelet returns different length signal than input

Hello,
I am using wavelets to decompose stock market price signals. I am inputting an array with length 1502 and getting a signal with 1504 entries. How should I deal with the extra values? Can I just remove the last two or first two?
Thank you.
Martin

Yes, any extra values should be removed from the end of the reconstructed signal. This is a known behavior for round trip forward/inverse transform at n levels when the input signal size is not a multiple of 2**n.


Add level creator

Make a level creator taht takes chunks of a level and makes a level

OK first I am going to need to make some level chunks and devise a simple way of connecting them

Whoops, Inheritance We are going to create a class LevelPeice using GDScript and use that to enforce level peices to have an end and start

Build the classes and start piece, simplified algorithm

Almost working end piece doesn't end right.