Avoid using RPM/DEB based installation and install managed plugin

fixes #1051: Complete the move to managed plugins - drop support for unmanaged ones
This PR addresses following things

make build-all builds managed plugin as well as push to docker hub after building
deploy-vm helper is corrected
new helper method installManagedPlugin for installing plugin
cleanVM helper is updated to remove managed plugin if installed
corrected tests to accommodate such change

Tested locally: (ran  make e2e-test)
nshukla-mbp:docker-volume-vsphere nshukla$ make clean-all build-all deploy-all
Makefile:54: warning: overriding commands for target `clean'
plugin/Makefile:88: warning: ignoring old commands for target `clean'
/Library/Developer/CommandLineTools/usr/bin/make --directory=vmdk_plugin clean-all
../misc/scripts/build.sh clean-as-root
make: Entering directory `/go/src/github.com/vmware/docker-volume-vsphere/vmdk_plugin'
rm -rf ../build .code_verify
make --directory=../esx_service clean
make[1]: Entering directory `/go/src/github.com/vmware/docker-volume-vsphere/esx_service'
rm -rf vmware-esx-vmdkops-0.13.90eea6a.vib ../build/libvmci_srv.so ../build/libvmci_srv.so.64 ../build/descriptor.xml ../build/payloads .lint_*
make[1]: Leaving directory `/go/src/github.com/vmware/docker-volume-vsphere/esx_service'
make: Leaving directory `/go/src/github.com/vmware/docker-volume-vsphere/vmdk_plugin'
../misc/scripts/deploy-tools.sh cleanvm "<IP_ADDRESS> <IP_ADDRESS>" ""vsphere:latest""

=> cleanvm: Cleaning up on<EMAIL_ADDRESS>: deb Wed Apr 12 17:14:47 PDT 2017

dpkg-query: no packages found matching docker-volume-vsphere
vsphere:latest
../misc/scripts/deploy-tools.sh cleanesx "<IP_ADDRESS>" esx-vmdkops-service

=> Cleaning up on ESX <IP_ADDRESS> Wed Apr 12 17:14:48 PDT 2017

Removal Result:
   Message: Operation finished successfully.
   Reboot Required: false
   VIBs Installed:
   VIBs Removed: VMWare_bootbank_esx-vmdkops-service_0.13.90eea6a-0.0.1
   VIBs Skipped:
/Library/Developer/CommandLineTools/usr/bin/make --directory=vmdk_plugin build-all
../misc/scripts/build.sh pylint
make: Entering directory '/go/src/github.com/vmware/docker-volume-vsphere/esx_service'
pylint --errors-only -d import-error,no-name-in-module --rcfile pylint_cfg.ini ../esx_service
pylint --errors-only -d import-error,no-name-in-module --rcfile pylint_cfg.ini ../esx_service/vmodl
pylint --errors-only -d import-error,no-name-in-module --rcfile pylint_cfg.ini cli
pylint --errors-only -d import-error,no-name-in-module --rcfile pylint_cfg.ini utils
make: Leaving directory '/go/src/github.com/vmware/docker-volume-vsphere/esx_service'
../misc/scripts/build.sh build
make: Entering directory `/go/src/github.com/vmware/docker-volume-vsphere/vmdk_plugin'
go tool vet .
gofmt -s -l -w .
GO15VENDOREXPERIMENT=1 go build --ldflags '-extldflags "-static"' -o ../build/docker-volume-vsphere github.com/vmware/docker-volume-vsphere/vmdk_plugin
GO15VENDOREXPERIMENT=1 go test -c -o ../build/vmdkops.test github.com/vmware/docker-volume-vsphere/vmdk_plugin/drivers/vmdk/vmdkops -cover
GO15VENDOREXPERIMENT=1 go test -c -o ../build/docker-volume-vsphere.test github.com/vmware/docker-volume-vsphere/vmdk_plugin -cover
make  --directory=../esx_service build
make[1]: Entering directory `/go/src/github.com/vmware/docker-volume-vsphere/esx_service'
cc -fPIC -m32 -shared -o ../build/libvmci_srv.so vmci/vmci_server.c
cc -fPIC -shared -o ../build/libvmci_srv.so.64 vmci/vmci_server.c
Staging in ../build/payloads
cp __init__.py vmdk_ops.py volume_kv.py vsan_policy.py cli/vmdkops_admin.py cli/vmdkops_post_update.11.1.py ../build/payloads/vmdkops/usr/lib/vmware/vmdkops/bin
cp utils/__init__.py utils/auth.py utils/auth_api.py utils/auth_data.py utils/auth_data_const.py utils/cli_table.py utils/convert.py utils/error_code.py utils/kvESX.py utils/log_config.py utils/threadutils.py utils/vmdk_utils.py utils/vsan_info.py  ../build/payloads/vmdkops/usr/lib/vmware/vmdkops/Python
cp -R tools/sqlite/sqlite3 ../build/payloads/vmdkops/usr/lib/vmware/vmdkops/Python/2
cp tools/sqlite/*.so ../build/payloads/vmdkops/usr/lib/vmware/vmdkops/Python/2
cp ../build/libvmci_srv.so ../build/payloads/vmdkops/usr/lib/vmware/vmdkops/lib/libvmci_srv.so
cp ../build/libvmci_srv.so.64 ../build/payloads/vmdkops/usr/lib/vmware/vmdkops/lib64/libvmci_srv.so
cp vmdk-opsd ../build/payloads/vmdkops/etc/init.d
VMDKOPS_PAYLOAD=../build/payloads/vmdkops make  --directory=vmodl copy_to_payload
make[2]: Entering directory `/go/src/github.com/vmware/docker-volume-vsphere/esx_service/vmodl'
Ignoring VMODL 'copy_to_payload' target.
See https://github.com/vmware/docker-volume-vsphere/pull/975 for details.
make[2]: Leaving directory `/go/src/github.com/vmware/docker-volume-vsphere/esx_service/vmodl'
vibauthor --debug --compose --vib=../build/vmware-esx-vmdkops-0.13.90eea6a.vib --stage-dir ../build --offline-depot ../build/vmware-esx-vmdkops-0.13.90eea6a.zip --force
WARNING: a VIB file with "accepted" acceptance level need to be signed
    you can do it later using vibauthor with --sign option
Successfully created ../build/vmware-esx-vmdkops-0.13.90eea6a.vib.
Successfully created ../build/vmware-esx-vmdkops-0.13.90eea6a.zip.
make[1]: Leaving directory `/go/src/github.com/vmware/docker-volume-vsphere/esx_service'
make: Leaving directory `/go/src/github.com/vmware/docker-volume-vsphere/vmdk_plugin'
../misc/scripts/build.sh deb
make: Entering directory '/go/src/github.com/vmware/docker-volume-vsphere/vmdk_plugin'
fpm --deb-no-default-config-files -p ../build -C package -s dir -n docker-volume-vsphere -v 0.13.90eea6a --provides docker-volume-vsphere -m<EMAIL_ADDRESS>--url https://github.com/vmware/docker-volume-vsphere --after-install ../misc/scripts/install/after-install.sh --before-remove ../misc/scripts/install/before-remove.sh --after-remove ../misc/scripts/install/after-remove.sh --description "Docker Volume Driver for vSphere" --architecture x86_64 --force -t deb .
Created package {:path=>"../build/docker-volume-vsphere_0.13.90eea6a_amd64.deb"}
make: Leaving directory '/go/src/github.com/vmware/docker-volume-vsphere/vmdk_plugin'
../misc/scripts/build.sh rpm
make: Entering directory '/go/src/github.com/vmware/docker-volume-vsphere/vmdk_plugin'
fpm --epoch 0 -p ../build -C package -s dir -n docker-volume-vsphere -v 0.13.90eea6a --provides docker-volume-vsphere -m<EMAIL_ADDRESS>--url https://github.com/vmware/docker-volume-vsphere --after-install ../misc/scripts/install/after-install.sh --before-remove ../misc/scripts/install/before-remove.sh --after-remove ../misc/scripts/install/after-remove.sh --description "Docker Volume Driver for vSphere" --architecture x86_64 --force -t rpm .
Created package {:path=>"../build/docker-volume-vsphere-0.13.90eea6a-1.x86_64.rpm"}
make: Leaving directory '/go/src/github.com/vmware/docker-volume-vsphere/vmdk_plugin'
/Library/Developer/CommandLineTools/usr/bin/make --directory=plugin all
Using the following config:
DOCKER_HUB_REPO shuklanirdesh EXTRA_TAG -dev VERSION_TAG 0.14
PLUGIN_NAME shuklanirdesh/docker-volume-vsphere:0.14-dev
=== Cleaning work files, images and plugin(s)...
rm -rf ./rootfs
rm -f docker-volume-vsphere
docker plugin rm shuklanirdesh/docker-volume-vsphere:0.14-dev -f
shuklanirdesh/docker-volume-vsphere:0.14-dev
docker rmi shuklanirdesh/docker-volume-vsphere:rootfs
Error response from daemon: No such image: shuklanirdesh/docker-volume-vsphere:rootfs
make[1]: [clean] Error 1 (ignored)
== building Docker image, unpacking to ./rootfs and creating plugin...
cp ../build/docker-volume-vsphere .
docker build -q -t shuklanirdesh/docker-volume-vsphere:rootfs .
sha256:9ca0f6c4ea031dc8d0a03d31ebd9013052286e64203787000f393d2c14f210e0
mkdir -p ./rootfs
docker create --name tempContainer shuklanirdesh/docker-volume-vsphere:rootfs
29d32963e25aa84faf1edd8ddc3cbb6aed26bf6d69e7aa438ad2e5d8fc89d72c
docker export  tempContainer  | tar -x -C ./rootfs
docker rm -vf  tempContainer
tempContainer
docker rmi shuklanirdesh/docker-volume-vsphere:rootfs
Untagged: shuklanirdesh/docker-volume-vsphere:rootfs
Deleted: sha256:9ca0f6c4ea031dc8d0a03d31ebd9013052286e64203787000f393d2c14f210e0
Deleted: sha256:b2052c66ddf53ebe3d1ffdfbb03aaacf728d93795e92be1eb72506feaaa83da7
Deleted: sha256:5f76289b756457a2ccb861a5477dcc3b244b635359388b2ab9ddb47041458abc
Deleted: sha256:8e23b23254748b2e14e76339c43589c6f4ca057c5dd0ad5ee272f531c194f402
Deleted: sha256:c0855efdb930680f266ce0b6552a5a1092fd9ee87ff3ec9f0a70ad80c04a2747
Deleted: sha256:a9dd24688f71cd098fb800fbf197b8d1945e98aa49e4c4b5c44e80de5ecfa5b7
Deleted: sha256:5d78ac447d700d797470105a5e3bf8e83672d7e0bbfc05663c43eef41facb6a7
-- Creating  plugin shuklanirdesh/docker-volume-vsphere:0.14-dev ...
docker plugin create shuklanirdesh/docker-volume-vsphere:0.14-dev .
shuklanirdesh/docker-volume-vsphere:0.14-dev
Pushing shuklanirdesh/docker-volume-vsphere:0.14-dev to dockerhub.io...
The push refers to a repository [docker.io/shuklanirdesh/docker-volume-vsphere]
c2f766e6d3bd: Pushed
0.14-dev: digest: sha256:9d61c99d78d0ecd622e221cf37822aff975ac4fad0e5de440b9b2a5ff10bdc77 size: 519
/Library/Developer/CommandLineTools/usr/bin/make --directory=vmdk_plugin deploy-all
../misc/scripts/deploy-tools.sh cleanesx "<IP_ADDRESS>" esx-vmdkops-service

=> Cleaning up on ESX <IP_ADDRESS> Wed Apr 12 17:17:19 PDT 2017

../misc/scripts/deploy-tools.sh deployesx "<IP_ADDRESS>" "../build/vmware-esx-vmdkops-0.13.90eea6a.vib"

=> Deploying to ESX<EMAIL_ADDRESS>Wed Apr 12 17:17:22 PDT 2017

Installation Result:
   Message: Operation finished successfully.
   Reboot Required: false
   VIBs Installed: VMWare_bootbank_esx-vmdkops-service_0.13.90eea6a-0.0.1
   VIBs Removed:
   VIBs Skipped:
vmdkops-opsd is running pid=1083647
../misc/scripts/deploy-tools.sh cleanvm "<IP_ADDRESS> <IP_ADDRESS>" ""vsphere:latest""

=> cleanvm: Cleaning up on<EMAIL_ADDRESS>: deb Wed Apr 12 17:17:51 PDT 2017

dpkg-query: no packages found matching docker-volume-vsphere
Error response from daemon: plugin "vsphere:latest" not found
../misc/scripts/deploy-tools.sh deployvm "<IP_ADDRESS> <IP_ADDRESS>" shuklanirdesh/docker-volume-vsphere:0.14-dev

=> Deploying<EMAIL_ADDRESS>: deb Wed Apr 12 17:17:53 PDT 2017

=> installManagedPlugin: Installing vDVS plugin [shuklanirdesh/docker-volume-vsphere:0.14-dev] Wed Apr 12 17:17:53 PDT 2017

0.14-dev: Pulling from shuklanirdesh/docker-volume-vsphere
8bf57e2daba8: Verifying Checksum
8bf57e2daba8: Download complete
Digest: sha256:9d61c99d78d0ecd622e221cf37822aff975ac4fad0e5de440b9b2a5ff10bdc77
Status: Downloaded newer image for shuklanirdesh/docker-volume-vsphere:0.14-dev
Installed plugin shuklanirdesh/docker-volume-vsphere:0.14-dev
28049
../misc/scripts/deploy-tools.sh deployvmtest "<IP_ADDRESS> <IP_ADDRESS>" ../build ../misc/scripts

=> Deploying test code to<EMAIL_ADDRESS>Wed Apr 12 17:18:00 PDT 2017

nshukla-mbp:docker-volume-vsphere nshukla$ make e2e-test
Makefile:54: warning: overriding commands for target `clean'
plugin/Makefile:88: warning: ignoring old commands for target `clean'
/Library/Developer/CommandLineTools/usr/bin/make --directory=vmdk_plugin e2e-test

=> Running target e2e-test Wed Apr 12 17:18:57 PDT 2017

../misc/scripts/build.sh e2e-dkrVolDriver-test "<IP_ADDRESS>" "<IP_ADDRESS>" "<IP_ADDRESS>"
make: Entering directory `/go/src/github.com/vmware/docker-volume-vsphere/vmdk_plugin'

=> Running target e2e-dkrVolDriver-test Thu Apr 13 00:18:58 UTC 2017

GO15VENDOREXPERIMENT=1 go test github.com/vmware/docker-volume-vsphere/vmdk_plugin/E2E_Tests -cover -v
=== RUN   TestVolumeCreationFirstTime

creating volume [abc_1492042738] on VM[<IP_ADDRESS>]
destroying volume [abc_1492042738]
creating volume [abc_1492042745] on VM[<IP_ADDRESS>]
destroying volume [abc_1492042745]--- PASS: TestVolumeCreationFirstTime (14.20s)
	createVolume_test.go:63:
		Testcase passed: successfully able to create volume [abc_1492042738
		]
	createVolume_test.go:63:
		Testcase passed: successfully able to create volume [abc_1492042745
		]
=== RUN   TestVolumeProperties
2017/04/13 00:19:12 running end_to_end tests - current time:  2017-04-13 00:19:12.877165984 +0000 UTC
2017/04/13 00:19:12 Running test on VM -  <IP_ADDRESS>
2017/04/13 00:19:13 Docker version:   Docker version 17.03.0-ce, build 60ccb22

2017/04/13 00:19:13 Creating a volume of Format Type -  thin
2017/04/13 00:19:18
 successfully created a  volume --  dockerVol_1492042753
2017/04/13 00:19:18 Verifying volume properties like size, disk-format and attached-to-vm fields at vm and esx for volume -  dockerVol_1492042753
2017/04/13 00:19:27 Finished verifying volume properties like size, disk-format and attached-to-vm fields at vm and esx for volume -  dockerVol_1492042753
2017/04/13 00:19:38 Creating a volume of Format Type -  zeroedthick
2017/04/13 00:19:44
 successfully created a  volume --  dockerVol_1492042778
2017/04/13 00:19:44 Verifying volume properties like size, disk-format and attached-to-vm fields at vm and esx for volume -  dockerVol_1492042778
2017/04/13 00:19:53 Finished verifying volume properties like size, disk-format and attached-to-vm fields at vm and esx for volume -  dockerVol_1492042778
2017/04/13 00:20:04 Creating a volume of Format Type -  eagerzeroedthick
2017/04/13 00:20:11
 successfully created a  volume --  dockerVol_1492042804
2017/04/13 00:20:11 Verifying volume properties like size, disk-format and attached-to-vm fields at vm and esx for volume -  dockerVol_1492042804
2017/04/13 00:20:20 Finished verifying volume properties like size, disk-format and attached-to-vm fields at vm and esx for volume -  dockerVol_1492042804
--- PASS: TestVolumeProperties (78.52s)
PASS
coverage: 0.0% of statements
2017/04/13 00:20:34 -----Clean-up finished - current time:  2017-04-13 00:20:34.207652826 +0000 UTC
ok  	github.com/vmware/docker-volume-vsphere/vmdk_plugin/E2E_Tests	95.530s
make: Leaving directory `/go/src/github.com/vmware/docker-volume-vsphere/vmdk_plugin'

Thanks @msterin  for the quick review; please find my comment inline.

Why are we cleanup up both, is is for making transition easy ?

Yeah, it was for making transition easy.

I think we can do manual step once to remove it.

I will update contributing.md (for one time removal manually) and remove the step from cleanVM part.
TODO: CONTRIBUTING.md will be added in the next diff.


Autosave form after change

First, thank you for this extremely useful library.
Is it possible to submit the form after a value change via the <Form/>'s onChange prop?
I know it's possible to get formState and formApi from it, but I've not worked out how to dispatch formApi.submitForm() without access to the <form/> node.
or, is this more of a react issue...?

Technically you could just call formPi.submitForm in the onChange because it gives you access to the api

Def anteresting use case tho .. typically you would not want to do somthing like this

you could just call formPi.submitForm in the onChange

Would you be able to give an example of this? I've tried calling it with no arguments, and I don't think it's triggering all normal lifecycle events as described here.

Def anteresting use case tho .. typically you would not want to do somthing like this

Any other ideas for solving the same issue? Thanks.

Sorry i should have been more clear, so when I said

Def an interesting use case tho .. typically you would not want to do somthing like this

What i meant is that forms are designed to be modified, and then finally when they are ready, submitted. Submission is a very well defined event and its designed to be the final step when a user is done modifying the form, not while they are actively modifying it.
I guess this is why im a little confused as to why you need to call forms OnSubmit on every onChange. onChange was designed for your use case. It is a function that gets called every time the form updates. So you could do things like, save the state externally, or do other things whenever the form updates ( this sounds like your use case ). Not sure there is a point to calling onSubmit each time. If you were to, for example, interact with a firstName field in your form it would call all the lifecycle methods its supposed to and then callOnChange. Calling onSubmit would probably be overkill as it would call every fields lifecylcle method for no reason. If you really need updating one field to affect another then you can simply use a formLevel validator <Form validate={someValidationFunction}> and that validate function would get called on every onChange with all values.

Thanks for your reply.

I guess this is why im a little confused as to why you need to call forms OnSubmit on every onChange.

I could certainly call my own submit handler inside onChange (the function usually passed to <Form/>) but this library does a lot of nice things after a normal submit I'd have to replicate in onChange. So far, I'm making use of the preSubmit function, and formApi.submitting in a child component.
Unless I'm mistaken, I'd be required to replicate a lot of functionality, something like this: (untested)
const onChange = (mySubmitHandler) => (formState, formApi) => {
    // (will need to return here before the next line if values haven't changed)
    formApi.setFormState({ ...formState, submitting: true })
    mySubmitHandler(preSubmit(formState.values)).then(a => {
        formApi.setFormState({ ...formState, submitting: false, submitted: true })
    });
}

export default ({ mySubmitHandler }) => (
    <Form onChange={onChange(mySubmitHandler)} />
)

My desire to set submitting by the way, is because I have a component like this:
// previously wrapped with withFormApi...
const SaveIndicator = ({ formApi }) => (
    <span>{formApi.submitting ? "Saving" : "Saved"}<span>
)

(I should also mention mySubmitHandler is debounced, as the next thing it does is perform a network request)
Hopefully this gives you a good idea of the position I'm in!


Blue Sky's composer should gather social graph data for PDFs and other non-HTML content too

Right now, if you put a link ending in .html into the composer (on Web), and ask the website to generate a card, you can watch the network panel make a request to https://cardyb.bsky.app/v1/extract.
For example, this URL makes a request in the network panel when you ask to make a card:
https://foo.com/foo.html
But these URLs, ending in .jpeg, .png, .pdf and .xml do not:
https://foo.com/foo.jpeg
https://foo.com/foo.png
https://foo.com/foo.xml
https://foo.com/foo.pdf
I understand the reasoning: In theory, those file endings are telling Blue Sky that they will not have Social Graph information, since that information only exists in HTML content.
That theory is correct, but at the website I run, we share millions of PDFs, and we have a neat hack in place to help fight misinformation and provide better details to our users. When we detect an open graph crawler, we redirect the crawler to an HTML page with open graph data (if it's not a crawler, we serve the PDF). I know that DocumentCloud also uses this trick.
 As far as I know, Blue Sky is the only one where it doesn't work.
To Reproduce

Paste this link into the web composer: https://foo.com/f.html

Open the browser's network panel.

Press the button in the composer to get the card.

Note that it returns an error (the link doesn't work), and that you see a request in the network panel:

Change the URL to https://foo.com/f.pdf

Press the button in the composer to get the card.

Note that it made no requests and throws no error.

Expected behavior
Blue Sky should go to the URL, regardless of the file ending, and test if it's actually HTML or a PDF. Heck, some horribly misconfigured website might end links with .pdf even when serving HTML. :)
Details

Platform: Firefox on Linux
Platform version: 118

Additional context
This bug is a bit of a bummer because one of the things that drove me to Blue Sky is that Twitter removed headlines. This bug means that the links from my website don't have twitter cards or headlines either. Darn!
I took a look around the code, but couldn't find where this is done. If somebody sends a pointer, we've got technical folks and volunteers that could help with this.

Well, OK, there's a workaround if you're the one posting the link, but it's still broken for everybody that's not this clever.
You can substitute %2e instead of the last period in your link. This works:
https://storage.courtlistener.com/recap/gov.uscourts.flsd.648654/gov.uscourts.flsd.648654.3.0.pdf
Not, um, exactly, great, but it's something!

One other thought here. It isn't part of open graph, but I've always thought it would be nice to serve open graph data via headers. In fact, I think Facebook must have gotten distracted while building the spec, and just didn't get around to this.
If BlueSky supported this one day, it'd make it possible to return detailed information and thumbnails when serving binary content.
(I've been banging this drum for a decade or so.)


Some fixes

remove a *Box::new hack
functional style hash digest calculation
more secure handling of signing_key_bytes when generating hashes

TODO:

the derivatives of signing key bytes (Hmac, ChaCha20Rng) should be zeroized too - but currently they don't implement the trait. Need to create PRs for the corresponding crates. Probably there are other non-obvious objects that should be zeroized, need to review.
some remaining explicit .zeroize() calls need to be replaced with zeroized-on-drop containers. The problem is that secrecy seemingly does not support exposing a mutable reference to a secret which we need e.g. in ceygen(). Trying to find a way to work around that.
should we use https://docs.rs/secrets instead of secrecy? The problem here is that it depends on an external libsodium, but provides better zeroization guarantees.

Have you seen SecretBytesMut? It wraps a BytesMut (ref: this example).

I have, but as I said the problem is that Secret (and SecretBytesMut is an alias for Secret<BytesMut>) does not provide a method to expose a mutable reference to the secret, so you can't actually mutate the BytesMut.

Curious why we use ChaCha20Rng instead of the Linux Kernel CSPRNG?

Would it work on Mac? Would it work in WASM and Windows (and do we need it to)? (I don't know the answers to these questions myself, just some things to consider)

I have, but as I said the problem is that Secret (and SecretBytesMut is an alias for Secret<BytesMut>) does not provide a method to expose a mutable reference to the secret, so you can't actually mutate the BytesMut.

Hmm... I just played around with this for 30 minutes, and you're right - well, the naming is certainly misleading 😅

Would it work on Mac? Would it work in WASM and Windows (and do we need it to)? (I don't know the answers to these questions myself, just some things to consider)

Mac: ✅
WASM: ✅ (Uses the browsers Crypto.getRandomValues)
Windows: ✅ (with a note that windows uses BCryptGenRandom which I have little insight into the security properties of.)
WASI: (Unclear, uses some function random_get which is nondescript in its documentation on the source of entropy).
See the full layout here.

One thing I don't follow is whether getrandom actually provides a seeded RNG that we need here? Am I not seeing something?
Also this question is probably worth a separate issue filed.

One thing I don't follow is whether getrandom actually provides a seeded RNG that we need here? Am I not seeing something?

So it depends on the underlying operating system {CS,}PRNG, why is it important that we seed the CSPRNG?
Most secure CSPRNGs (like the Linux Kernel) offer a function like get_random_bytes(void *buf, size_t len), since from a user API perspective users just want the CSPRNG to  "give me cryptographically secure random bytes" and let the CSPRNG implementation handle seeding, among the other things, that make a CSPRNG secure.

why is it important that we seed the CSPRNG?

It was important to the authors of tofn. @thor314 may provide a better insight, but I imagine it was for the reproducibility of shares. If seeding is not important, then certainly we have a much wider range of options here.

why is it important that we seed the CSPRNG?

It was important to the authors of tofn. @thor314 may provide a better insight, but I imagine it was for the reproducibility of shares. If seeding is not important, then certainly we have a much wider range of options here.

Agree that reproducibility of shares seems the most likely reason.


Add fine-grained control over root certificates

As suggested in https://github.com/seanmonstar/reqwest/issues/1843#issuecomment-1557500147, this introduces ClientBuilder::no_webpki_root_certs() and ClientBuilder::no_native_root_certs().
Fixes https://github.com/seanmonstar/reqwest/issues/1843.

I think would expect it to be something like no_certs(), applied for all ways of loading default certificates.

That's already available as ClientBuilder::tls_built_in_root_certs(), which probably could get improved by removing the crate feature gates, but it is a separate problem.
The original problem is to control more tightly which root certificates to load. Right now, if both webpki-roots and rustls-native-certs crate features are enabled, you can only disable or enable both. The goal is to e.g. only enable webpki-roots even when both crate features are enabled.

Rebased and CI passes.
This is ready to be reviewed again.
Now that #1850 is merged, it could be argued that if fine-grained control over root certificates is required users can build their own configuration. So if this out-of-scope, let me know.

Sorry for the delay. I've come back to this, and wanted to fix up the merge conflict, and also had a change in thinking as to how to enable/disable. The new PR is at #2232.


add globbing to extra files

Example usage:
Before:
$ django-admin runserver_plus --extra-file=app/templates/base.html --extra-file=app/templates/user_form.html

After:
$ django-admin runserver_plus --extra-file=app/templates/*.html

or
$ django-admin runserver_plus --extra-file=app/templates/*.html --extra-file=app/templates/layouts/*.html

or even
$ django-admin runserver_plus --extra-file=app/templates/{*/,}*.html

ps.  Why am I watching HTML? crispy-forms.  There is a use case for this feature.

As a follow up suggestion:  create  a settings option for these extra watch files.

Nice, but consider quoting the glob pattern in the shell examples and the help (to be done): otherwise shells might complain, e.g. no matches found with Zsh (depending on options), because the shell will try to expand the glob (which included the option as a prefix).
And also with --extra-file foo/*/*.py it should not get expanded by the shell.
While at it, it would be nice to support templates/**/*.html (for all html files in all subdirs), but that's not possible with the standard lib IIRC, is it?!

As a follow up report, watching too many .html files reliably causes my computer to crash.  Not sure what's happening there, could be just my rig.  I'm just bringing this up as something to look out for.  It could have been the fact I  "ghetto forked" this library by copying and pasting the minimum amount into a command, so I could add the globbing.  This "ghetto fork" could have been part of the problem.

watching too many .html files reliably causes my computer to crash
According to the werkzeug changelog, it does not use inotify anymore, but probably is polling instead.

You could use strace -f to see what's going on, and that might be the issue.

@bionikspoon could you add a few lines to the documentation on using globs ?

@bionikspoon can you?


Handle resources DLLs

Resources DLLs don't have any AssemblyReferences and therefore won't resolve System.Object.
fixes #697

Thanks @chucker . Can you add a test for this? Most tests like this are easiest with some IL. We have examples in the test project.
Also, can you move the logic into GetSystemRuntimeAssemblyInformation?

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: chucker sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@twsouthwick I'm not too familiar with this area. Is there anything else we need to do in order to get this merged?


RetroAchievements added for PSP

Hi,
RetroAchievements were added for PSP with this PR:
https://github.com/RetroAchievements/rcheevos/pull/155
along with some other updates.
Can this be added to ES?
https://github.com/batocera-linux/batocera-emulationstation/tree/master/external/libcheevos
It's already added to batocera.linux with this PR:
https://github.com/batocera-linux/batocera.linux/pull/4825
Thank you

Commited. Works only with ISO file format. CSO & PBP formats are not supported.


Add config option for x-ray-like overlay rendering

Had an issue with shaders where I could see the overlay even through solid blocks, which was actually helpful for finding stray air pockets when mining. I'd like a configuration option to render the overlay through everything, to the same effect.

what


Responsive UI for Computer screen

As these are concept designs, I made a copy, so this doesn't change the original.
https://youtu.be/Tj4UFgpCjvY

I did it in my own fork this time are you proud of me

ok well I'm merging lol


Why does the next element after a spliced element in an array reuse the spliced element's scope?

This demonstration maybe overkill.
After clicking delete to splice an array, the next element that takes its place will actually re-use its scope which is an unexpected behaviour, IMO. http://plnkr.co/edit/dLRfdcQhyFDB2MYuTj1p?p=preview
I tried dumping the scope variable before and after the delete, and they both use the same scope id. Why  is this case? I thought all the children have their own scopes? So I would expect the next element to reference its original scope instead of the spliced element's scope.

This is the expected behavior as you are using track by $index


Match Cord - Cloudrip Mountain fieldMap array is uneditable [Python]

The array built at the beginning named fieldMap is uneditable, and even copying it to another array still locks it.  I'm trying to edit values within the array, and no error message pops up, but trying the exact same commands with a custom-made array of the same name confirms the fieldMap array is uneditable.
>>>>fieldMap[0][1] -> 'x' # value starts as x
fieldMap[0][1] = '.' # change value to .
>>>>fieldMap[0][1] -> 'x' # value is still x
map2 = fieldMap
>>>>map2[0][1] -> 'x'
map2[0][1] = '.'
>>>>map2[0][1] -> 'x'

Yes, because is a protected property.
map2 = fieldMap -- it's not a copy, it's a reference.
To make a copy you need use other methods.

Shouldn't we have an error message when trying to write into a protected property, though? How is it protected?

@nwinter You are right - it's not a property. fieldMap is the result of the unit's custom method getMap(). So I'm not sure it's a referee bug or the interpreter.

Oh!  @Kevitto you can't change lines of the array because they are strings and immutable. @nwinter should be an error in this case? If yes, then I think it's an editor/interpreter bug.


Update README.md

Minor corrections.

Coverage remained the same at 55.003% when pulling 6406ede8cf094364fb571b014cf63f35f7ff3c64 on kirlf:master into 4f52e83bb77f3c6bd5e465c3a97ad8c7dff65e37 on veeresht:master.

@kirlf Thanks for the fixes!


Navigation Menu Auto Resizing

I am in the process of converting an internal tool over to use Shadcn UI. With the navigation menu I'm seeing the drop down resize itself until it disappears. I've checked all the tailwind classes trying to find the culprit and can't seem to determine a cause. Am I accidentally applying a class or a prop to cause this?

Here's the code.
"use client";

import * as React from "react";
import { cn } from "@/lib/utils";

import {
  NavigationMenu,
  NavigationMenuList,
  NavigationMenuItem,
  navigationMenuTriggerStyle,
} from "@/components/ui/navigation-menu";
import Image from "next/image";
import Link from "next/link";
import logo from<EMAIL_ADDRESS>import {
  NavigationMenuContent,
  NavigationMenuLink,
  NavigationMenuTrigger,
} from "@radix-ui/react-navigation-menu";

import MaxWidthWrapper from "../MaxWidthcontainer";
import { useMsal } from "@azure/msal-react";

const tools: { title: string; href: string; description: string }[] = [
  {
    title: "Domain Tool",
    href: "/tools/domain",
    description: "Helper for looking up domain records such as A and MX",
  },
  {
    title: "Option 43 Helper",
    href: "/tools/opt43",
    description: "Generates Option 43 Strings for Sophos and other appliances",
  },
  {
    title: "Option 125 Helper",
    href: "/tools/opt125",
    description: "Generates Option 125 Strings for Mitel Phones",
  },
  {
    title: "SMDR Helper",
    href: "/tools/smdr",
    description: "Reads and converts SMDR logs to a human readable format",
  },
];

export default function Navigation() {
  const { accounts } = useMsal();

  return (
    <MaxWidthWrapper className="space-x-1 box-border shadow-md mt-2 mb-5 bg-white rounded-md">
      <NavigationMenu>
        <NavigationMenuList>
          <NavigationMenuItem>
            <Link href="/" legacyBehavior passHref>
              <NavigationMenuLink>
                <Image src={logo} alt="" width="70" />
              </NavigationMenuLink>
            </Link>
          </NavigationMenuItem>
          <NavigationMenuItem>
            <Link href="/" legacyBehavior passHref>
              <NavigationMenuLink className={navigationMenuTriggerStyle()}>
                Dashboard
              </NavigationMenuLink>
            </Link>
          </NavigationMenuItem>
          <NavigationMenuItem>
            <NavigationMenuTrigger className={navigationMenuTriggerStyle()}>Tools</NavigationMenuTrigger>
            <NavigationMenuContent>

                {tools.map((tool) => (
                  <ListItem
                    key={tool.title}
                    title={tool.title}
                    href={tool.href}
                  >
                    {tool.description}
                  </ListItem>
                ))}

            </NavigationMenuContent>
          </NavigationMenuItem>
          <NavigationMenuItem>
            <Link href="/" legacyBehavior passHref>
              <NavigationMenuLink className={navigationMenuTriggerStyle()}>
              </NavigationMenuLink>
            </Link>
          </NavigationMenuItem>
        </NavigationMenuList>
      </NavigationMenu>
    </MaxWidthWrapper>
  );
}

const ListItem = React.forwardRef<
  React.ElementRef<"a">,
  React.ComponentPropsWithoutRef<"a">
>(({ className, title, children, ...props }, ref) => {
  return (

      <NavigationMenuLink asChild>

{title}

            {children}

      </NavigationMenuLink>

  );
});
ListItem.displayName = "ListItem";

Hi @jonwgeorge ,
Just tried your code and everything seems to be fine.
Unfortunately, I'm not sure what's happening in <MaxWidthWrapper /> and I can't check the useMsal() hook.
Try removing them and see if the problem persists.

@iTsygancov <MaxWidthWrapper /> ensures all pages/views stay within our preferred screen size and useMsal() is authentication with our Azure backend. I had previously removed them and tested and it still has the same behavior. I was afraid this was something deeper. I'll keep poking around all the components on screen and see what I get. It's happening in all browsers on my test environment.

@jonwgeorge If anything, I'd be happy to help =)
Good luck!

@iTsygancov does this give any context? This is the resulting HTML after the menu has completely resized itself. I'm not seeing an issue that pops out at me.

  <div
    data-state="open"
    data-orientation="horizontal"
    class="origin-top-center relative mt-1.5 h-[var(--radix-navigation-menu-viewport-height)] w-full overflow-hidden rounded-md border bg-popover text-popover-foreground shadow-lg data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-90 md:w-[var(--radix-navigation-menu-viewport-width)]"
    style="
      --radix-navigation-menu-viewport-width: 0px;
      --radix-navigation-menu-viewport-height: 204px;
    "
  >

Domain Tool

              Helper for looking up domain records such as A and MX

              Option 43 Helper

              Generates Option 43 Strings for Sophos and other appliances

              Option 125 Helper

              Generates Option 125 Strings for Mitel Phones

SMDR Helper

              Reads and converts SMDR logs to a human readable format

@jonwgeorge I was able to reproduce your problem.
You are using some of the components that are imported directly from radix-ui library and mix them with components from shadcn-ui.
To fix your issue you can remove this import:
import {
  NavigationMenuContent,
  NavigationMenuLink,
  NavigationMenuTrigger
} from "@radix-ui/react-navigation-menu";

And update this import:
import {
  NavigationMenu,
  NavigationMenuItem,
  NavigationMenuList,
  NavigationMenuContent,
  NavigationMenuLink,
  NavigationMenuTrigger,
  navigationMenuTriggerStyle
} from "@/components/ui/navigation-menu";

If you still want to use <NavigationMenuContent /> component directly from radix-ui library than you can add absolute class to it and your problem will be resolved.
I hope that helped =)

@iTsygancov That was it! Thanks for the second pair of eyes!

I have the same issue but, I'm not importing components from the Radix library. Please any idea?
"use client"

import * as React from "react"
import Link from "next/link"

import { cn } from "@/lib/utils"
import {
    NavigationMenu,
    NavigationMenuContent,
    NavigationMenuItem,
    NavigationMenuLink,
    NavigationMenuList,
    NavigationMenuTrigger,
    navigationMenuTriggerStyle,
} from "@/components/ui/navigation-menu"
import { navigationMenu } from "./menu"
import { DropdownMenu, DropdownMenuContent, DropdownMenuTrigger } from "@radix-ui/react-dropdown-menu"
import { NavigationMenuViewport } from "@radix-ui/react-navigation-menu"

export function SuperMenu(props:any) {

    if(!navigationMenu || navigationMenu.length == 0) {
        return null;
    }

    return (
        <NavigationMenu {...props}>
            <NavigationMenuList>
                {navigationMenu.map( (menu , index) => {
                    const hasChildren = menu.children && menu.children.length > 0;
                    const width = index%2 ? '500px':'750px';
                    if(hasChildren){
                        return (
                            <NavigationMenuItem>
                                <NavigationMenuTrigger className="">{menu.title}</NavigationMenuTrigger>
                                <NavigationMenuContent className={cn("w-["+width+"]")}>

                                        {menu.children?.map(childmenu => {
                                            return (

                                                    <Link href={childmenu.href}>
                                                    <NavigationMenuLink>
                                                        {childmenu.title}
                                                    </NavigationMenuLink>
                                                </Link>

                                            )
                                        })}

                                </NavigationMenuContent>
                            </NavigationMenuItem>
                        )
                    }else{
                        return (
                            <NavigationMenuItem>
                                <Link href={menu.href}>
                                    <NavigationMenuLink className={navigationMenuTriggerStyle()}>
                                        {menu.title}
                                    </NavigationMenuLink>
                                </Link>
                            </NavigationMenuItem>
                        )
                    }
                })}
            </NavigationMenuList>
        </NavigationMenu>
    )
}

I tried everything, but same issue.

@noyessie when I gave the item wrappers ( in your code) the flex col css property, it worked for me


Event decoding for unnamed parameters is wrong

A solidity event like
event MyEvent(uint8, uint16, uint32, uint64, uint128);

is compiled into an artifact that has a list with these parameters with each one having an empty "" name. In ethabi's event decoding code Event::parse_log parameters are stored in a hash map of event name to token. This leads to every parameter using a clone of the same Token. In this particular case it can cause for example the uint8 to have a value that is larger than 255 because it uses the same token as the uint128.

This line  will cause the problem for indexed event parameters. Also this chains (topics_named_tokens.chain(data_named_tokens)) first named and then data tokens, which caused issues when I implemented some derive macros, in ethers-rs, what I did instead was:
filter the params in indexed and not indexed, then decode the all, but then but them in the correct order. The way this can work is:
let mut tokens = Vec::with_capacity(topic_tokens.len() + data_tokens.len());
for input in &event.inputs {
  if input.indexed {
      tokens.push(data_tokens.remove(0));
 } else {
      tokens.push(data_tokens.remove(0));
 }
}

I could fix this by copying this to Event::parse_log

@vkgnosis @nlordell are there any plans on fixing this? we hit that in Foundry as seen above recently

I don't remember the details of this after I reported it a year ago. We'd surely accept a PR that fixed this but no one is actively working on it.


some notes failed to open after updated to v.0.9.99

as said in title, some notes can not be opened and an error message popped up at top right conner says Failed to open ""
with make.md abled, the inline title of the note will be shown and then, the error popped.
After the error message appeared, to open the note, make.md must be disabled, uninstalled, shut up obsidian and then re-open obsidian.
Previous version of make.md has no such problem.
Tested and affirmed is caused by the plugin update.
console message shows below:

Thanks for reporting this, will be issuing a hot patch today

This has been patched, let me know if you continue to experience this issue

Thanks for the quick patch.
But it seems the problem still there, and I think I got the reason cause the problem.
if a note contains a double link in one line, like [[note one]][[note two]], will lead to error. however, if there a space between the double link, it is OK without any problems, like  [[note one]] [[note two]] will be OK.
Some my notes contains [[note 1]][[note 2]]. These notes can not be opened. When change to [[note 1]] [[note 2]], it is OK to open.

Thanks for tracking down the issue, it's patched in 0.9.992

Problems gone! Thanks a lot!


Restore filtering of test events in 2.5 upgrade

Need to restore the filtering of test events in the 2.5 upgrade.

It seems like this was already done in https://github.com/datamade/la-metro-councilmatic/commit/45256e00c0905899adfd54230bf5bc88c44eda58#diff-154e85485bdcc3826d208aa9dbc8d8e3R252, but we do need to restore bill filtering still!


Add an option to disable inlay hints for functions with a single parameter

Inlay parameter hints are useful when there are multiple arguments potentially with the same type. Functions with a single argument and often functions from the c++ standard library have meaningless names for their parameters and it just ends up as noise. For example

the c++ standard library begin and end functions are obvious what they do and the parameter name adds no information.
Introducing an option to disable inlay parameter hints for functions with a single argument would remove this noise

/extCpp


or separation in an event list

Failing grammar:
always_ff @(posedge CLK or negedge RST_X)  // <- or separation fails

Fixed v.0.1.2


Feature/add kotlin and convert models

I was just going to add kotlin support and convert the model classes but I got on a roll and converted everything except the MainActivity and the classes in the ui package. So far, 15% fewer lines of code. :)

LGTM! :+1:


WIP implement Service::connect

This change is

r? @vinipsmaker
(maidsafe_highfive has picked a reviewer for you, use r? to override)


tr: first implementation

My apologies. Fixed.

Post-merge bug found, please fix: https://github.com/rustcoreutils/posixutils-rs/issues/129


Define capability endianness

An implementation of Zcheri where endianness is switchable at runtime must use a fixed endianness for capabilities, since RISC-V requires byte sequences* to remain invariant across endian switches, and if an invariant byte sequence were reinterpreted with the opposite endianness it would result in a catastrophic loss of provenance.
Such an implementation must present a capability endianness opposite its integer endianness in one mode, so a requirement that capability endianness equal integer endianness is unsupportable in the general RISC-V setting.  Allowing the capability endianness to be implementation defined separate from the integer endianness would contradict the requirement of an architectural capability representation that can be inspected in byte dumps.
As such: Require that capability representation be little-endian on all systems, or only define Zcheri for fixed little-endian systems.

The precise wording of the requirement is that a byte written at some address will remain readable at this address.  Narrowly, this means that byte invariance is only required within a CLEN-bit region if the tag is zero, and an implementation could allow the capability endianness to appear to vary by reversing the bytes of CLEN-bit regions on loads and stores conditionally on a set tag.  This is unlikely to be microarchitecturally reasonable.

I agree that the representation needs to be fixed at runtime. Mandating little-endian seems like the most sensible thing to me. I'm not sure if Zcheri will ever exist on an bi-endian RISC-V implementation, but as a point of reference, Morello has the following constraints:
An implementation of the Morello architecture does not support the following:
• The AArch32 state.
• Mixed-endian at any Exception level.
• Fixed big-endian. The architecture only supports fixed little-endian

Also having worked with big-endian capabilities before (CHERI-MIPS), it is a lot nicer for software if the first quantity is the address as it allows code that simply fetches an address-size quantity (debuggers such as gdb/kernel backtracers/etc.) to work without additional changes.

Also having worked with big-endian capabilities before (CHERI-MIPS), it is a lot nicer for software if the first quantity is the address as it allows code that simply fetches an address-size quantity (debuggers such as gdb/kernel backtracers/etc.) to work without additional changes.

It’d have the right bytes for the address but they’ll be backwards. Only a true little-endian system lets you do that.

Also having worked with big-endian capabilities before (CHERI-MIPS), it is a lot nicer for software if the first quantity is the address as it allows code that simply fetches an address-size quantity (debuggers such as gdb/kernel backtracers/etc.) to work without additional changes.

It’d have the right bytes for the address but they’ll be backwards. Only a true little-endian system lets you do that.

Agreed, I should have been more clear on my comment that I meant little-endian caps on little-endian systems vs big-endian caps on big-endian systems.

I'm happy with little-endian only like Morello


Add proto schema for resources instrumentation

This PR adds the proto definitions that can be used to in
the instrumentation of resources. The main idea is that each
resource_poll operation will be instrumented as a separate span
plus event(s) representing the result of the method invocation.
related: #40
Signed-off-by: Zahari Dichev<EMAIL_ADDRESS>

Closed in favor of #77


Provide trace context tracking in google-cloud-core

We move TraceContext tracking into google-cloud-core. (Currently this functionality is partially implemented in google-cloud-logging.) I want to centralize it because the Trace instrumentation work requires that multiple services (currently trace and logging, but likely to include the other Stackdriver services in the future) be able to cooperate in managing this context data.
This introduces an object for TraceContext. It also introduces a common algorithm for obtaining and memoizing TraceContext in the Rack environment so that different services will see a consistent state for the same request.
I bumped the google-cloud-core version to 0.22 because when I modify google-cloud-logging and google-cloud-trace to use this mechanism, they will be dependent on this change and thus require a new minor release of those gems. I intend to release google-cloud-core 0.22.0 first, and then later when I've made downstream dependent changes to other gems I'll bump their versions.

Will create a separate gem for this, per offline conversation with @blowmage


update examples in readme to quote filenames

Without this it fails if there is a space in the file path.

Hello,
I have bad experience with getting quoting working on multiple platforms. Have you tested these work on macOS, Linux and Windows shells?

Linking to https://github.com/okonet/lint-staged/pull/875 because it introduces an escape helper for these situations.

Hmm I've only tried it on mac, but https://www.npmjs.com/package/string-argv understands the quotes.
See this example


Fix links in Ethics

https://smo-wiki.leibniz-hbi.de/Research-Ethics-and-Law
Needs proper Markdown URLs to work in "rendered" version.

Fixed. It's enough to have links in double square brackets for them to appear properly in the rendered wiki: like this
[[https://example.com]]


Validation is not working if form is not filled

Hi ,
I am using brutusin:json-forms in my project to create form dynamically. But I have found one issue with validation. If form is empty then validation is not working for below json
JSON Data=>
{
"title": "Person",
"type": "object",
"properties": {
"firstName": {
"type": "string"
},
"lastName": {
"type": "string"
},
"age": {
"description": "Age in years",
"type": "integer",
"minimum": 0
}
},
"required": ["firstName", "lastName"]
}

Please help me to fix this issue.

if empty,the required fields are not okay,so it can't validate ?!? i didn't understood?

Validation is not working if no fields are filled in the form. the form is generated through above JSON schema in http://brutusin.org/json-forms/.

Hi,
Any Update ??

just happened it to me, fixed!
just find that line
if (parentObject[prop] !== null) {
replace with
if (parentObject[prop] == null) {
and it will work,
cheers!


Update vpp version

https://github.com/networkservicemesh/govpp/blob/main/Dockerfile#L1
TODO

Update vpp to the latest main version. Because it contains commits that were not merged into the latest release (af_xdp for example)
The work with memif interfaces (abstract sockets) has been changed. We need to figure out how to configure it now.
It makes sense to update git.fd.io/govpp.git to the latest release (https://github.com/FDio/govpp). This can help with unstable calico-vpp tests. (Calico-vpp updates govpp regularly)
Pass cmd-forwarder-vpp docker-tests
Pass integration-tests

We have used the artgl/cmd-forwarder-vpp:vpp_c3f505fe7b7f image in our tests. The AF_PACKET related issue described in VPP-2081 occurred again but this time right after deployment. Forwarder restart has not improved the situation but made it worse since more forwarders started suffering from the same problem.

Do you have SELinux enabled? Could you try to disable it and test if the issue appears again?

@szvincze I think we might need to check other versions of forwarder-vpp. It could help to detect problems.
v1.9.0
v1.8.0
v1.7.1
v1.6.2

If you will have a chance check the problem with these versions ☝☝☝

@szvincze @ljkiraly
I have a few questions:

I'm not sure that SLES is fully supported in VPP - https://s3-docs.fd.io/vpp/23.10/aboutvpp/supported.html
Is it possible to check the problem on openSUSE?

I have rebuilt the forwarder-vpp with af-packet v3 - artgl/cmd-forwarder-vpp:vpp_c3f505fe7b7f_af_packet_v3
Please take a look

Can you please tell if the problems are observed on the NSC interfaces? How many clients (approximately) are required?

Additional questions:

Could you please share your configurations? How do you run multiple forwarders? Are they on the same node?

Could you share your environment? Is SLES a node or container operating system?

Hello,
SELinux is disabled on the impacted cluster.
Various versions cannot be easily tested do to limitation on hardware and manpower (packaging a given version is not necessary trivial and the testing cycle is roughly one build per day, sometimes two).
@glazychev-art

Yes, the issue was there with OpenSUSE as well.
We are working on testing this.
I think we have observed it on both the nse and nsc sides. Up to 10 clients.
There was reproduction with Ubuntu base a month or so ago.
We need to check the MTU settings.
VPP is deployed as a DaemonSet. One pod on each eligible k8s nodes. Then on top of NSM/VPP we use Meridio that utilizes the extra networking and creates a complete NSC-NSE cross connect to handle higher level load balancing (eg. sticky session like setup).
The issue manifests on a "CNIS" cluster. This is a bare metal k8s setup using Ericsson's CCD. SLES is used as operating system on bare metal compute nodes and those are joined into the k8s cluster. Then by default NSM/VPP/Spire components are packaged with SLES being the base Docker image.

Best Regards,
Gábor Barna

Hello,
The issue was reproduced on a relatively new kernel: 5.14.21-150400.24.63-default. Search for 5.14.21-150400.24.63 on the page.
Best Regards,
Gábor Barna

Thanks @robaganrab !
If you have a chance, could you also check this forwarder image: artgl/cmd-forwarder-vpp:vpp_c3f505fe7b7f_no_calico_af_packet_v3
Also, when you see the problem, please attach the command output from vpp: show hardware-interfaces

Hello, @robaganrab , @szvincze
I'd like to suggest three ways for diagnostic the problem.

test  artgl/cmd-forwarder-vpp:vpp_c3f505fe7b7f_no_calico_af_packet_v3
I think we could also do some diagnostics (before/after reproducing)

2.1. vppctl show int
2.2. vppctl show hardware
2.3. vppctl trace add af-packet-input 1000 (do it once)
2.4. vppctl show trace (after reproducing)

And also as  @edwarnicke pointed  we could try to disable tap and try to use veth interfaces (just cut this and build forwarder https://github.com/networkservicemesh/sdk-vpp/blob/main/pkg/networkservice/mechanisms/kernel/client.go#L36)

Hello,
We managed to get everything built and set up for test run with artgl/cmd-forwarder-vpp:vpp_c3f505fe7b7f_no_calico_af_packet_v3. The test will include the vppctl commands and hopefully will provide useful output.
Best Regards,
Gábor Barna

Hello @denis-tingaikin and @glazychev-art,
Test results with artgl/cmd-forwarder-vpp:vpp_c3f505fe7b7f_no_calico_af_packet_v3 is quite good, the AF_PACKET issue wasn't seen during the test run.
What is the way forward? Which govpp is the base of this image? Or the sha  c3f505fe7b7f is a vpp commit ID? We would like to build a SUSE based image from this version of vpp/govpp.
Regards,
Laszlo

Hello @ljkiraly
It's a nice new!
We're used this patch https://github.com/networkservicemesh/govpp/pull/11
Its based on this commit from vpp https://gerrit.fd.io/r/gitweb?p=vpp.git;a=commit;h=c3f505fe7b7fbecb35494863a6c9de3cad6e6d2d

@denis-tingaikin, @glazychev-art: There were more attempts with the image that Artem provided and the issue did not come at all. So, it seems to be fine.
However as @ljkiraly mentioned above there was a test with a newly built image based on SUSE and the issue happened immediately. We will give it another try with an openSUSE based image as well to see if SUSE is the culprit or openSUSE is also suffering from the same thing. We will keep you informed.

@szvincze
Got it, thanks!
Did you use this PR to build your image - https://github.com/networkservicemesh/govpp/pull/11?
Now there are two main suspects:

Calico-vpp patches
You are using tap interfaces, not af_packet.

Therefore, it would be very cool to see vppctl show hardware-interfaces output from your forwarder (on failed tests)

@glazychev-art: Yes, we used that PR for building the image.
For the next trial I requested the output you mentioned.

Hi @glazychev-art,
By calico patches you mean: patch/0004-capo-Calico-Policies-plugin.patch?
Can you elaborate how was the tap support removed? This would help us to build an openSUSE/SUSE based image similar to yours.
BR/Laszlo

@ljkiraly
I mean this patches:
https://github.com/networkservicemesh/govpp/blob/5482a9ac8fba3ab4cfc26b2142c67b8da0d671b8/patch/patch.sh#L15-L22
There are 1 cherry_pick and 5 files
By the way, I've prepared new PRs today (they contain the latest main vpp version) and removed the calico-patches in one of them:
https://github.com/networkservicemesh/govpp/pull/12
https://github.com/networkservicemesh/govpp/pull/13
You can use it
To disable tap interfaces just delete this lines:
https://github.com/networkservicemesh/sdk-vpp/blob/main/pkg/networkservice/mechanisms/kernel/client.go#L35-L37
and
https://github.com/networkservicemesh/sdk-vpp/blob/main/pkg/networkservice/mechanisms/kernel/server.go#L35-L37

I think we can close the issue, because we merged all PRs
The discussion was moved to - https://github.com/networkservicemesh/cmd-forwarder-vpp/issues/927


booster_bdd: Include runtime version into BOOSTER_RUNTIME variable.

Can one of the admins verify this patch?

[test]

[test]


Apache Feather Pandas Inclusion?

Is there any possibility to consider including feather as built in PANDAS I/O module?
https://blog.rstudio.org/2016/03/29/feather/

duplicate of #13092


0.9.91-alpha closes and re-opens the app when doing LoginWithBehavior with LoginBehaviorWebViewOnly

Issue is described here
http://stackoverflow.com/questions/28429483/facebook-client-0-9-91-alpha-is-a-step-backwards-for-logging-in-to-fb

Replied on stackoverflow already. Closing.


How to save spatial understanding results ?

After scanning the room and trying to "understand" it, is there a way to save the result of all this "understanding" process (on the device,locally) in order to avoid rescanning/understanding again at each launch ?

I'm also interested in this.
It should be possible, if i'm not wrong Asobo is doing something like this in Fragments.
If my memory is correct, they are even warning the user when the differences between saved and current room are to big.

Closing issues older than 180 days. If this is still an issue, please reactivate with recent information.


Fix: infinite while loop by removing the try except wrapper in the task report

There was a while infinite loop, caused by a double try-except in evaluate.py. One in bench is enough.

Member

Only reason I can think of is if you run this function on its own? Otherwise there is the other try/except that takes care of this?


Site only responds without 'www' in the URL

no response: http://www.hotsapi.net/
functional: http://hotsapi.net/
this is something that needs to be set on the back end, either through your domain registrar or web host

I just didn't add "www" record in DNS. Also I'm not sure that my SSL certificate will work (technically it's a subdomain and I don't have a wildcard cert).
What is the reason to add www anyway? I always thought it's a legacy practice and it's better to just drop it.

you should be able to easily forward any requests to the 'www' to the version without so that the SSL cert works. as for the 'www' itself, i wouldn't say it's a legacy practice. automatic link parsing picks up on it, and many people will still type 'www' before a web address out of habit.
i actually opened this issue because i typed www.hotsapi.net in a reddit comment and someone responded saying it doesn't work (and i was too lazy to type 'http://' on my phone)

you should be able to easily forward any requests to the 'www' to the version without so that the SSL cert works

How does that work? SSL handshake is made before any HTTP requests so it encounders domain mismatch errors before it could ever receive a redirect. CNAME DNS enrty will not work for this too.

my bad, you are absolutely correct. I think you are right about needing this as part of the SSL cert. Here's a resource I found: https://blog.dnsimple.com/2016/08/https-redirects/

Most certificate providers allow for you to secure both www and non-www with one certificate, meaning you do not need to purchase multiple certificates nor do you need a wildcard certificate, however, you cannot retroactively add an additional domain to a certificate. You will need to either purchase another certificate, or you could switch to a free certificate authority like Let's Encrypt. Paying for simple DV SSL certificates nowadays is unnecessary as Let's Encrypt provide everything you need.
The ideal configuration is one where all domains that a user might expect to exist, exist, and they redirect to your chosen domain, meaning you should pick one of the following and redirect the rest to your choice.

[ ] https://www.hotsapi.net
[ ] https://hotsapi.net
[ ] http://www.hotsapi.net
[ ] http://hotsapi.net

There are some situations in which you need to allow your service to be accessible over http (legacy client access for example) but in most situations, you'd want to only provide your service over https for security.

Another vote for Let's Encrypt. Some languages provide autocert packages that will automatically generate free certs for you (https://letsencrypt.org/docs/client-options/).

+1 for let's encrypt. It's super easy to set up, see:
https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-16-04
or
https://www.digitalocean.com/community/tutorials/how-to-secure-apache-with-let-s-encrypt-on-ubuntu-16-04

added redirect from www domain


Drop the org.apache.tomcat:catalina dependency

Fixes strongbox #1565
Sorry for late PR, spent some time trying to make the strongbox-web-integration-tests work on my local environment. I couldn't find any catalina dependency in the strongbox project so only update other two projects. The tests all passed successfully on my machine after dropping this dependency.
Task

[x] Remove this from the strongbox-proto-parent project.
[x] Remove this from the strongbox-parent project.
[x] Remove this from all pom.xml files in the strongbox project.

Acceptance Tests

[x] Building the code with mvn clean install -Dintegration.tests still works.
[x] Running mvn spring-boot:run in the strongbox-web-core still starts up the application correctly.
[x] Building the code and running the strongbox-distribution from a zip or tar.gz still works.
[x] The tests in the strongbox-web-integration-tests still run properly.

Hi!
Thank you for your contribution!
Would you mind signing the ICLA, as described in the Contributing page?
Also, please, feel free to join our chat channel, if you'd like to learn more about the project and/or like to find out what else you could help with.
Kind regards,
Martin

Hi Martin,
I've mailed you my ICLA, please check, thanks!

Thanks for signing the ICLA!
@steve-todorov : Please, merge!

Thank you! Congratulations on your first pull request! :)


Llama  'tuple' object has no attribute 'max_seq_length'

Some basic example code using LLama3 from 4bit from Unsloth HF repos:
model = FastLanguageModel.get_peft_model(
    model,
    r = 32,
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj",],
    lora_alpha = 64,
    lora_dropout = 0,
    bias = "none",
    use_gradient_checkpointing = "unsloth",
    random_state = 3407,
    use_rslora = False,
    loftq_config = None,
    max_seq_length=6144,
)

File /usr/local/lib/python3.10/dist-packages/unsloth/models/llama.py:1617, in FastLlamaModel.get_peft_model(model, r, target_modules, lora_alpha, lora_dropout, bias, layers_to_transform, layers_pattern, use_gradient_checkpointing, random_state, max_seq_length, use_rslora, modules_to_save, init_lora_weights, loftq_config, temporary_location, **kwargs)
   1614 SUPPORTS_LOFTQ  = "loftq_config" in signature
   1615 SUPPORTS_RSLORA = "use_rslora"   in signature
-> 1617 assert(max_seq_length <= model.max_seq_length)
   1619 if lora_dropout != 0:
   1620     logger.warning_once(
   1621         f"Unsloth: Dropout = 0 is supported for fast patching. You are using dropout = {lora_dropout}.\n"\
   1622         f"Unsloth will patch all other layers, except LoRA matrices, causing a performance hit."
   1623     )

AttributeError: 'tuple' object has no attribute 'max_seq_length'

Freshly installed using pip install "unsloth[cu121-ampere-torch211] @ git+https://github.com/unslothai/unsloth.git"

Apologies on the long delay - I'll take a look!


[C++] Pass the request by reference

This eliminates a bunch of unnecessary copying of fields like headers, payload, etc.

thanks!


az image copy does not work with dev/test labs

The az image copy command works, but the custom image never shows up in the target Azure dev/test lab, even though it is in the resource group alongside other images.  Please enable the ability to copy images between Azure Dev/Test labs.

image copy.

@qwordy Do you have any updates on this issue.


[Feature Request] Show the External IP in the logs when going through a reverse Proxy

Description
When using a reverse proxy logging shows the internal docker ip instead of the External IP
Example:
2022-06-20T09:52:17.637Z [warn][API]: Failed sign-in attempt using invalid Overseerr password {"ip":"::ffff:<IP_ADDRESS>","email":"fakeuser"}
Desired Behavior
2022-06-20T09:52:17.637Z [warn][API]: Failed sign-in attempt using invalid Overseerr password {"ip":"::ffff:<IP_ADDRESS>","email":"fakeuser"}
Additional Context
No response
Code of Conduct

[X] I agree to follow Overseerr's Code of Conduct

Non Issue, Didn't set proxy support under General Settings.


RESET button is stenciled Reboot. [nit]

ALL GOOD!
...
"RESET button on bottom of the Gateway."
There is no button labeled reset, do you mean Reboot

Are you referring to something in UI? What exactly?

The last screen in the configuration setup.
index.html line 564
section id="thankyou"

This one? Thanks for running the simulator on a public server btw.

The correct wording would be If you want to modify the settings, press and hold CONFIGURE button on bottom of the Gateway for at least five seconds.
@TheSomeMan ^

Revised title

@ojousima , what is the correct translation of this phrase in Finnish? "If you want to modify the settings, press and hold CONFIGURE button on bottom of the Gateway for at least five seconds."

@TheSomeMan "Jos haluat muokata asetuksia, paina ja pidä pohjassa vähintään viisi sekuntia CONFIGURE-näppäintä gatewayn takapuolella"

Thanks


Using Verify  with Prism's NavigationService and NavigationParameter

I'm trying to verify that a navigation method on the NavigationService is called exactly once with the following code:
var navigationService = new NavigationServiceMock();
var viewModel = new CatalogPageViewModel(navigationService);

// Call
viewModel.CreateNewItemCommand.Execute(null);
var item = GetLatestCreatedItem();
var parameters = new NavigationParameters();
parameters.Add("id", item.Id);

// Verify
navigationService.Verify(x => x.NavigateAsync("EditPage", parameters)).WasCalledExactlyOnce();

However it yields the following exception:
PCLMock.VerificationException : Verification that NavigateAsync(It.Is("EditPage"), It.Is(?=id=89003df7-36a0-4319-8299-1195f3b7b5de [Prism.Navigation.NavigationParameters])) was called exactly once failed because it was called 0 times.
 at
PCLMock.VerifyContinuation.ThrowVerificationException (System.String format, System.Object[] args) [0x00011] in
....

I've added breakpoints and the NavigateAsync sure is being hit with the exact same parameters, so I assume it fails because it expects the parameters to be an exact object and not another instance of the same class.
I would like to verify that a NavigationParameters is passed to the NavigateAsync and that the NavigationParameters should have a id key with the value of the Item.Id.
Is this possible?

It.Is (which is what is automatically used if you just pass in arguments directly, like "EditPage" and parameters in your code) uses object.Equals so overriding Equals in your NavigationParameters may be one option if it's under your control. You could also use It.Matches instead.

I love it, thanks! ❤
var parameters = new NavigationParameters();
parameters.Add("id", item.Id);

navigationService
  .Verify(x => x.NavigateAsync("EditPage", It.Matches<NavigationParameters>(y => y.ToString() == parameters.ToString())))
  .WasCalledExactlyOnce();


add "catalog.xml" to thredds url if missing

... just for convienice to use a thredds url like: http://example.com/thredds

added in pull request #4


Make inspector a single stack of NEW + DEFAULT content, with scrolling

Right now, we have the new Inspector replacing the old one in the DOM, and then a switch to get to the old one. Since the new one will be empty for most things most of the time, a much better UX would be to insert the new Inspector content inside the old content, below the object title and ID:
Right in here:

Need to see how that element is structured and if we can change our Javascript to insert there.

In the new code and a PR coming right up, the new inspector appears above the "classic" OSM inspector, and the classic inspector is not hidden.

In the new code and a PR coming right up, the new inspector appears above the "classic" OSM inspector, and the classic inspector is not hidden.


OSBS-6224 reject COPY --from

Since pulling and updating the image in this construct is complicated, disallow doing so. There's a workaround if someone really needs this.

Thanks for reviews; should be ready for another.


List minecraft saves in a folder

It would be nice to have an utility to list all the minecraft worlds and return the corresponding data. There doesn't seem to be any built-in functionality in this package apart from the nbt module. If you want, I can implement it here.
Edit:
Oh wait, I didn't see the docs for @xmcl/world. It would still be nice, however, to have an utility to list the saves.

Yes, there is not pure listing function, but that can be simple, just scan the folders under the saves, and if you want, you can check if level.dat exists.

I can open a PR for this if you want. Or if you don't want this, just close this issue.

Thanks. I'll consider these kind of high level API later if I have more time.


[stable/minio]: fix extraArgs values being ignored by minio binary inside pod

What this PR does / why we need it:
Fix extraArgs to be on same line as docker-entrypoint.sh, else the arguments will be ignored inside the pod.
Improvement for PR #19904
Which issue this PR fixes

fixes #20089

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Title of the PR starts with chart name (e.g. [stable/mychartname])

Can we get any traction here?

Would like this fix as we currently have to fork our own chart just as a workaround..

+1, really need this PR merged...

Trying again, only change since last time is the chart version which has been bumped to 5.0.15.
/assign @krisis

/lgtm

How can we merge this request?
I tried to fix the DCO signoff, but it removed the lgmt label.
Can someone point me to the right direction how to fix the dco signoff?

/lgtm

/ok-to-test

How do we go on from here?  Apparently this fails because the Chart version got bumped on master in the meantime.  And @hutr has been inactive lately.

Please bump the version in this PR @bad

I will bump the chart version to 5.0.20

You have multiple things still missing in this PR @hutr - the DCO is not complete.

You have multiple things still missing in this PR @bad - the DCO is not complete.

@harshavardhana : @hutr is updating his pull request.  You need to mention him.

/retest

/ok-to-test

@hutr you haven't completed DCO without that we can't merge this PR to helm repo.

Check what do to do here https://github.com/helm/charts/pull/20091/checks @hutr

You have multiple things still missing in this PR @bad - the DCO is not complete.

@harshavardhana : @hutr is updating his pull request. You need to mention him.

Regarding DCO, i forgot to sign the first commit i did back in january. Not sure how to fix this. Any pointers?

Regarding DCO, i forgot to sign the first commit i did back in january. Not sure how to fix this. Any pointers?

Squash all commits locally sign them into one commit

Regarding DCO, i forgot to sign the first commit i did back in january. Not sure how to fix this. Any pointers?

Squash all commits locally sign them into one commit

Or do an interactive rebase and reword the first commit

Regarding DCO, i forgot to sign the first commit i did back in january. Not sure how to fix this. Any pointers?

Squash all commits locally sign them into one commit

Or do an interactive rebase and reword the first commit

Thanks guys, DCO is fixed now

/ok-to-test

/lgtm

Regarding DCO, i forgot to sign the first commit i did back in january. Not sure how to fix this. Any pointers?

Squash all commits locally sign them into one commit

Or do an interactive rebase and reword the first commit

Thanks guys, DCO is fixed now

Thank you guys for handling this so quickly.


[mupx] Deployment Failed: npm ERR! fetch failed https://registry.npmjs.org/fibers/-/fibers-1.0.5.tgz

mupx setup completed successfully. However, when I try to run mupx deploy
$ mupx deploy

Meteor Up: Production Quality Meteor Deployments
------------------------------------------------
Configuration file : mup.json
Settings file      : settings.json

Meteor app path    : /bar/foo-app
Using buildOptions : {}

Started TaskList: Deploy app 'foo-app' (linux)

[foo-app] - Uploading bundle
[foo-app] - Uploading bundle: SUCCESS
[foo-app] - Sending environment variables
[foo-app] - Sending environment variables: SUCCESS
[foo-app] - Initializing start script
[foo-app] - Initializing start script: SUCCESS
[foo-app] - Invoking deployment process
[foo-app] - Invoking deployment process: SUCCESS
[foo-app] - Verifying deployment
[foo-app] x Verifying deployment: FAILED

        -----------------------------------STDERR-----------------------------------
        s some small
        npm WARN deprecated backwards-incompatible changes made to `npm run-script` and
        npm WARN deprecated semver behavior.
        npm WARN package.json<EMAIL_ADDRESS>No description
        npm WARN package.json<EMAIL_ADDRESS>No repository field.
        npm WARN package.json<EMAIL_ADDRESS>No README data
        npm ERR! fetch failed https://registry.npmjs.org/fibers/-/fibers-1.0.5.tgz
        npm ERR! fetch failed https://registry.npmjs.org/promise/-/promise-7.0.4.tgz
        npm ERR! fetch failed https://registry.npmjs.org/meteor-promise/-/meteor-promise-0.5.0.tgz
        npm ERR! fetch failed https://registry.npmjs.org/semver/-/semver-4.1.0.tgz
        npm ERR! fetch failed https://registry.npmjs.org/source-map/-/source-map-0.1.32.tgz
        npm ERR! fetch failed https://github.com/meteor/node-source-map-support/tarball/1912478769d76e5df4c365e147f25896aee6375e
        npm ERR! fetch failed https://github.com/meteor/node-eachline/tarball/ff89722ff94e6b6a08652bf5f44c8fffea8a21da

        => Redeploying previous version of the app

        -----------------------------------STDOUT-----------------------------------

        To see more logs type 'mup logs --tail=50'

        ----------------------------------------------------------------------------

Tried to increase "deployCheckWaitTime": 240 which unfortunately did not fix the issue.
Any ideas?

Same problem. Do you have a MONGO_OPLOG_URL in your mup.json? That actually started causing the problem for me. If I don't have that environmental variable set it work...Doesn't really solve the issue though.
Similar to #841

@sscaff1 No I don't have MONGO_OPLOG_URL in mup.json...I have also tried your recommendation with no luck... :( Thanks tho

@derek-fong what version of node are you using? Make sure it's not 6+. I installed nvm in order to use node 4.4.5 when I deploy.


Add neutron-network triggers for sles12 nodes.

I will close this PR because both triggers are now integrated directly in Jenkins by @bmwiedemann
Thanks!


[ZEPPELIN-328] Interpreter page should clarify the % magic syntax for interpreter group.name

Currently the Interpreter page like the interpreters as
hive %hql
However, this does not work unless hive is the default group - otherwise one would require the full %group.name.
It seems it would be better to list interpreter as %group.name on the page.

Before

After

Others

Any comment is welcome

If I remember well (from last time I looked in that part of the code). Its not mandatory for an interpreter to have a group. The tutorial https://zeppelin.incubator.apache.org/docs/development/writingzeppelininterpreter.html didn't even mentioned it. So if settings.name is indeed set, in the case of %angular and %md it would be better to not have the group

I guess it's not required but the name is used as the group if absent
Interpreter.java
  public static void register(String name, String className) {
    register(name, name, className);
  }

  public static void register(String name, String group, String className) {
    register(name, group, className, new HashMap<String, InterpreterProperty>());
  }

  public static void register(String name, String group, String className,
      Map<String, InterpreterProperty> properties) {
    registeredInterpreters.put(group + "." + name, new RegisteredInterpreter(
        name, group, className, properties));
  }

Agreed if there is only one interpreter in the group then it's better to have %sh instead of %sh.sh
Ideally we should know if the interpreter is the default of its group in that case we could list %spark, %spark.pyspark, %spark.sql, %spark.dep?

Better?

If the rule is to not show interpreter.name for the first,  wouldn't it show %hive instead of %hive.hql?
could be better to have: if the group name = interpreter name, just show the interpreter name
But it's my guess since I don't understand well the problem of default group

I think that is in fact correct - you could run the default interpreter in any group with %group but not %name - ie. %hive works but %hql does not (it fails with hql interpreter not found error)
Also in such case group name != (default) interpreter name

I see, didn't expect %hive not to work, better always use %hive.hql than %hive then?

I think you mean %hql (interpreter name) not working.
Using %hive (hql is the default interpreter) or %spark (spark/scala is the default interpreter) seems fine to me...

any more feedback on this?

I guess the question is:
What is the rule to show the %group.%name
Removing the %name for the first element of each interpreter doesn't sound that reliable.
Like you said before:

Agreed if there is only one interpreter in the group then it's better to have %sh instead of %sh.sh
Ideally we should know if the interpreter is the default of its group in that case we could list %spark, %spark.pyspark, %spark.sql, %spark.dep?

To me it should be: if %group = %name we only show %group.
I'm not sure there is a default settings, or at least we don't receive any from the server

I looked into this more.
The default interpreter for a given group is the first interpreter:
https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/NoteInterpreterLoader.java#L131
      // next, assume replName is 'group' of interpreter ('name' is ommitted)
      // search interpreter group and return first interpreter.
      for (InterpreterSetting setting : settings) {
        intpGroup = setting.getInterpreterGroup();
        Interpreter interpreter = intpGroup.get(0);

In such case the name can be omitted and only the group is required.
I think this matches the behavior here.
It might be better if there is a way from a group of interpreters to decide which is the default interpreter rather then the first one in ZEPPELIN_INTERPRETERS (https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java#L396), but that might over complicate the design.

Please take a look https://issues.apache.org/jira/browse/ZEPPELIN-74 about omitting 'group' or 'name'.
If zeppelin can display 'group' and 'name' in a way user can learn omit rule as well, that would be really cool.

Updated to handle default group. My understanding is the default group is the first interpreter group from config.
Here's what it looks like:

Why isn't %pyspark -> %spark.pyspark
Or %ignite -> %ignite (default) ?

because when spark is the default group, you can run pyspark by simply putting %pyspark% at the beginning of the paragraph. This is my understanding of @Leemoonsoo feedback - and matches the currently implemented omit rules.
(default) here is the default interpreter of the default group, which means you can omit %spark or %spark.spark and the paragraph is will run as Spark Scala. If there is a better way to represent that, please let me know.

Okay, got it

any more thoughts on this?

No, it's good to go.
I will be merging if no more comments.


Add a big ole edit button on every page

There are so many scenarios where a typo could be edited in Github and a PR raised, all that is needed is a button along the lines of "Edit this Page on GitHub".
Is there any opportunity to add this to every /web page somehow and some where?

There is a "View Page Source" link in the footer, but it simply links to the WebFundamentals project on GitHub, not the actual source of the page. Finding the actual page can take a little bit of time, unless you do a Find file. For example, the path to the "Offline Cookbook" article Markdown is
src/content/en/fundamentals/instant-and-offline/offline-cookbook/index.md
So it would definitely be great to have an "Edit this page" button, which stands out and links directly to the page Markdown.

Should be do-able

We used to have it, but iirc we don't control the footer..
On Feb 24, 2018 04:18, "Kayce Basques"<EMAIL_ADDRESS>wrote:
Should be do-able
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/WebFundamentals/issues/5789#issuecomment-368127441,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACxxvUxTHBunWqEQHLE2yVHZJTSMEU0ks5tXx0igaJpZM4SFu4E
.


Require argparse

The egg built requires argparse whereas it's useless in Python >= 2.7.

It's left over from supporting 2.6 earlier. Will remove it when hit the repo next. Thanks.


Method 'id' must be implemented by CompUnit::Repository because it is required by a role

$ prove -e perl6
t/01-basic.t .. 1/7 Method 'id' must be implemented by CompUnit::Repository because it is required by a role
  in any compose_method_table at gen/moar/m-Metamodel.nqp line 2801
  in any apply at gen/moar/m-Metamodel.nqp line 2811
  in any compose at gen/moar/m-Metamodel.nqp line 2984
  in any make_pun at gen/moar/m-Metamodel.nqp line 1683
  in any find_method at gen/moar/m-Metamodel.nqp line 1711
  in method find at C:\rakudo\share\perl6\site\sources\B669D8030E71BA9B7E30E7C0A830FE5471B285EE line 10
  in sub find-lib at C:\rakudo\share\perl6\site\sources\14D26D794E5727F370995EF0603546A1142B2D81 line 12
  in sub guess_library_name at C:\rakudo\share\perl6\sources\075EFE4B4CDAAF73190194EA876F81A1F128D1A2 line 175
  in method setup at C:\rakudo\share\perl6\sources\075EFE4B4CDAAF73190194EA876F81A1F128D1A2 line 225
  in method CALL-ME at C:\rakudo\share\perl6\sources\075EFE4B4CDAAF73190194EA876F81A1F128D1A2 line 241
  in block <unit> at t/01-basic.t line 11

# Looks like you planned 7 tests, but ran 1
t/01-basic.t .. Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 6/7 subtests

Test Summary Report
-------------------
t/01-basic.t (Wstat: 65280 Tests: 1 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 7 tests but ran 1.
Files=1, Tests=1,  2 wallclock secs ( 0.05 usr  0.09 sys +  0.00 cusr  0.05 csys =  0.19 CPU)
Result: FAIL

This line, apparently.
Might also be a problem with Find::Bundled, in which case the relevant line is here.

Closing because this is a problem with Find::Bundled.


fix: separate rule for RCE detection in User-Agent header

Rule 932200 detects RCEs in the User-Agent header (apart from ARGS etc.). Because the User-Agent header is essentially a free-form field, 932200 produces many FPs with common user agents.
Move the RCE detection in the User-Agent header to a separate rule, akin to the special detection already implemented for the Referer header. Only target URL-like values in the User-Agent, as any other detection is too generic (e.g., the equivalent of rule 932206 doesn't work for the User-Agent header).
Fixes #3163

Is it worth it to create a new rule for this?

I'm not sure, TBH. This rule would only check things that look like a URL at the beginning of the string, which is somewhat unlikely. From my perspective we could also drop the rule. As I wrote in the commit message, running the full detection of 932200 on the User-Agent header leads to many FPs and I don't really know how to reduce that number, since that header is pretty much free-form.

Luckily, 93220x are not part of the Log4Shell defences, so I' leaning towards dropping the rule in favor of less FPs. We can always improve on strictness later on.

Yes, let's drop this if it only means trouble.


Memory leak/memory management continuous push

We have an app that a couple of times every minute updates a document of around 100kb. No attachments. We then have continuous sync on. After a couple of hours the app runs out of memory, freezes, crashes, can't open more cursors... A bit different behavior depending on app history apparently. Also if there are a lot of changes to be synced, the app can't even be restored after restart. Just opening it can find more than around 1000 changes and then runs out of memory, before able to sync a single record.
Made the following test code that can reproduce the error depending on the sleep time between writes and the servers response time. http://pastebin.com/3mBsENy1
Expected behavior would be to only sync the last revision or at least not trying to load all non synced revisions into memory at the same time, as it looks now, and therefore completely fail with the sync forever until app reinstall.
Client: both couchbase-lite-android.1.0.4 and couchbase-lite-android:1.1.0-532
Server: Both tested with Sync Gateway 1.0.3 and master(bfb402f)

Hi @hegra,
Thank you very much for your report.
From your code snippet.

Document size: 800KB ( 8byte * 100K) per a doc
Number of Docs: 4K docs
Total size: 3.2GB (800KB * 4K)
Use continuous push replication

As result -> App crashes or causes Out-Of-Memory error
Honestly speak, we did not spend enough time to optimize push replication because most of use-cases are heavier pull replication. I will look into this issue by next major release (v1.2). In general, this kind of problems are pretty challenging to solve.
Thanks!

Thanks for looking into it.
Some clarification: It never reaches 4k docs since it usually crashes around a few hundred updates, I just put whatever large number in there. Also it is the same document that is updated all the time. It works very fine if you have 1 second sleep inbetween the pushes since then it is sent successfully to the server and unloaded from memory. The issue is when the writes to local db is faster than the pushes, then the sync tries to load all documents in memory that it would like to push as it looks.
If you try to run the script and play around with the sleep time you will see.
But the main issue is that if the app does not have contact with the server for a long time or the server connection is slow, it makes a long changelist in memory that it will never be able to execute on since it loads all docs into memory before starting pushing. This also happens at restart of the app so that a crashed app will not be able to recover and needs to be reinstalled and thus loose all it's data.

Hi @hegra,
Thank you for additional information.
Whenever database has new record/update/delete, it fires DocumentChange notification. It becomes trigger of push replication. Because of too many Change notification, I assume push replication creates many halfway objects in memory for next step (data upload). However, because of slower network, halfway objects kept increasing, then it causes Out-Of-Memory or crash. I will look into it. Please give us time to fix this.
Thanks!

Did some more testing today. It looks like if you only create a lot of new documents that are large and then call manual sync on them afterwards, then it also crashes on out of memory (no change notification even fires). Looks like it tries to put all changes it finds in local db into memory and push all of them at same time? Are there no limit on how much data at a time it will try to send?

The issue looks likely to be that the buffer overflow code hasn't been ported to java in PusherInternal.java:450
//TODO: port this code from iOS
/*
bufferedSize += [CBLJSON estimateMemorySize: properties];
if (bufferedSize > kMaxBulkDocsObjectSize) {
[self uploadBulkDocs: docsToSend changes: revsToSend];
docsToSend = $marray();
revsToSend = [[CBL_RevisionList alloc] init];
bufferedSize = 0;
}
*/

note to self:
simplest scenario to reproduce this issue:

Create 2000  11KB documents at a device. (w/wo Transaction does not matter.)
Start push replication (continuous/not-continuous does not matter)
After around 1500 doc changes are handled, Out of memory error occurs.

Note: Used Genymotion.


Bug: Hostnames of entries in Systems tab are showing up as short name of entries in OSINT tab

I can't upload the exact example .fox file because it contains actual case data, but I will see if I can reproduce with dummy data.
Essentially when we add systems to the systems tab, entries are appearing in the OSINT tab where the system hostnames are in the short name field. Deleting items from the OSINT tab deletes the corresponding system from the system tab. I'm assuming something about the case file is corrupted but we haven't done anything particular unusual to get to this situation. I will try to reproduce but we've just been adding systems both manually and by importing csv in the system tab. Everything in the system tab worked fine, but the OSINT table seems to have got linked to the system table somehow.

Yes, that's a known bug. I'll fix this one but most of the work now goes into the new version


JSDoc @typedef not working as expected

VSCode Version: 1.24.1
OS Version: macOS 10.13.5

@typedef support seems a bit buggy. There are a couple of things going on, but related.
First, typdefs with properties don't seem to work as expected. I have the following file, funcs.js:
/**
 * API options.
 * @typedef {Object} ApiOptions
 * @property {number} artificialDelay Number of milliseconds for an artificial delay
 * @property {boolean} debug Debug mode
 */

/**
 *
 * @param {string} url URL to call
 * @param {ApiOptions} [options] API options
 */
export function apiCall(url, options = {}) {}

And in index.js, I import apiCall but the JSDoc hints don't show the properties of ApiOptions:

Also, typdefs should be able to be "exportable". I modified funcs.js to this:
/** @module api */

/**
 * API options.
 * @typedef {Object} module:api.ApiOptions
 * @property {number} artificialDelay Number of milliseconds for an artificial delay
 * @property {boolean} debug Debug mode
 */

/**
 *
 * @param {string} url URL to call
 * @param {module:api.ApiOptions} [options] API options
 */
export function apiCall(url, options = {}) {}

I added a new file called middle.js:
import { apiCall } from './funcs';

/**
 *
 * @param {module:api.ApiOptions} options
 */
export function test(options) {}

VSCode doesn't seem to understand this usage at all:

It would be great if VSCode fully supported this for projects that aren't using TypeScript.

Does this issue occur when all extensions are disabled?: Yes

What's weird is Intellisense/VSCode already exports typedef across a project by default BUT it fails if the custom uses other custom types.
Here are some test cases.

Declaration of CustomType in a file :

Declaration of a variable of type CustomType  in another file :

Quick info works. 👍
Displayed type is correct. 👍
Displayed type property is correct. 👍

Declaration of CustomType using another custom type CustomType2 in a file :

Declaration of a variable of type CustomType  in another file :

Quick info doesn't work. 👎

Declaration of CustomType using another custom type CustomType2 AND declaration of a variable of type CustomType in the same file :

Quick info works. 👍
Displayed type is correct. 👍
Displayed type property is correct. 👍

Declaration of CustomType AND declaration of a variable of type CustomType in the same file BUT declaration of ```CustomType2`` in another file :

Quick info works. 👍
Displayed type is correct. 👍
Displayed type property is wrong. 👎

I hope this breakdown of different cases help solve the issue !

This issue was moved to Microsoft/TypeScript#25386


Different violation output e.g. json

related: #470

We do have violation output in json. Closing this.


Make _src repo

From Mike Love:
Did I mention earlier why i had compbio and compbio_src as two repos for the same course? This may be useful for you as well. I can explain over email or phone, whichever works better.
The short version is that I wanted a very slim repo to keep the source documents (Rmd's), while also having a course website with lots of images (which I might overwrite multiple times while reworking the material). This adds up to a pretty big repo in terms of disk space. So the only way to have the very slim repo for Rmd and also images is to split it into two repos.
So I just had the two repos on my machine sitting next to each other, and once I was ready to render an Rmd in compbio_src, I would do this and copy over the rendered HTML to the compbio dir. then lastly you have to commit and push both (add/update to Rmd only in the source repo, add/update HTML in the web repo) also the web repo is kept locally always in the gh-pages branch


[Backport] [2.1] Compatibility issue with /_mget: RHLC 2.x connected to OpenSearch Cluster 1.x (#4812)

Backport of https://github.com/opensearch-project/OpenSearch/pull/4812 to 2.1

Codecov Report

Merging #4878 (d8f3f5a) into 2.1 (9fb360e) will increase coverage by 0.08%.
The diff coverage is 66.66%.

@@             Coverage Diff              @@
##                2.1    #4878      +/-   ##
============================================
+ Coverage     70.59%   70.68%   +0.08%
- Complexity    56359    56440      +81
============================================
  Files          4515     4515
  Lines        271433   271436       +3
  Branches      39959    39960       +1
============================================
+ Hits         191617   191863     +246
+ Misses        63674    63457     -217
+ Partials      16142    16116      -26

Impacted Files
Coverage Δ

...va/org/opensearch/action/get/MultiGetResponse.java
70.32% <50.00%> (-0.46%)
:arrow_down:

server/src/main/java/org/opensearch/Version.java
82.54% <100.00%> (-0.40%)
:arrow_down:

.../java/org/opensearch/node/NodeClosedException.java
50.00% <0.00%> (-50.00%)
:arrow_down:

.../org/opensearch/client/indices/AnalyzeRequest.java
31.00% <0.00%> (-45.00%)
:arrow_down:

...regations/metrics/AbstractHyperLogLogPlusPlus.java
51.72% <0.00%> (-44.83%)
:arrow_down:

...adcast/BroadcastShardOperationFailedException.java
33.33% <0.00%> (-44.45%)
:arrow_down:

...opensearch/persistent/PersistentTasksExecutor.java
22.22% <0.00%> (-44.45%)
:arrow_down:

...h/action/ingest/SimulateDocumentVerboseResult.java
60.71% <0.00%> (-39.29%)
:arrow_down:

...cluster/coordination/PublishClusterStateStats.java
33.33% <0.00%> (-37.51%)
:arrow_down:

...ion/admin/cluster/node/info/PluginsAndModules.java
53.12% <0.00%> (-34.38%)
:arrow_down:

... and 469 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.