nerdctl images displays the wrong image id

Description
Whenever nerdctl displays lists of images (nerdctl images, nerdctl image list), the ID being reported are the repo sha, not the image ID.
Compare with Docker.
Steps to reproduce the issue

nerdctl images
docker images

Describe the results you received and expected
Expected: Docker behavior - see the image ID.
Received: repo digest
What version of nerdctl are you using?
1.7.6
Are you using a variant of nerdctl? (e.g., Rancher Desktop)
None
Host information
No response

I see.
Was there a reason for that choice?
The problems I see:

it does make nerdctl and docker fundamentally disagree on what defines the "same image".

it will also lead to internal inconsistent behavior with nerdctl wrt to commands like rmi.

Specifically:
sudo nerdctl pull debian >/dev/null 2>&1
sudo nerdctl tag debian apostasie/nerdctl-test
sudo nerdctl push apostasie/nerdctl-test

shortid="$(sudo nerdctl inspect apostasie/nerdctl-test | jq -rc .[0].RepoDigests[0])"
shortid="${shortid##*:}"
shortid="${shortid:0:8}"

sudo nerdctl rmi -f "$shortid"

sudo nerdctl images # <- empty

# Now...

sudo nerdctl pull debian  >/dev/null 2>&1
sudo nerdctl pull apostasie/nerdctl-test  >/dev/null 2>&1

shortid="$(sudo nerdctl inspect apostasie/nerdctl-test | jq -rc .[0].RepoDigests[0])"
shortid="${shortid##*:}"
shortid="${shortid:0:8}"

sudo nerdctl rmi -f "$shortid"

sudo nerdctl images # <- not empty

This is obviously due to the fact that the RepoDigest will be different for the very same image depending on whether it has been pushed&pulled or not.
I can fix maybe fix it, by querying and keying images during lookup along the "docker" Id, but it is going to make things more complicated and possibly lead to some other types of discrepancies.

containerd's image store is designed to use a single object for multi-platform image index, while Docker's image store is designed to allocate an object per platform.

BTW nerdctl image inspect has been using Docker-compatible ID which is incompatible with the ID shown in nerdctl images:
https://github.com/containerd/nerdctl/blob/3ca6e8bae534471f6621cefc3f6a4c0518582308/pkg/inspecttypes/dockercompat/dockercompat.go#L331
This behavior looks inconsistent and needs some remedy...

Relevant:

https://github.com/moby/moby/issues/47680

Agreed.
Overall, all of it does not look good.
I strongly believe we cannot use repodigest hash as a way to define what an image is - this is just wrong and will lead to many issues (among which the fact that an image will "change" after push according to nerdctl).
I will have a look at the Moby PR and see what we can do here.

BTW nerdctl image inspect has been using Docker-compatible ID which is incompatible with the ID shown in nerdctl images:
https://github.com/containerd/nerdctl/blob/3ca6e8bae534471f6621cefc3f6a4c0518582308/pkg/inspecttypes/dockercompat/dockercompat.go#L331
This behavior looks inconsistent and needs some remedy...

Well... image list will definitely show the repo sha in the ID column (and not the Docker ID).
And image inspect does NOT allow querying by Docker id.
My "inspect" PR also respected that so far.
We might need a design discussion here.

containerd's image store is designed to use a single object for multi-platform image index, while Docker's image store is designed to allocate an object per platform.

The store is one thing, what we conceptually present to the user is another.
The core issue here is: what we currently show to the user is inconsistent, and breaks reproducibility and some of our commands in certain conditions.


Missing Saleor Variable in .env.template

Saleor requires and additional COMMERCE_IMAGE_HOST from what I can see from your examples in the readme in the Saleor package. I could not get even your Saleor Demo working without that setting in my .env.

Hey there! Thank you for your contribution. We have decided to take Next.js Commerce in a new direction and will be closing out current PRs and issues due to this change. Please see this PR for more details: https://github.com/vercel/commerce/pull/966


VS2017 support

Very quick and dirty update for VS2017 support.  "It works on my machine".  Should still support VS2013/15 with these changes, but requires VS2017 to build.

Great timing Brian (https://twitter.com/OlivierGuinart/status/896499395314581505)! I’ll get on your PR asap.
Thanks!
Olivier.
From: Brian Peek<EMAIL_ADDRESS>Sent: Friday, September 22, 2017 8:12 PM
To: OlivierGuinart/Merlin32Language<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: [OlivierGuinart/Merlin32Language] VS2017 support (#4)
Very quick and dirty update for VS2017 support. "It works on my machine". Should still support VS2013/15 with these changes, but requires VS2017 to build.

You can view, comment on, or merge this pull request online at:
https://github.com/OlivierGuinart/Merlin32Language/pull/4
Commit Summary

VS2017 support

File Changes

M Merlin32Language.csprojhttps://github.com/OlivierGuinart/Merlin32Language/pull/4/files#diff-0 (87)
M Merlin32Language.slnhttps://github.com/OlivierGuinart/Merlin32Language/pull/4/files#diff-1 (13)
M Resources/data.Designer.cshttps://github.com/OlivierGuinart/Merlin32Language/pull/4/files#diff-2 (2)
M Resources/directives.Designer.cshttps://github.com/OlivierGuinart/Merlin32Language/pull/4/files#diff-3 (2)
M Resources/opcodes.Designer.cshttps://github.com/OlivierGuinart/Merlin32Language/pull/4/files#diff-4 (2)
M source.extension.vsixmanifesthttps://github.com/OlivierGuinart/Merlin32Language/pull/4/files#diff-5 (49)

Patch Links:

https://github.com/OlivierGuinart/Merlin32Language/pull/4.patch
https://github.com/OlivierGuinart/Merlin32Language/pull/4.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/OlivierGuinart/Merlin32Language/pull/4, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALe73f1o2pJrGe41fk71UXiWibARsxVEks5slHbsgaJpZM4PhbE9.

The Shell.12.0 thing is a bit hacky (using a NuGet package someone created that has the proper DLL in it).  You can perhaps avoid NuGet by using a redirection as described here:  https://msdn.microsoft.com/en-us/magazine/mt493251.aspx but I haven't tested this.  Happy to help if there's something else I can do!  :)
Overall, I was just looking for 6502 syntax highlighting...using cc65 assembler and tools in a project, but this is close enough!

Good job! Just submitted my feedback for discussion\implementation. Tested with an older version of VS and it still works with VS2015 Community 😊 (C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\devenv.exe).
Looking forward your updated PR and to push your changes into master.
Thanks!
Olivier.
From: Brian Peek<EMAIL_ADDRESS>Sent: Saturday, September 23, 2017 11:02 AM
To: OlivierGuinart/Merlin32Language<EMAIL_ADDRESS>Cc: OlivierGuinart<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [OlivierGuinart/Merlin32Language] VS2017 support (#4)
The Shell.12.0 thing is a bit hacky (using a NuGet package someone created that has the proper DLL in it). You can perhaps avoid NuGet by using a redirection as described here: https://msdn.microsoft.com/en-us/magazine/mt493251.aspx but I haven't tested this. Happy to help if there's something else I can do! :)
Overall, I was just looking for 6502 syntax highlighting...using cc65 assembler and tools in a project, but this is close enough!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/OlivierGuinart/Merlin32Language/pull/4#issuecomment-331659230, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALe73R3KxUhBI8_QbGxUHpj5IR13wwMAks5slUewgaJpZM4PhbE9.

Cool, will update this weekend.  I guess maintaining the ref to 12.0 wasn’t necessary after all.  Had I known that I’d just have jumped to 15 and moved on.  Will update the PR today or tomorrow.  Thanks!
From: OlivierGuinart<EMAIL_ADDRESS>Sent: Saturday, September 23, 2017 3:10 PM
To: OlivierGuinart/Merlin32Language<EMAIL_ADDRESS>Cc: Brian Peek<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [OlivierGuinart/Merlin32Language] VS2017 support (#4)
Good job! Just submitted my feedback for discussion\implementation. Tested with an older version of VS and it still works with VS2015 Community 😊 (C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\devenv.exe).
Looking forward your updated PR and to push your changes into master.
Thanks!
Olivier.
From: Brian Peek<EMAIL_ADDRESS>Sent: Saturday, September 23, 2017 11:02 AM
To: OlivierGuinart/Merlin32Language <Merlin32Language<EMAIL_ADDRESS>Cc: OlivierGuinart <olivier_guinart+github<EMAIL_ADDRESS>Comment <comment<EMAIL_ADDRESS>Subject: Re: [OlivierGuinart/Merlin32Language] VS2017 support (#4)
The Shell.12.0 thing is a bit hacky (using a NuGet package someone created that has the proper DLL in it). You can perhaps avoid NuGet by using a redirection as described here: https://msdn.microsoft.com/en-us/magazine/mt493251.aspx but I haven't tested this. Happy to help if there's something else I can do! :)
Overall, I was just looking for 6502 syntax highlighting...using cc65 assembler and tools in a project, but this is close enough!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/OlivierGuinart/Merlin32Language/pull/4#issuecomment-331659230, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALe73R3KxUhBI8_QbGxUHpj5IR13wwMAks5slUewgaJpZM4PhbE9.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/OlivierGuinart/Merlin32Language/pull/4#issuecomment-331663691, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACNtCzHHEHV74pw81Ukd43_cVGWUUQDmks5slVeWgaJpZM4PhbE9.

Conversion to VS2017 done one month ago onto my fork.

Super Jean-Michel ! Anything we’re missing?
Merci !
Olivier.
From: Jean-Michel Paris<EMAIL_ADDRESS>Sent: Saturday, September 23, 2017 12:27 PM
To: OlivierGuinart/Merlin32Language<EMAIL_ADDRESS>Cc: OlivierGuinart<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [OlivierGuinart/Merlin32Language] VS2017 support (#4)
Conversion to VS2017 done one month ago onto my fork.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/OlivierGuinart/Merlin32Language/pull/4#issuecomment-331664649, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALe73Sly5SjrjERzZNaTij-kQ1qtedEsks5slVuQgaJpZM4PhbE9.

I never did a PR on GitHub :) I Should have tried one month ago in order to save you time.
An absolute improvement to do is to add the labels defined in code, to the completion words list!

Made changes, add a .gitignore and removed some binaries and other junk.  I haven't tested this against anything earlier than VS2017.

I'll remove the refs.  I would suggest just doing a squash/merge on the PR which will only import the "final" version of the PR and not the "steps" in between.  You can enable squash merges in the repo settings.  Easier than trying to rewind time in git and you get a clean PR at the end of the process.

Sounds like a plan 😊
Thanks!
Olivier.
From: Brian Peek<EMAIL_ADDRESS>Sent: Sunday, September 24, 2017 4:07 PM
To: OlivierGuinart/Merlin32Language<EMAIL_ADDRESS>Cc: OlivierGuinart<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [OlivierGuinart/Merlin32Language] VS2017 support (#4)
I'll remove the refs. I would suggest just doing a squash/merge on the PR which will only import the "final" version of the PR and not the "steps" in between. You can enable squash merges in the repo settings. Easier than trying to rewind time in git and you get a clean PR at the end of the process.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/OlivierGuinart/Merlin32Language/pull/4#issuecomment-331746594, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALe73bjGOvIp5NsjJaOlUxf0mNk-UmHVks5sluCWgaJpZM4PhbE9.


allow implicit cast from &T to ?&c_void

Should we allow implicitly casting pointers to ?&c_void?

This looks both convenient for reducing clutter in code, and should be perfectly safe, but seeing the cast to ?&c_void is a nice reminder that you are interfacing with unsafe/c code.

I've been interfacing with more and more C code and especially with the new developments around casting I think this feature would be pretty welcome.

I appreciate that feedback. What do you think about the proposal to add another pointer type specifically for C interop? #1059

Yet another pointer type sounds awful, but the convenience of not having to manually correct your imports is really tempting... I think Zig has to do everything it can to make C interop completely seamless. I'll comment more on that issue's thread.
Since any 'real' binding to a C library need work beyond what translate-c can do, any problems inherent to this kind of pointer won't be an issue for 'real code'. I say go for it.

implemented in e2a9f2ef988e6ff4ec9f943ead9285c67c56ffd7 by @raulgrell


Support for kdump/kexec on WSL2

I am trying to set up kernel core dumps (preferably kdump but vmcore is OK) on WSL2  4.19.128-microsoft-standard.   I'm wanting to use this for testing, and debugging (I'm working on some kernel panic / dump parsers) and was hoping I could use WSL2 to generate kdumps via sysrq or driver causing an OOPS.   I can install kdump-tools and kernel-kdump but I cannot get it all configured, such as setting crashkernel kernel variabl
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
Unclear if there are workarounds outside of not using WSL2 for kdump. For sysrq issue that can be resolved by building a driver to trigger a panic.
Additional context
kdump-config show

kdump is not supported by this kernel
no crashkernel= parameter in the kernel cmdline
DUMP_MODE:        kdump
USE_KDUMP:        1
KDUMP_SYSCTL:     kernel.panic_on_oops=1
KDUMP_COREDIR:    /var/crash
crashkernel addr:
/var/lib/kdump/vmlinuz
kdump initrd:
/var/lib/kdump/initrd.img
cat: /sys/kernel/kexec_crash_loaded: No such file or directory
/usr/sbin/kdump-config: line 155: [: too many arguments
current state:    Not ready to kdump

You need to enable the relevant config option and add it to .wslconfig

Where would I as a curious user, go to change that? Suppose, mind you suppose, I had an interest in using either that feature, or the kexec one?

So I don't know which kernel option enables these but you can create a .wslconfig with notepad or some other text editor and place it at C:/Users/user. You can find examples on what to put for config options here in the github. The release notes have the template (or you can just use a search engine, plenty of blogs about it).

I guess that will work. I'm also interested in kexec as well.

https://www.kernel.org/doc/html/v5.8/admin-guide/kdump/kdump.html should do the trick. Per the doc, kdump uses kexec to execute. Now we both know the kernel option needed.

Thanks, I did not know about .wslconfig I will check that out.  I will
search to learn some more, such as if the file goes in the root of your
profile does that mean one config for all installed instances.
Cheers,
Graham
On Fri, Sep 4, 2020, 7:25 AM WSLUser<EMAIL_ADDRESS>wrote:

https://www.kernel.org/doc/html/v5.8/admin-guide/kdump/kdump.html should
do the trick. Per the doc, kdump uses kexec to execute. Now we both know
the kernel option needed.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/WSL/issues/5823#issuecomment-687178273, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABQBADTRBCN4IKUSPUGB3LTSED2FJANCNFSM4QNQ6ZSQ
.

@WSLUser so you're saying that I should create configure in my user name file, that contains the appropriate terms in it? And of course it would be called .wslconfig? Interesting. @grahamm78 what's your take on this?

https://github.com/microsoft/WSL/issues/4694#issuecomment-556095344 Here's an example for you and comes straight from one of the MS devs.

I've gotten a lot further now!  I've still got a few pieces to figure out though.
I built 2 kernels with updated .config based on the 4.19.128-microsoft-standard kernel.  I followed some steps herewhich were helpful on building the kernel, then went back to the kdump documentation to set config options

Build the kernels

System Kernel
cp Microsoft/wsl-config .config-syskernel
CONFIG_LOCALVERSION="grahamm_syskernel" # to easily tell it's loaded via uname -a
CONFIG_KEXEC=y   # this is there but commented by default
CONFIG_DEBUG_INFO=y     # wasn't in there at all, perhaps superceded by CONFIG_KALLSYMS kdump doc may be old?
CONFIG_NET_9P_VIRTIO=y # based on reading as my host systems /mnt/c etc were not appearing

cp .config-syskernel .config
make  (take some defaults on prompts for config options)
cp arch/x86_64/boot/bzImage /mnt/c/Windows/System32/lxss/tools/syskernel
Dump-capture kernel
cp Microsoft/wsl-config .config-dumpkernel
CONFIG_LOCALVERSION="grahamm_dumpkernel"
CONFIG_CRASH_DUMP=y
CONFIG_SMP=n
CONFIG_NET_9P_VIRTIO=y # based on reading as my host systems /mnt/c etc were not appearing

cp .config-dumpkernel .config
make  (take some defaults on prompts for config options)
cp arch/x86_64/boot/bzImage /mnt/c/Windows/System32/lxss/tools/dumpkernel

wsl --shutdown

Created a %userprofile%.wslconfig file with custom kernel, crashkernel and some other options.

[wsl2]
kernel=c:\\Windows\\system32\\lxss\\tools\\syskernel
kernelcommandline= crashkernel=0M-2G:128M,2G-6G:256M,6G-8G:512M,8G-:768M, panic=30, oops=panic, panic_print=ff

Now I launched wsl and I saw the custom kernel and kernelcommandline take effect.

set up kexec to point to the new dumpkernel.  I could not figure out some of the parameters like root device (I could not find the root device name for current OS) so I skipped some parameters as a test.

root@GRAHAMM-Z2:/dev# kexec -p /mnt/c/Windows/system32/lxss/tools/dumpkernel
root@GRAHAMM-Z2:/dev# kdump-config show
DUMP_MODE:        kdump
USE_KDUMP:        1
KDUMP_SYSCTL:     kernel.panic_on_oops=1
KDUMP_COREDIR:    /var/crash
crashkernel addr: 0x18000000
/var/lib/kdump/vmlinuz
kdump initrd:
/var/lib/kdump/initrd.img
current state:    ready to kdump
root@GRAHAMM-Z2:/dev# cat /sys/kernel/kexec_crash_loaded
1
But - I tried to test it with kdump-config test and it dies because KDUMP_KERNEL doesn't exist.
root@GRAHAMM-Z2:/dev# kdump-config test

/etc/default/kdump-tools: KDUMP_KERNEL does not exist: /var/lib/kdump/vmlinuz

so I need to find the places to fix the rest of the configuration settings

/etc/default/kdump-tools: KDUMP_KERNEL does not exist: /var/lib/kdump/vmlinuz
USE_KDUMP:         1
KDUMP_SYSCTL:      kernel.panic_on_oops=1
KDUMP_COREDIR:     /var/crash
crashkernel addr:  0x18000000
kdump kernel addr:
kdump kernel:
/var/lib/kdump/vmlinuz
kdump initrd:
/var/lib/kdump/initrd.img
kexec command to be used:
/sbin/kexec -p --command-line="initrd=\initrd.img panic=-1 nr_cpus=12 swiotlb=force pty.legacy_count=0 panic=30, oops=panic, panic_print=ff reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll nousb ata_piix.prefer_ms_hyperv=0" /var/lib/kdump/vmlinuz

One note of advice - don't put your custom kernels in  c:\Windows\system32\lxss\tools\
It looks like you lose them after a build upgrade..
I'll keep working on this and post an update if I find a solution.


jwm: 1653 -> 1685

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Success on x86_64-linux (full log)
Attempted: jwm
Partial log (click to expand)
glibPreFixupPhase
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/3n2mfn0fx8n0bvjc7sr1lqhhjsrh87cg-jwm-1685
shrinking /nix/store/3n2mfn0fx8n0bvjc7sr1lqhhjsrh87cg-jwm-1685/bin/jwm
gzipping man pages under /nix/store/3n2mfn0fx8n0bvjc7sr1lqhhjsrh87cg-jwm-1685/share/man/
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/3n2mfn0fx8n0bvjc7sr1lqhhjsrh87cg-jwm-1685/bin
patching script interpreter paths in /nix/store/3n2mfn0fx8n0bvjc7sr1lqhhjsrh87cg-jwm-1685
checking for references to /build in /nix/store/3n2mfn0fx8n0bvjc7sr1lqhhjsrh87cg-jwm-1685...
/nix/store/3n2mfn0fx8n0bvjc7sr1lqhhjsrh87cg-jwm-1685

Success on aarch64-linux (full log)
Attempted: jwm
Partial log (click to expand)
glibPreFixupPhase
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/f9anxlhmdxncz1jkh29i1crrfxinr7hs-jwm-1685
shrinking /nix/store/f9anxlhmdxncz1jkh29i1crrfxinr7hs-jwm-1685/bin/jwm
gzipping man pages under /nix/store/f9anxlhmdxncz1jkh29i1crrfxinr7hs-jwm-1685/share/man/
strip is /nix/store/gp7fylxwn18b7pl2c18ks89hsiaxyfvf-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/f9anxlhmdxncz1jkh29i1crrfxinr7hs-jwm-1685/bin
patching script interpreter paths in /nix/store/f9anxlhmdxncz1jkh29i1crrfxinr7hs-jwm-1685
checking for references to /build in /nix/store/f9anxlhmdxncz1jkh29i1crrfxinr7hs-jwm-1685...
/nix/store/f9anxlhmdxncz1jkh29i1crrfxinr7hs-jwm-1685

Success on x86_64-darwin (full log)
Attempted: jwm
Partial log (click to expand)
install -d -m 0755 /nix/store/j1pda1c36mxyz2xzw0a83250jrqvjkbz-jwm-1685/etc
install -m 644 example.jwmrc /nix/store/j1pda1c36mxyz2xzw0a83250jrqvjkbz-jwm-1685/etc/system.jwmrc
install -d -m 0755 /nix/store/j1pda1c36mxyz2xzw0a83250jrqvjkbz-jwm-1685/share/man/man1
install -m 644 jwm.1 /nix/store/j1pda1c36mxyz2xzw0a83250jrqvjkbz-jwm-1685/share/man/man1/jwm.1
glibPreFixupPhase
post-installation fixup
gzipping man pages under /nix/store/j1pda1c36mxyz2xzw0a83250jrqvjkbz-jwm-1685/share/man/
strip is /nix/store/kdff2gim6417493yha769kh00n63lnrw-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/j1pda1c36mxyz2xzw0a83250jrqvjkbz-jwm-1685/bin
patching script interpreter paths in /nix/store/j1pda1c36mxyz2xzw0a83250jrqvjkbz-jwm-1685


HBASE-23568: Improve Threading of Replication

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m  7s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 1 new or modified test files.

_ master Compile Tests _

+1 :green_heart:
mvninstall
5m 58s
master passed

+1 :green_heart:
compile
0m 59s
master passed

+1 :green_heart:
checkstyle
1m 30s
master passed

+1 :green_heart:
shadedjars
5m  3s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 37s
master passed

+0 :ok:
spotbugs
4m 37s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
4m 36s
master passed

_ Patch Compile Tests _

-1 :x:
mvninstall
2m 41s
root in the patch failed.

-1 :x:
compile
0m 41s
hbase-server in the patch failed.

-1 :x:
javac
0m 41s
hbase-server in the patch failed.

+1 :green_heart:
checkstyle
1m 30s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

-1 :x:
shadedjars
3m 44s
patch has 20 errors when building our shaded downstream artifacts.

-1 :x:
hadoopcheck
1m 57s
The patch causes 20 errors with Hadoop v2.8.5.

-1 :x:
hadoopcheck
3m 58s
The patch causes 20 errors with Hadoop v2.9.2.

-1 :x:
hadoopcheck
6m  3s
The patch causes 20 errors with Hadoop v3.1.2.

+1 :green_heart:
javadoc
0m 33s
the patch passed

-1 :x:
findbugs
0m 41s
hbase-server in the patch failed.

_ Other Tests _

-1 :x:
unit
0m 40s
hbase-server in the patch failed.

+1 :green_heart:
asflicense
0m 11s
The patch does not generate ASF License warnings.

37m 51s

Subsystem
Report/Notes

Docker
Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/934

JIRA Issue
HBASE-23568

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile

uname
Linux f9a8a4d08fd4 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-934/out/precommit/personality/provided.sh

git revision
master / 85a081925b

Default Java
1.8.0_181

mvninstall
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/1/artifact/out/patch-mvninstall-root.txt

compile
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/1/artifact/out/patch-compile-hbase-server.txt

javac
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/1/artifact/out/patch-compile-hbase-server.txt

shadedjars
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/1/artifact/out/patch-shadedjars.txt

hadoopcheck
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/1/artifact/out/patch-javac-2.8.5.txt

hadoopcheck
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/1/artifact/out/patch-javac-2.9.2.txt

hadoopcheck
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/1/artifact/out/patch-javac-3.1.2.txt

findbugs
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/1/artifact/out/patch-findbugs-hbase-server.txt

unit
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/1/artifact/out/patch-unit-hbase-server.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/1/testReport/

Max. process+thread count
83 (vs. ulimit of 10000)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/1/console

versions
git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 19s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 1 new or modified test files.

_ master Compile Tests _

+1 :green_heart:
mvninstall
6m 17s
master passed

+1 :green_heart:
compile
1m 10s
master passed

+1 :green_heart:
checkstyle
1m 38s
master passed

+1 :green_heart:
shadedjars
5m 21s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 43s
master passed

+0 :ok:
spotbugs
5m 20s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
5m 18s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
5m 43s
the patch passed

+1 :green_heart:
compile
1m  3s
the patch passed

+1 :green_heart:
javac
1m  3s
the patch passed

+1 :green_heart:
checkstyle
1m 37s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedjars
5m 23s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
18m  4s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.

+1 :green_heart:
javadoc
0m 38s
the patch passed

+1 :green_heart:
findbugs
4m 52s
the patch passed

_ Other Tests _

-1 :x:
unit
315m  0s
hbase-server in the patch failed.

+1 :green_heart:
asflicense
0m 37s
The patch does not generate ASF License warnings.

381m 56s

Reason
Tests

Failed junit tests
hadoop.hbase.replication.TestReplicationSmallTests

hadoop.hbase.replication.TestReplicationKillSlaveRS

hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory

hadoop.hbase.replication.TestReplicationSmallTestsSync

hadoop.hbase.client.TestAsyncTableAdminApi

hadoop.hbase.client.TestFromClientSideWithCoprocessor

hadoop.hbase.client.TestFromClientSide

Subsystem
Report/Notes

Docker
Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/934

JIRA Issue
HBASE-23568

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile

uname
Linux 5aff247275c6 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-934/out/precommit/personality/provided.sh

git revision
master / 85a081925b

Default Java
1.8.0_181

unit
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/2/artifact/out/patch-unit-hbase-server.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/2/testReport/

Max. process+thread count
4982 (vs. ulimit of 10000)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/2/console

versions
git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 30s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 1 new or modified test files.

_ master Compile Tests _

+1 :green_heart:
mvninstall
6m 27s
master passed

+1 :green_heart:
compile
1m  9s
master passed

+1 :green_heart:
checkstyle
1m 41s
master passed

+1 :green_heart:
shadedjars
5m 18s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 40s
master passed

+0 :ok:
spotbugs
5m  3s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
4m 59s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
6m  1s
the patch passed

+1 :green_heart:
compile
1m  3s
the patch passed

+1 :green_heart:
javac
1m  3s
the patch passed

+1 :green_heart:
checkstyle
1m 33s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedjars
5m 22s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
17m 53s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.

+1 :green_heart:
javadoc
0m 36s
the patch passed

+1 :green_heart:
findbugs
4m 36s
the patch passed

_ Other Tests _

-1 :x:
unit
237m 44s
hbase-server in the patch failed.

+1 :green_heart:
asflicense
0m 27s
The patch does not generate ASF License warnings.

304m 14s

Reason
Tests

Failed junit tests
hadoop.hbase.client.TestAsyncRegionAdminApi2

Subsystem
Report/Notes

Docker
Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/934

JIRA Issue
HBASE-23568

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile

uname
Linux d8f413dfe0af 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-934/out/precommit/personality/provided.sh

git revision
master / 2d76457577

Default Java
1.8.0_181

unit
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/3/artifact/out/patch-unit-hbase-server.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/3/testReport/

Max. process+thread count
5304 (vs. ulimit of 10000)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/3/console

versions
git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m  8s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 1 new or modified test files.

_ master Compile Tests _

+1 :green_heart:
mvninstall
6m  0s
master passed

+1 :green_heart:
compile
0m 59s
master passed

+1 :green_heart:
checkstyle
1m 30s
master passed

+1 :green_heart:
shadedjars
5m  1s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 38s
master passed

+0 :ok:
spotbugs
4m 37s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
4m 34s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
5m 28s
the patch passed

+1 :green_heart:
compile
0m 59s
the patch passed

+1 :green_heart:
javac
0m 59s
the patch passed

+1 :green_heart:
checkstyle
1m 29s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedjars
5m  2s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
17m 36s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.

+1 :green_heart:
javadoc
0m 36s
the patch passed

+1 :green_heart:
findbugs
4m 58s
the patch passed

_ Other Tests _

-1 :x:
unit
244m 30s
hbase-server in the patch failed.

+1 :green_heart:
asflicense
0m 28s
The patch does not generate ASF License warnings.

307m 48s

Reason
Tests

Failed junit tests
hadoop.hbase.quotas.TestClusterScopeQuotaThrottle

Subsystem
Report/Notes

Docker
Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/934

JIRA Issue
HBASE-23568

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile

uname
Linux 7863872b06c4 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-934/out/precommit/personality/provided.sh

git revision
master / 2d76457577

Default Java
1.8.0_181

unit
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/4/artifact/out/patch-unit-hbase-server.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/4/testReport/

Max. process+thread count
5139 (vs. ulimit of 10000)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/4/console

versions
git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Have you gotten to try this out on a cluster? I don't think we have a PerfEval equivalent for replication.

Nah.  My intention here was not to improve performance, just trying to get
the code "up to code" by using JDK and Guava facilities to better manage
threads.  My goal was to untangle the existing setup and to make trouble
shooting and adding capabilities easier later.
 I don't think we have a
PerfEval equivalent for replication.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/hbase/pull/934?email_source=notifications&email_token=AC766E3MWR2FQ6QIXR2AJF3QYOHJ5A5CNFSM4J2B5FE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG2AYGQ#issuecomment-565447706,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC766E3CGXXB43HTWOVQQQ3QYOHJ5ANCNFSM4J2B5FEQ
.

I'm not looking for a performance improvement, just that we don't degrade or further beat up HDFS.
In deployments that have high number of peers (e.g. when using the lily indexer with a bunch of solr collections) even the current replication system does a number on system stability.

Well, maybe that's something I can take a look at.  With this threadpool
introduced in this PR, perhaps the number of worker threads can be bounded
a bit.  As I currently have it, the threadpool is unbounded in order to
keep the same behavior as the current implementation.
On Fri, Dec 13, 2019, 12:35 PM Sean Busbey<EMAIL_ADDRESS>wrote:

I'm not looking for a performance improvement, just that we don't degrade
or further beat up HDFS.
In deployments that have high number of peers (e.g. when using the lily
indexer with a bunch of solr collections) even the current replication
system does a number on system stability.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/hbase/pull/934?email_source=notifications&email_token=AC766E45IMN4HGEUT2OZDXDQYPBVTA5CNFSM4J2B5FE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG2VZRA#issuecomment-565533892,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC766E3V4BU6V6M65FNLJGDQYPBVTANCNFSM4J2B5FEQ
.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 51s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 1 new or modified test files.

_ master Compile Tests _

+1 :green_heart:
mvninstall
6m  7s
master passed

+1 :green_heart:
compile
1m  2s
master passed

+1 :green_heart:
checkstyle
1m 35s
master passed

+1 :green_heart:
shadedjars
5m 14s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 39s
master passed

+0 :ok:
spotbugs
5m 29s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
5m 26s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
5m 58s
the patch passed

+1 :green_heart:
compile
1m  0s
the patch passed

+1 :green_heart:
javac
1m  0s
the patch passed

+1 :green_heart:
checkstyle
1m 28s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedjars
5m 16s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
17m 54s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.

+1 :green_heart:
javadoc
0m 38s
the patch passed

+1 :green_heart:
findbugs
5m 21s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
194m 33s
hbase-server in the patch passed.

+1 :green_heart:
asflicense
0m 30s
The patch does not generate ASF License warnings.

261m 34s

Subsystem
Report/Notes

Docker
Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/934

JIRA Issue
HBASE-23568

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile

uname
Linux<PHONE_NUMBER>ed 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-934/out/precommit/personality/provided.sh

git revision
master / 923ba7763e

Default Java
1.8.0_181

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/5/testReport/

Max. process+thread count
4655 (vs. ulimit of 10000)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/5/console

versions
git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

I'm not looking for a performance improvement, just that we don't degrade or further beat up HDFS.
In deployments that have high number of peers (e.g. when using the lily indexer with a bunch of solr collections) even the current replication system does a number on system stability.

This is not the purpose of the patch here? I think the patch here is only a refactoring. The current solution of replication uses different wal readers for different peers, I think this is the problem that causes heavy read pressure on HDFS. This is another problem...

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 12s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 1 new or modified test files.

_ master Compile Tests _

+1 :green_heart:
mvninstall
6m  8s
master passed

+1 :green_heart:
compile
1m  0s
master passed

+1 :green_heart:
checkstyle
1m 16s
master passed

+1 :green_heart:
shadedjars
5m  7s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 37s
master passed

+0 :ok:
spotbugs
4m 42s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
4m 38s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
5m 36s
the patch passed

+1 :green_heart:
compile
1m  0s
the patch passed

+1 :green_heart:
javac
1m  0s
the patch passed

+1 :green_heart:
checkstyle
1m 14s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedjars
5m  7s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
17m 42s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.

+1 :green_heart:
javadoc
0m 36s
the patch passed

+1 :green_heart:
findbugs
4m 44s
the patch passed

_ Other Tests _

-1 :x:
unit
100m 29s
hbase-server in the patch failed.

+1 :green_heart:
asflicense
0m 29s
The patch does not generate ASF License warnings.

163m 57s

Reason
Tests

Failed junit tests
hadoop.hbase.replication.TestReplicationSmallTestsSync

hadoop.hbase.replication.TestReplicationDroppedTables

hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput

Subsystem
Report/Notes

Docker
Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/934

JIRA Issue
HBASE-23568

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile

uname
Linux 1375918e083d 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-934/out/precommit/personality/provided.sh

git revision
master / 867b1e9cbc

Default Java
1.8.0_181

unit
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/6/artifact/out/patch-unit-hbase-server.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/6/testReport/

Max. process+thread count
6419 (vs. ulimit of 10000)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/6/console

versions
git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Just saw a couple of replication unit tests failures on the last build. Could it be anyhow related to this change?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
3m 41s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 1 new or modified test files.

_ master Compile Tests _

+1 :green_heart:
mvninstall
6m  7s
master passed

+1 :green_heart:
compile
0m 59s
master passed

+1 :green_heart:
checkstyle
1m 15s
master passed

+1 :green_heart:
shadedjars
5m  7s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 38s
master passed

+0 :ok:
spotbugs
4m 33s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
4m 32s
master passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
5m 35s
the patch passed

+1 :green_heart:
compile
0m 59s
the patch passed

+1 :green_heart:
javac
0m 59s
the patch passed

+1 :green_heart:
checkstyle
1m 11s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
shadedjars
5m  6s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
hadoopcheck
17m 37s
Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.

+1 :green_heart:
javadoc
0m 36s
the patch passed

+1 :green_heart:
findbugs
4m 51s
the patch passed

_ Other Tests _

-1 :x:
unit
105m 11s
hbase-server in the patch failed.

+1 :green_heart:
asflicense
0m 28s
The patch does not generate ASF License warnings.

170m 55s

Reason
Tests

Failed junit tests
hadoop.hbase.replication.TestReplicationSmallTestsSync

hadoop.hbase.replication.TestReplicationDroppedTables

Subsystem
Report/Notes

Docker
Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/7/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/934

JIRA Issue
HBASE-23568

Optional Tests
dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile

uname
Linux 11b47d871ce9 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux

Build tool
maven

Personality
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-934/out/precommit/personality/provided.sh

git revision
master / 93c5e7691f

Default Java
1.8.0_181

unit
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/7/artifact/out/patch-unit-hbase-server.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/7/testReport/

Max. process+thread count
6457 (vs. ulimit of 10000)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-934/7/console

versions
git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Is this one till active? Mind addressing the above review comments?


Why PCA?

Hi, thank you for the code!
Any specific reason the dimensionality reduction is done with PCA? I understand the flexibility of storing the PCA matrix and controlling the dimensionality after fitting, but a learnable linear projection at the end of the network may deliver better performance, did you test that?
Thanks!

Hello, thanks for your interest in our work. At the beginning, we chose PCA just because I followed the benchmark repository to implement our VPR system, which can easily add PCA to reduce dimensionality. I know that some recent work prefers to use the learnable linear projection for dimensionality reduction and has achieved good results. In recent days I have also tried adding a (10752, 4096)-linear projection at the end of the model to reduce the dimension to 4096-D. However, it seems that the model is overfitting after training. Perhaps it would be better to reduce the channel dimension from 768-D to a lower dimension beforehand, which uses a linear projection with fewer parameters. We are currently improving the dimensionality reduction in our model.


Feature: Order Event locations alphabetically

Event locations shall be sorted alphabetically.
This would make the event easier to find if there are multiple events in a certain country eg. India

cc: @MichaelKohler

Makes sense!


Report shows unused keys, even though they are used by other translations

Hi, I am adding an example below. Report says that foo is not used, even though fooView.fooUsed is used in the code.
Any ideas how we could solve this?
{
    "foo": "Foo translation",
    "fooView": {
        "fooUsed": "@:foo"
    },
}

Thank you in advance.

Hey @r0kk is @: a way to reference another translation, because if so, that won't work with this tool. This tool just does static code analysis to match key names in language files to names of translations in code.


Show last used time on device info page

@keybase/react-hackers
(Visdiff should be able to show this change.)

@chrisnojima Oops, thanks.  Backed out that line, and added a comment to the top of the generated file saying that it's auto-generated.

👍

Ah!  There are no visdiff changes because this was already mocked out.  So here's a screenshot of the live changes in the smart component on prod, for completeness:


add workaround for inserting space on autocomplete on Mac

Closes #1323

something weird with travis builds timing out after hours

works for me on mac.

@mdeuser, perhaps you can speak to Windows on this issue.

@dubeejw - sure.  can you give me a specific example to try out.  the command auto completion seems to work fine on windows/vagrant/ubuntu for me, but i'm sure there's some subtlety..

Yeah it works on Linux and Windows without the workaround


Problem Details - only for production

https://learn.microsoft.com/en-us/aspnet/core/web-api/handle-errors?view=aspnetcore-8.0


Default helo is localhost

From what I can see the default helo that gomail sends out is "localhost" and multipass has no option to change this. This makes sending email over smtp fail if the server requires a correct FQDN.
 <localhost>: Helo command rejected: need fully-qualified hostname

I am considering removing support for SMTP/MTA altogether and go with supporting only mail clients. MSA support was requested in #11 and added in 99ad323.
Implementing a correct MTA requires managing and queueing mail which is outside the scope of this project.

In that case, the documentation should probably be updated for that.

The smtp_client feature is documented in the README.md file. The example uses the sendmail command which is available on most *nix systems.


Issue 14775

Purpose

Hex floating point literals are always float.

Fixes #14775
Check List

[ ] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Chiran seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

Codecov Report

Merging #22640 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #22640   +/-   ##
=======================================
  Coverage   14.59%   14.59%
=======================================
  Files          51       51
  Lines        1411     1411
  Branches      219      219
=======================================
  Hits          206      206
  Misses       1189     1189
  Partials       16       16

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 24eec40...ae3786b. Read the comment docs.


[5.1] Catching Log Events

I'm not too sure how to go about this, but technically shouldn't the outcome of the code below be Log was written to?:
Route::get('/', function () {

    Event::listen('illuminate.log', function()
    {
        die('Log was written to');
    });

    throw new Exception('Exception');
});

When exceptions are thrown, a log entry is created, however it doesn't seem the event is dispatched when this happens. Only when you explicitly call the log will the event be dispatched. For example the output of this is Log was written to:
Route::get('/', function () {

    Event::listen('illuminate.log', function()
    {
        die('Log was written to');
    });

    Log::info('info');

    throw new Exception('Exception');
});

I'm not exactly sure if this is a bug or not? I'm guessing not, and if that's the case, does anyone have any tips on doing this? I just need to catch when the log is being written to and save the information along with it.
https://github.com/laravel/framework/blob/5.1/src/Illuminate/Log/Writer.php#L295.

Please ask on the forums.


User definable x and y axis values

There is a usecase for this with, for example, trios. I have three samples and I want to compare them to each other (and ultimately include read coverage distribution in a final report). For comparison across samples, I want to ensure that the x- and y- axes limits are set to be the same, e.g. as I tab across the three bam.iobio instances, it is easy to see a discrepancy in one file.

Al, I have this working.  Do you have a preference for UI?
This is what I have in a popup window currently, but I can rename everything as we see fit.  I can also send in the actual axes names like "define lower limit for frequency" instead of just saying 'y axis'.
Also, I have reset clearing everything and reverting to default ranges.  Would you want the ability to reset back to what is currently on the screen?  Like 'reset' would put back the values on the screen and there'd be a third option called 'restore to default' or something like that.
Finally, I have this accessible by double clicking the either of the axis labels, is this okay?

I think this works; at least to get in a staging version of bam for people to test and comment on.

I agree that it would be better to pull in axis names, so that it is clear what values you are setting.

I think the reset would just return to what bam.iobio had set it's own axis values to. Probably no need for a third option, but again, if we get the basic functionality available, maybe others will have a use case that I haven't considered.

We probably need something a little more obvious to the user for modifying this, but as an initial solution for testing, I think this is fine. We could look at having a settings button for the chart that would provide this functionality, and then collect all available actions together (the only other action currently is to pull up info). This may be no clearer though!!


Add to FAQ about permissions issues

Many users dont know what the chown command is. We need an FAQ on this issue

I'll do it :o))

https://github.com/dgibbs64/linuxgsm/wiki/FAQ#im-getting-ownership-issues
If it's OK to you, you can close. :)


Add switch autocomplete on enum

In Android Studio, Dart does not compile if a switch is used on an enum and some clause are missing.
Though I do not see why should that trigger a fatal compiler error, (Why not just skip missing clause) one would intuitively place the cursor on the error and press the keys: ALT+ENTER but nothing happens.
I suggest a feature that auto-generate all missing clause with an empty: break statement or just not consider this a fatal compiler error at all.

It makes sense to have a fix to remove the error. Just adding default: break; is likely not sufficient :)
(See https://github.com/dart-lang/sdk/issues/36127 for the ongoing discussion about whether this should be a warning or error. The fix makes sense in either case).

https://dart-review.googlesource.com/c/sdk/+/97780


Ets 236 phac auth endpoint

Added passport strategy for the new phac endpoint
@ashtonmeuser We may want to merge this one first and then change the passport.authenticate('jwt', { session: false }) to passport.authenticate('jwt-phac', { session: false }) on the PHAC upcoming endpoint

Looks good to me, just need to remove that one console out 😄


v2.8.1

Docker Image for build 1 is available on DockerHub as jc21/nginx-proxy-manager:github-pr-955


Send multiple annual emissions parameters and return data

Definition of Ready

[x] Does this meet the INVEST criteria?
[x] Does this have a clearly defined user?
[x] Does this have an estimate?
[x] Does this have acceptance criteria?
[x] Does this have sufficient context (wireframes, business rules, etc.)?
[x] Is this traceable to a feature, epic, or PI?

User Story
As a CAMPD API user I want to send filter criteria to the annual emissions API endpoint so that I can see results
Acceptance Criteria
Given that a user has entered one or more valid values for all the parameters (see parameters below)
When API endpoint is executed
Then the system displays the records that match the criteria (containing data elements below)
Given that a user has entered one or more valid values for all the parameters in an upper case or lower case format
When API endpoint is executed
Then the system displays the records that match the criteria
Given that a user has entered one or more valid values for all the parameters without white space(s) before or after the delimiter
When API endpoint is executed
Then the system displays the records that match the criteria
Given that a user has entered one or more valid values for all the parameters with white space(s) before or after the delimiter
When API endpoint is executed
Then the system displays the records that match the criteria
Given that a user has made a GET request with the attach file parameter set to true
When request is executed
Then the system produces a file in the requested file format (json or csv)
AND the file contains the records that match the criteria (see context)
Parameters

Year: Required, multiple values allowed, Format: YYYY,  Range: 1980, 1985, 1990, 1995-latest reporting quarter, Allow indiv. Years, ranges, and comma sep. list
Facility ID (ORIS code): Optional, multiple values allowed
State: Optional, multiple values allowed, 2 letter postal abbreviation for the 50 states plus Puerto Rico.
Unit Type: Optional, multiple values allowed
Unit Fuel Type: Optional, multiple values allowed, query will filter on both primary and secondary fuel type
Control Technologies: Optional, multiple values allowed
Program: Optional, multiple values allowed, excludes MATS

Data Elements
Results:
When the system identifies at least one record that matches the criteria, then for each record the system will display the data below and ordered by Facility > Unit > Year
STATE
FACILITY_NAME
ORISPL_CODE
UNITID
ASSOC_STACKS
YEAR
SUM_OP_TIME
COUNT_OP_TIME
GLOAD
SLOAD
SO2_MASS
SO2_RATE
NOX_MASS
NOX_RATE
CO2_MASS
CO2_RATE
HEAT_INPUT
PRIMARY_FUEL_INFO
SECONDARY_FUEL_INFO
UNIT_TYPE_INFO
SO2_CONTROL_INFO
PART_CONTROL_INFO
NOX_CONTROL_INFO
HG_CONTROL_INFO
PRG_CODE_INFO
Dev Tasks

[x] Update apportioned emissions controller, service and module
[x] Add annual unit data repository
[x] Add annual field mappings
[x] Add annual apportioned emissions DTO and paramsDTO
[x] Add annual unit data entity and update unit fact entity with relation
[x] Add annual apportioned emissions map
[x] Add unit tests

Context
Dependency: Need annual emissions data labels @maheese
The csv file  must contain the appropriate data labels. See the aggregated emissions data label doc
Testing
Test Case: TC192
Tester: @JanellC
Definition of Done

[x] Have all Acceptance Criteria been agreed to and validated by the scrum team?
[x] Has the code been unit tested and peer reviewed?
[x] Have the functional tests been executed?
[x] Have all defects been reviewed and dispositioned, resolved, or deferred?
[ ] Has the user story been accepted by the Product Owner?
[x] Is the user story ready to be deployed to the test/staging environments?

TC192 passed @tlageman-epa


HSEARCH-2319

A trivial one which has been bugging me.
https://hibernate.atlassian.net/browse/HSEARCH-2319

Merged, thanks!


Getting {"message":"Not found"}

Hello,
I have everything up and running, but when I click the report button a new window opens with "{"message":"Not found"}".
My backend is served up on a different port. The link reflects the correct port, as 8686 will bring up connection error. Maybe I changed the port number incorrectly? Any help with this?
Sorry, sort of a newbie to all of this.

My backend is served up on a different port.

On what port is it set up? Can you paste the log output of the granfa reporter service? Then we can ensure that the reporter is configured with the correct grafana IP.

HI! I got this working. I reconfigured the backend to serve on port :3333 (grafana-reporter -port :3333).
Is there any instructions on pushing this up to github and working for our customers to also generate these pdf reports?

Is there any instructions on pushing this up to github and working for our customers to also generate these pdf reports?

I'm afraid you will have to elaborate; I do not understand this question.


When I use a Signed Cookie then I cannot impersonate

Hello,
I realised that when I use Signed Cookies then I cannot impersonate another user. The system just destroys my active session.
Ideas?
Thanks,

Not sure what's happening exactly, but I think the rules to force signed cookies and SSL is causing this problem. I will review these settings on an isolated environment.


User-level permission expansion/override

This is  a feature request.

Prerequisites

[x] Are you running the latest version?
[x] Are you reporting to the correct repository?
[x] Did you check the documentation?
[x] Did you perform a cursory search?

Description

The bulk of our permissions are defined on a general role basis; however (as we have a granular rollout of new features, amongst other reasons) we need to define/override certain permissions on a user-level
Proposal
Base
Add a permission_user pivot that is also checked along with permission_role to verify permission.
This includes introducing a permissions() object on the user directly; that will check the role's permissions too (to remain DRY)
TODO:

[ ] add permissions() relationship in the User Model (similar to the one in the Role Model)
[ ] write accompanying db migration (new permission_user table)
[ ] update from role->permissions() to user-based required in:

[ ] menumanager: MenuTree's setPermissions()
[ ] core: AppState's routes()
[ ] searchable: Finder's permissions()
[ ] permissionmanager: AuthServiceProvider's boot() (access-route)

[ ] Investigate implications for ConfigWriter (in rolemanager)

Expansion/optional (depending on your feedback)

[ ] Add a flag to above pivot indicating if it's a allow or deny (to allow overriding in both directions.
[ ] implement permission-edit-UI coupled to user (our implementation only requires code-wise changes; but since the UI is already there for roles it shouldn't be a big extra to implement this; making the feature more solid for more use cases)

Note that I'll be starting on this first thing next week, hopefully as a contribution to the framework as I believe there's many use cases for this (depending on your comments, will otherwise just override what I need)

At the first sight having roles and then overriding permissions seems spaghetti.
I'm trying not to jump to conclusions, so please help me to better understand the business need.

The bulk of our permissions are defined on a general role basis; however (as we have a granular rollout of new features, amongst other reasons) we need to define/override certain permissions on a user-level

How many users do you have in the app? How many roles? Give me a few 'rollout feature' examples that build the case.

I got lost after this

We have about 1000 users logging in, with about 5 roles total

In my opinion having the option to add / suppress permission for 1000 users it's not something that I would pursuit.
Let's say that your roles are the ones below:

Superadmin - this has all the permissions
Admin
Supervisor
PM
User

You have lots of features that you roll out every week for every "subset" of users.
Just create 4 new roles with identical permissions to the original ones:

Admin beta tester
Supervisor beta tester
PM beta tester
User beta tester

Move the users that do the tests / explaining the tools to the beta tester roles.
Once you have new features, add the permissions first to the beta testers, and when they are mature also to the original roles.
BOOM.

The issue is that feature A might roll out to subset 1 while feature B might rollout to subset 2; and it might take months (or in some cases years) before everyone has all features. This is due to existing business logic and not something that I'll be able to change,
The granularity could be limited to permission groups instead; but those aren't tied into anything currently.

especially as there's business logic tied into the role (different role = different actions)

we have to do id/name based checks on roles to give different actions; creating a role for every possible feature combination would cause an even bigger mess in that logic.
I'm not in favor of having user-level permissions like this either but it's business logic that I'm not in a position to change; so there's no way around it for us.
(the "multi-year" differences could be turned into a couple extra roles; but this can't easily happen for the rollout as it's tied into external factors with random orders; that would easily introduce bigger issues)
Also to clarify: those who do the explaining are explaining it on the account of the user; and once explained it should be available for that user.
Simultaneously, it's possible that a different group is doing the same for a different feature.
I considered what you suggested at first; but it would easily get too complicated that way (+ the shown role would look weird as it would need a programmatic definition of the tool subset so it can be dynamically applied on login; tied into AD/NAV)

I guess we can close this, or you could try to move it on your fork.

i.e. not to be considered for the framework? 👌 🔒

I'll keep you posted (might take a while as we'll only need it at a later stage; but we will definitely need a solution for it by early next year)


WIP Create a device independent buffered uarte

Use existing BufferedUarte for nRF to create a generic BufferedUart that
can be used for other platforms that implement traits for operating a
uart in an interrupt driven way.
Add wrapper types that can be used to split TX and RX to operate
independently, and add a split() method to the nRF implementation that
uses these wrapper types
@Dirbaio I haven't really vetted this for soundness, I'm just looking for initial comments on the approach I'm taking. Could also be this need further modifications to work with stm32, but the goal is to implement the same TimerPeripheral and UartPeripheral traits for stm32 serial.

New info: with STM32 DMA it's possible to query how many bytes have been received without stopping DMA. This means DMA can be kept receiving constantly, and only needs restarting when buffer is actually filled.
So the "wait for idle and restart the RX to flush" is a nrf-specific limitation.
I'm not sure how this impacts the device-independent trait :sweat_smile:, but it certainly does look like something we should take advantage of.

New info: with STM32 DMA it's possible to query how many bytes have been received without stopping DMA. This means DMA can be kept receiving constantly, and only needs restarting when buffer is actually filled.
So the "wait for idle and restart the RX to flush" is a nrf-specific limitation.
I'm not sure how this impacts the device-independent trait 😅, but it certainly does look like something we should take advantage of.

My take on this is after seeing the differences between nRF and STM32 and the way one would go about
doing DMA, I'm not sure how much there is to gain from having a generic BufferedUart, and that the
maintenance burden be less when having two independent implementations of the AsyncWrite +
AsyncBufRead traits.
Maybe the RingBuffer + Waker logic can be abstracted into some helper type that could be shared, but
I think it will be easier to do that once having proper implementations for both nRF and STM32 to
see the common patterns.  Given that the architectures don't seem to change dramatically, I think
the maintenance burden of having two separate implementations will still be low.
I think what would be more valuable is to be able to optionally split TX and RX into separate types,
which makes it easier to work with for UART-based modems IMO.
Wdyt?

:+1: to waiting to have one for stm32 and reconsidering later
:+1:  to adding split to Uarte/BufferedUarte. I've missed that when messing with modems too.


CTRL_P not working as intended?

Is this a feature that has yet to be fully implemented? I see the groundwork laid out, but It doesn't seem to be functional (or I'm just using it wrong). What is InteractivePrediction supposed to do?

I'm up for discussing what ctrl+p should do. Right now it will try to guess which other file you'd like to open.

with one file open, suggest 'alternate' file (like if it's a .c file, suggest the .h file)
with multiple files open, suggest the most recently used file (like ctrl+tab)
with no files open, suggest a new, empty buffer

In some cases, alternate files are included at the bottom of the list (in case the suggestion is wrong and the alternate was desired).
Something I don't like about this arrangement, is that it's not easy to know what it's going to suggest.
The navigation to select a file can be done with ctrl+p (or up arrow) and ctrl+n (or down arrow).
Typing in the p: prompt does have any effect at the moment. That bit is WIP.

What about using it to go to the next/previously edited files?
CTRL_TAB/CTRL_SHIFT_TAB can't be differentiated on my computer. Both show up as CTRL_I keys in curses, so unless we can figure out some hack to determine modifier keys (I really want CTRL_backspace :(  ), then I don't think we'll be able to have this functionality. What if we moved this tab/file switching to CTRL_P?

I'm not sure how well the prediction thing would work. It seems like more often than not, it would maybe show up the incorrect file. But even if we do use some algorithms to predict what files the user wants to go to, I feel like this would be something that could be tacked onto CTRL_O as an enhancement rather than a new feature.

CTRL_TAB/CTRL_SHIFT_TAB can't be differentiated on my computer
It would be great to have that work.

so unless we can figure out some hack to determine modifier keys
afaik we'd need to find a terminal that did that. (or create/modify one to do it).

Here are the features I'm looking for (that can be combined into ctrl+p or separated)

switch to another open file (like ctrl+tab)
switch to an alternate file (like .c to .h)
switch to perviously opened files (like a 'recently opened' list)
open the source file that defines/declares a code element like a class or function (like symbol lookup)
open file referred to by #include/import (if the cursor was on such a line)
open url link in browser (if the cursor was on such)
maybe some other lookups, like define word or show results of equation.

That was kinda too much stuff to make separate commands for (I thought), so I wondered about a key that would say 'figure out what I want' to the editor. :) (which I grant, may be intractable).
So far I'm not sure if combining the first two items above are good.

I like the 4th bullet point. Been using it a lot recently after discovering it in some other editors. I don't know if there is a more suitable key for that though. I know the other editors have CTRL_SHIFT_R to search for all files in the project for declaration and CTRL_R to search for the current file for class/function declaration. Maybe we could just use it for switching to alternate files, though not sure how limited that would be. It could be for .c/.h for C,  model/model_controller/model.html.* for ruby, etc so I could see that being useful.


Uploaded ThemeEditor.Zip, fixed README.md links

In both 84ece79 and 0240065 the Theme Editor link was accidentally changed instead of the Review Tool link.
I have uploaded the ThemeEditor.zip that was previously on pahandsonlab.blob.core.windows.net to the Theme Editor App folder and resolved the issue with the two links.

Closing old PR since this was fixed anyways


Pull doesn't update?

I'm using the latest code. I have a few questions here:
1 - I have a local branch. There is a nwe commit in the Remote server.
2 - I do a Pull using a brnach ie:
err := git.Pull(&git.PullOptions{RemoteName: "origin", Auth: a.getAuth(), SingleBranch: true, ReferenceName: b, Progress: os.Stdout })
CheckIfError(err)

Looks like, the Pointer gets updated in the Worktree.go file
if err := w.updateHEAD(ref.Hash()); err != nil {
                return err
        }

But then,  in the Reset that is in the Pull section, it detect UnstaggedChanges, and then doesn't update the code. I believe this was working with an older version ?.
if err := w.Reset(&ResetOptions{
                Mode:   MergeReset,
                Commit: ref.Hash(),
        }); err != nil {
                return err
        }

I've managed to comment the unstaged check, since for pull request I dont really mind if my files that I am working with gets replaced.
Then I had to also add the following in the resetIndex if a file is modified...
if err := w.checkoutChange(ch, t, idx); err != nil {
                                return err
                        }

So the file get reseted with the one just pulled.
and i had to make sure w.resetWorktree doesn't run on "git pull" as other wise if i change local files not yet commited, then the pull will simply remove my change and I would not be happy of losing changes.
Overall i think its a solution that works, I am not sure what are the impacts.
Regards.


How to edit the attributes inside the internal table?

Hi, I am new to SAP ABAP. How to edit the internal table attributes? want to add one more row but not sure how to do. Any suggestion and teaching is appreciated..

Hi,
It looks like the question is related to general ABAP code, this repository is for abapGit
Suggest opening a question on https://answers.sap.com
closing


Do not merge

This PR is being used to try an repro an issue with GitHub Preview.

@JohanFreelancer9 Copy editing is needed for this PR. Thanks.

Hi @scanum,
No reviews needed for this PR. I am not intending to merge any changes. A user reached out and mentioned they were having issues viewing changes, from a different branch, in the GitHub Preview. This branch is just to repro the issue on my end for troubleshooting purposes.


has_one nested form always required?

I followed instructions for doing a has_one association and using the nested form approach @ https://github.com/sferik/rails_admin/wiki/Has-one-association. The nested form shows up fine in the parent model, however, the parent record doesn't save because the has_one nested form is always required to be filed out now. How do I make the has_one form optional?

Fixed by #1632.


Very high CPU usage

This extension is causing the VSCode electron instance to use a lot of CPU. I have no more info.

I noticed that too. It was also very hard to find out what extension caused about 50% CPU usage even when idle/no project opened. VSCode profiling through code --status shown nothing 😬

VS Code has its own settings synchronization for quite a while, so I think it is suggested to uninstall the extension for good.

Yes, I ve seen the same here. Its the rigth extension which causes this.

When you enable it will instantly jump to 50-100%

Same here.

OS: macOS Monterey 12.4
VS Code: Version: 1.68.0

I just did a vscode bisect, and pin down the incredible slowdown from my vscode is supposed to be this extension. Especially if I open multiple windows.

Has anyone checked the processes/what they're communicating with?

+1 for high CPU usage. I have to disable this extension. Any fix?

This has been an issue for me as well, have to disable it for now.

It looks like there's a PR to fix it that needs to be merged - https://github.com/shanalikhan/code-settings-sync/pull/1379


failed for yarn build command in 0.20.3 release

Do you want to request a feature or report a bug?
report a bug
What's the current behavior?
For 0.20.3 release version, failed to do the 'yarn build'. 'yarn build' is ok for the previous release build.
D:\download\react-big-calendar-0.20.3>yarn build
yarn run v1.12.3
$ yarn clean && yarn build:cjs && yarn build:umd && yarn assets && yarn less
$ rimraf lib
$ babel src --out-dir lib
Successfully compiled 52 files with Babel.
$ BABEL_ENV=esm yarn rollup -c
'BABEL_ENV' 不是内部或外部命令，也不是可运行的程序
或批处理文件。
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I ran into this too. yarn install fixed it.


Allow to save figure as svg

Since d3 generate svg it should be cool to allow user to save current figure to svg.
D3 generate svg client side, so to do that we have to find a way to send client data (svg code) back to the ipython kernel and then to save it as svg file on hard drive (I don't think it's possible/good to do it client side).
What do you think ?

such botton would be great.

Note: you can save an SVG directly from matplotlib, if that is something you desire. I don't see this happening in mpld3 very soon.


There's currently no way for admins to remove an image from Campaign, Activity

@tonysurma Can this be assigned to me, I've got a rough version I'd like to get some eyes on.

Considering the following scenario:

User chooses a picture and saves the campaign
This uploads the picture to Azure blob storage
User edits the campaign and uploads another picture
This overrides the picture saved against the campaign but doesn't delete the old picture

With regards to (4), is this acceptable at the moment, as it could leave orphaned images in blob storage?
I've been working on a soft delete scenario, so an image is flagged for deletion but doesn't actually get removed or disassociated from the campaign until the user actually clicks "Save". This ensures the user can change their mind and back out of the changes without losing the picture.
If I don't have to worry about cleaning up orphaned images, I'm very close to a working solution but if cleaning up is preferred I'll need to rethink some stuff.

Go for it @YannickMeeus I like the idea and okay with orphaned images as long as we have some way of knowing they are orphaned (and not just having to do a cross ref of current images referenced in one list and the contents of a folder in the other)  Thx

@YannickMeeus was there ever a PR for this Issue?

@mgmccarthy There hasn't been, no. I'm afraid I completely got stuck in work for the last couple of months, with little to no time to spend on AllReady I'm afraid. If someone else wants to pick up this change, they're more than welcome to it as I won't be able to address this any time soon. Apologies guys.

@YannickMeeus no need to apologize, life happens as they say 😄

@tonysurma do yo wan to reassign? Is this still a P3 issue?

sure, do you want to take it @mgmccarthy ?

@tonysurma, yep.

@tonysurma, can you please update the the title of this issue to that says Activity to Event?
Also, how important is this change? Right now, you can update an image, just not remove one. Worst came to worst, if you wanted no image, then upload a small write .png or something along those lines.
And it looks like we stick these images into a CloudStorageAccount. I'm not too familiar with how this works, or the API for it, but I'm assuming we AllReady should aim to keep the amount of storage to a minimum, hence, orphaning the images might not be the best approach?

@tonysurma @MisterJames
I'm going to need some UI guidance here.
Here is the edit Campaign page after I've uploaded an image:

"Browse for image" shows "no file choosen" and the image is shown on the screen. Purely from a UI perspective, how do you think we should mark this for deletion so when I click on the Save button at the bottom of the edit screen, the image associated with the Campaign will be deleted?

title updated.
For UI, the ideal would be a button next to the Image that says remove and when clicked is a seperate action from save that just removes the image.
You could also put in a simple checkbox or something to say [] remove image and check that on save for now.
For deleting past, just log that as an issue and someone else/some when else can fix that 😸

@tonysurma, I like the ideal scenario; the button next to the Image that says "Remove" or "Delete".
This is a better UI experience as the user can take care of deleting that image right from the edit page w/out having to hit the "Save" button, and send the entire Campaign back through the edit process.
I think it's confusing to have a checkbox next to an existing image with the word "Remove", but also to have the Browse For image button in the same form powered by the same "Save" button to me.
The changes I've made to the existing functionality delete an existing image if another image is selected for upload on the edit page.
That being said, I'm not really a javascript or a knockout guy, so I think I'm going to open a new Issue for adding a Remove button for existing images on the Campaign and Event page and let a javascript/knockout person take care of it.
For someone with that skill set, it should be relatively straight forward as I've already written the code that deletes the image from Azure blob storage. They should just be able to use their javascript magic to invoke a controller action that will call into the DeleteImageAsync on IImageService.
Sample code for how that method is used is in Admin.Controllers.CampaignController's Edit POST action method.


ffs: create wallet address

choose type

[x] BLS
[x] Secp256k1
[ ] Multisig (not sure there's anything server side to do here)

@jsign have any ideas what might be meant by multisig and how it might relate to pg?

@jsign have any ideas what might be meant by multisig and how it might relate to pg?

Filecoin supports multisig addresses.
These addresses are controlled by multiple actors, and things can be signed for these addresses with a minimum configurable threshold of K (K<=N) actors. Say, for sending funds from this account you need K from the N people that own the address.
Currently, multisig addresses aren't used in PG. How they can be related, I'm not that sure about. The only obvious thing that I might think about is adding support to create these kinds of wallets, but only for send-receive actions. Using them for deals might be possible, but I don't have that much clear.

Removing multisig here because it's really a separate thing that will take some time. Tracking here https://github.com/textileio/powergate/issues/361


Reply feature

Adds the ability to reply to messages in the chat!

Client sides finds message by ID, so no risk of loading old or purged messages
Acts as a mention to the user being replied to
Ability to click on message previews to jump to them
Message previews not in client's message history listed as "Message couldn't be found"
Message previews that refer to messages that have been purged will be replaced by "Message was purged"
Users that have permission to see purged messages will instead see the preview styled the same as purged messages would be
^and the same thing for shadow banned messages, in the weird case that one is replied to (ex: by the shadow banned user)

Required database changes

replying_to_id - default -1 to avoid problems with a 0 indexed message, stores the id of the message that is being replied to
reply_should_mention - default doesn't really matter, stores if the author of the message being replied to should be mentioned

ALTER TABLE chat_messages ADD COLUMN replying_to_id BIGINT NOT NULL DEFAULT -1;
ALTER TABLE chat_messages ADD COLUMN reply_should_mention BOOL NOT NULL DEFAULT true;

Oh yeah, and I did a quick look over to make sure that it looks OK on every theme, but of course it's only a quick edit if anything needs improvement

Edited "Required database changes" in original pr message

Now this should actually be ready to merge.

Now this should actually be ready to merge.

"Now this should actually be ready to merge."

Once again wrong, more changes incoming


Implement str::FromStr for Version and VersionReq

This lets you use str::parse(), eg,
let v: Version = "1.0".parse().unwrap();

Not sure how best to handle the tests here. I considered testing a couple of cases to be sure it worked, but that doesn't necessarily protect against weird refactorings. I ended up doing some copypasta. Easily changed depending on your preferences!

I like it. Thanks!


Zoom icons function

Is the cross hairs zoom button suppose to work as a zoom to extents/all, or as it currently functions where it zooms to the center of mass at the 100% setting?
If I start with a large or spread out program like this zoomed out to see it all:

then clicked on cross hair:

It is mean to reset the zoom level to the default value set on Blockly injection, which in this case would the 100% (/ardublockly/ardublockly_blockly.js#L48)

Thanks.


Kodi v17 changed name of the player overlay showing codecs and such info

For v17 a different window should be triggered for showing the codec/video info which is under longpress  button

b7ece61951590b6adbd0bcad04a8f61df6574566 and 18811371a7df8f9942d15d6a9c4bf94cada45bde should fix this.

Works. Thx


How can i change message time format?

How can i changes message time format?please help me.
@stfalcon

Hi @BhargavVGhodasara you should check documentation
For example you can implement DateFormatter.Formatter to your Fragment or Activity with chat and override method format
override fun format(date: Date): String {
        return when {
            DateFormatter.isToday(date) -> getString(R.string.chat_date_header_today)
            DateFormatter.isYesterday(date) -> getString(R.string.chat_date_header_yesterday)
            else -> DateFormatter.format(date, "EEEE, dd MMMM")
        }
    }

messagesAdapter.setDateHeadersFormatter(this)

I think OP was asking about the message time format (the one below each message bubble), not the header one. How can I change that?


Shape safe API in Scala: verify tensor compatibility at compile-time

Description
We would like to introduce a scala API that actively validates the feasibility of tensor operations while being written, NOT while being tested or deployed. The API will be based on the scala library shapesafe, which I'm co-maintaining.
The capability of shapesafe is a superset of several prior arts (see references): It validates not only tensor with literal shape, but also tensor as an output of heavy computations (e.g. einsum or several NN layers). This makes both prototyping & critical production less error-prone, while avoiding computation overhead after deployment (as type information are erased at runtime)

After integration, a typed tensor API will be the main entry point to define an NN architecture. Calling each of its method will define both compile-time shape and runtime computation at the same time, yielding more typed tensors.
Unfortunately this API will only be for Scala, maybe it is not a bad choice considering the overlapping audience of Apache Spark.
Will this change the current api? How?
No, but a new API exclusive to Scala user can be introduced which enables this feature
Who will benefit from this enhancement?
ML Users of Scala, Apache Spark, CHISEL, SpinalHDL ... who are prone to make mistakes caused by mismatched tensor sizes.
References
Prior Art

Evan Spark for first showing the possibility
Tongfei Chen et al. for Nexus
Breandan Considine et al. for Kotlin∇
Dougal Maclaurin et al. for Dex
Maxime Kjaer et al. for tf-dotty

Yeah, actually I already made a promise to one of your colleagues, @lanking520 (is he still in the team?), to start the integration last year. But then the other committer dropped out for his PhD, so I have to do it on my own, and much slower.
Who is the specialist for Scala/Spark integration?

@tribbloid please let me know if you have any Spark/Scala related issues

@lanking520 Nice! I thought you moved to another team.
At this moment, there is no scala code in master branch, should the new API be in a new module?

@lanking520 Nice! I thought you moved to another team.
At this moment, there is no scala code in master branch, should the new API be in a new module?

do you think it's better have another repo? Since we have our deepjavalibrary org, we can create a new repo for scala.


Try pipeline as code & gitea

PaC Works with webhook:
https://gitea.apps.cluster-2q7fv.2q7fv.sandbox2813.opentlc.com/team-1/starter-app-python/settings/hooks

And a token: https://gitea.apps.cluster-2q7fv.2q7fv.sandbox2813.opentlc.com/user/settings/applications

At the OpenShift Cluster
apiVersion: pipelinesascode.tekton.dev/v1alpha1
kind: Repository
metadata:
  creationTimestamp: '2024-06-27T09:05:18Z'
  generation: 4
  name: git-starter-app-python
  namespace: team-1-dev
spec:
  git_provider:
    secret:
      key: provider.token
      name: git-starter-app-python-token-8gxxt
    url: 'https://gitea.apps.cluster-2q7fv.2q7fv.sandbox2813.opentlc.com/'
    webhook_secret:
      key: webhook.secret
      name: git-starter-app-python-token-8gxxt
  url: 'https://gitea.apps.cluster-2q7fv.2q7fv.sandbox2813.opentlc.com/team-1/starter-app-python'

Create token via API:
$ curl -X 'POST'   'https://gitea.apps.cluster-2q7fv.2q7fv.sandbox2813.opentlc.com/api/v1/users/team-1/tokens'   -H 'accept: application/json' -u opentlc-mgr:openshift   -H 'Content-Type: a
pplication/json'   -d '{"name": "asdf","scopes": ["write:repository"]}'
{"id":2,"name":"asdf","sha1":"b2bf8019e15f0d38f9dceb2ba85b07ee24ee9e0e","token_last_eight":"24ee9e0e","scopes":null}
$ curl -s -H "Content-Type: application/json" -u opentlc-mgr:openshift ${GITEA}/api/v1/users/team-1/tokens | jq
[
  {
    "id": 2,
    "name": "asdf",
    "sha1": "",
    "token_last_eight": "24ee9e0e",
    "scopes": [
      "write:repository"
    ]
  },
  {
    "id": 1,
    "name": "tesr",
    "sha1": "",
    "token_last_eight": "c0d53d97",
    "scopes": [
      "write:repository"
    ]
  }
]
$

Webhook api

$ curl -s -H "Content-Type: application/json" -u opentlc-mgr:openshift ${GITEA}/api/v1/repos//team-1/starter-app-python/hooks | jq
[
  {
    "id": 1,
    "type": "gitea",
    "branch_filter": "*",
    "config": {
      "content_type": "json",
      "url": "https://pipelines-as-code-controller-openshift-pipelines.apps.cluster-2q7fv.2q7fv.sandbox2813.opentlc.com"
    },
    "events": [
      "push"
    ],
    "authorization_header": "",
    "active": true,
    "updated_at": "2024-06-28T15:46:35Z",
    "created_at": "2024-06-27T09:02:30Z"
  }
]

$ curl -X 'POST' \
  'https://gitea.apps.cluster-2q7fv.2q7fv.sandbox2813.opentlc.com/api/v1/repos/team-1/starter-app-python/hooks' \
  -H 'accept: application/json'  -u opentlc-mgr:openshift  \
  -H 'Content-Type: application/json' \
  -d '{
  "active": true,
  "branch_filter": "*",
    "config": {
      "content_type": "json",
      "url": "https://pipelines-as-code-controller-openshift-pipelines.apps.cluster-2q7fv.2q7fv.sandbox2813.opentlc.com"
    },
    "events": [
      "push"
    ],
"type": "gitea"
}'
{"id":2,"type":"gitea","branch_filter":"*","config":{"content_type":"json","url":"https://pipelines-as-code-controller-openshift-pipelines.apps.cluster-2q7fv.2q7fv.sandbox2813.opentlc.com"},"events":["push"],"authorization_header":"","active":true,"updated_at":"2024-06-28T16:34:29Z","created_at":"2024-06-28T16:34:29Z"}
$ curl -s -H "Content-Type: application/json" -u opentlc-mgr:openshift ${GITEA}/api/v1/repos//team-1/starter-app-python/hooks | jq
[
  {
    "id": 1,
    "type": "gitea",
    "branch_filter": "*",
    "config": {
      "content_type": "json",
      "url": "https://pipelines-as-code-controller-openshift-pipelines.apps.cluster-2q7fv.2q7fv.sandbox2813.opentlc.com"
    },
    "events": [
      "push"
    ],
    "authorization_header": "",
    "active": true,
    "updated_at": "2024-06-28T15:46:35Z",
    "created_at": "2024-06-27T09:02:30Z"
  },
  {
    "id": 2,
    "type": "gitea",
    "branch_filter": "*",
    "config": {
      "content_type": "json",
      "url": "https://pipelines-as-code-controller-openshift-pipelines.apps.cluster-2q7fv.2q7fv.sandbox2813.opentlc.com"
    },
    "events": [
      "push"
    ],
    "authorization_header": "",
    "active": true,
    "updated_at": "2024-06-28T16:34:29Z",
    "created_at": "2024-06-28T16:34:29Z"
  }
]
$

pipeline is available in test and runs on a new tag


The word 'Bubble' has been replaced by 'Pill' in README.

This is for issue #427

Well... we are not the 1st anymore 🙃


Create external data: Master key password not validated until going through connection page

Steps to Reproduce:

Open create external data wizard and go to master key page
Enter a really simple password that won't pass validation (e.g. 4 letters, no uppercase or special chars)
Click next

Expected:

Wizard blocks with error
Actual:
Wizard goes to next page. On completing connnection info & clicking next, then you get a "password is not complex enough" error. This is very confusing to the user as it might be related to the password field on the current page.

It works without the issue after full build. Can you test it again in your end?

Interesting - I can't repro. Will close for now.


Fix for bean params

fix for bean params

This change is

Coverage increased (+0.02%) to 62.38% when pulling e2343eea19b970becc670281656c5985d11b9a8a on action-param-fix into 2a5242b912af18a1c5f30f354752a9c62664abfd on master.


Fix selection of DRES/DRPES

This is a small fix for #177. Previously, DRPES was selected based on the presence of the RANK environment variable. This led to issues for us during training with pytorch lightning.
In this PR, we allow passing a keyword argument parallel_retrieval to MTEB.run, which is further processed by AbsRetrieval.evaluate and BeIRTask.load_data to toggle the selection of DRES/DRPES. As a consequence, we also need to  pass all kwargs of MTEB.run to task.load_data. I am not sure whether this could possibly lead to problems in other parts of MTEB.

I think we can even remove DRPES entirely - Do you ever use it? I'm not sure if people are even using it & I think doing parallel retrieval is much easier to do on the model side (i.e. in the users encode function).

Our team has indeed not used DRPES, since we decided to execute different tasks on the individual GPUs. However, if I understand the idea of DRPES correctly, I could imagine that it makes sense in some cases. Embedding on the model side and retrieval on the MTEB side are mostly separate, so it would be my understanding that DRPES  could be used in combination with parallelism on the model side, leading to some additional speedups (?).

I encountered the same problem. This solution should be good. Why it still not be able to merge?

Merged sorry! I was hoping @NouamaneTazi would take a look as he wrote the code it affected;
We will refactor the BEIR integration soon to make it more usable!


This tells git to ignore certain nonessential files from export allowing for faster installing from AssetLib

Not sure if this is actually welcome, but this gives a faster and easier install flow where user doesn't need to select what to keep.
Note: this works by removing ignored files from zip downloaded from github. This also affects releases archives, so you might want to keep something like README or LICENSE. But other than that folders like .github and files like .git* are really have no reason to be exported.
Awesome project :)

I belive the readme is quite important to include, same with the docs, but the other git stuff can be left out :)


Black Backgorund

I have an issue whilst using Light themes. My fragment's backgrounds are black instead of white. PreferenceFragment for example shows black text on black background then. Switching to a dark theme lets the text appear in white and it is visible. I am never setting any black backgrounds. What am I doing wrong?

It seems that my view with  android.support.v4.widget.SwipeRefreshLayout as its root layout is causing this. Running the application with Preferences viewed at first, it is shown correctly. Calling my listview then and switching back to another fragment afterwards keeps the black background. :( Is this a known problem? Is there any best practice for a swipe/pull to refresh behaviour with MaterialNavigationDrawer?


Wrong ccount on Windows

On Windows the ccount is incorrectly calculated to be the number of commits from the beginning of the repository and to HEAD. This is due to the ^ character being a escape character in the command prompt shell on Windows (since shell=True is used to subprocess).
https://github.com/dolfinus/setuptools-git-versioning/blob/14a4adc6c93e33b5acc3098869c1f60cc238a5c2/setuptools_git_versioning.py#L86
By quoting the name argument to git everything works as expected: \"^{name}\"

Hi.
Thank you for a fix. I've applied it into master branch and released v1.2.10, please check.

Works correctly with both cmd and powershell on Windows now. Thanks.


Haneefa Communication Availability added

[x] Cloned the repository
[x] Created my own branch
[x] Added my available time in communication file
[x] Ran the formatting scripts
[x] Pushed the file changed to Github
[x] Created a pull request in git Github

Issue linked


How to add a dateset to the get_coord_add function

I want to train the imdb dataset

@chaserxin @SafaaDaf  U can run the code once as it is , u will get the shape_mismatch error , Just look the shape which u want and then extend the given matrix by scaling in both the dimensions ,such that it resolves the error.

@chirag-singh That's what I tried and the error gone, but I wonder how this matrix would affect the accuracy ?