Feature/dockerized dubbo admin

Dockerize dubbo-admin to deploy easily
https://hub.docker.com/r/xinmeng/dubbo-admin/
docker-compse file and instruction has been included

please add apache license to your new added file

hi, the licence has been added

Dockerize dubbo admin is a good idea. we can have a docker hub account and let docker hub to compile code and build docker image.

@htynkn You are right, the image repository is easy to update in Maven configuration. And we can automatically update it by Travis.

Hi, maintainers.  Months past, will this PR merge into develop branch?

This PR is against master branch. However, develop is currently our active branch.

ok, let me change the base branch to develop, is it right? @biyuhao

Codecov Report

Merging #51 into develop will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             develop      #51   +/-   ##
==========================================
  Coverage      19.38%   19.38%
  Complexity       289      289
==========================================
  Files             91       91
  Lines           3637     3637
  Branches         566      566
==========================================
  Hits             705      705
  Misses          2857     2857
  Partials          75       75

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 45c0d41...ea5e3c2. Read the comment docs.

Hi @xmeng1
Sorry for the late response. After a review of your work and some study of the docker module of Apache nifi, I have several suggestions.

Since we have dubbo-admin-distribution module in the develop branch, I think we can put this function into that module along with other packing stuff.
Binding with docker profile and package phase. Then we may package with this command ./mvnw clean package -Pdocker -DskipTests
We may have to use Apache Dubbo<EMAIL_ADDRESS>for maintainer in Dockerfile. Please refer to https://github.com/apache/nifi/blob/master/nifi-docker/dockermaven/Dockerfile#L20
Seems the ASF do have a official dockerhub account and there are already some repos here. Dubbo is still under incubation I'm not sure if we can publish to the official account. (Need some guide from PPMC)

any advice is welcome

Binding with docker profile and package phase. Then we may package with this command ./mvnw clean package -Pdocker -DskipTests

Hi, @biyuhao I agree with your suggestions.
However considering users who don’t have Docker installed, they run mvn package will get an error. Building an image should not be a common operation I think.  So I recommend not binding the compilation phase.  With mvn dockerfile:build or mvn dockerfile:push .

@ettingshausen Yes, I have noticed that.
We can add a maven profile just like the release profile we used. https://github.com/apache/incubator-dubbo-ops/blob/develop/dubbo-admin-distribution/pom.xml#L69
The gpg sign and bin-release action is executed only if you enable the release profile. (specify with -Prelease) Many users didn't setup gpg environment either.
For the better convenience, I prefer to use -Pdocker.
I hope this looks good to you :)

Hi @xmeng1

Sorry for the late response. After a review of your work and some study of the docker module of Apache nifi, I have several suggestions.

Since we have dubbo-admin-distribution module in the develop branch, I think we can put this function into that module along with other packing stuff.

+1

Binding with docker profile and package phase. Then we may package with this command ./mvnw clean package -Pdocker -DskipTests

+1

We may have to use Apache Dubbo<EMAIL_ADDRESS>for maintainer in Dockerfile. Please refer to https://github.com/apache/nifi/blob/master/nifi-docker/dockermaven/Dockerfile#L20

+1

Seems the ASF do have a official dockerhub account and there are already some repos here. Dubbo is still under incubation I'm not sure if we can publish to the official account. (Need some guide from PPMC)

Yes, we can.

any advice is welcome

the docker image has been intergrated in #356


Create token54

What

It's not even the right filename

I cannot believe you didn't merge!!!! This pull request would of saved lives!!

it's like a defibrilator used on living people - it won't


Remove opencv-python as an installation requirement

I'm working on a system that does not allow pip install opencv-python. I am unable to install mmdetection on my system, because opencv-python is a requirement of setup.py file.
I won't be using opencv in my work, so is there any way I can complete the installation without installing opencv-python? (remove it as a requirement maybe?)
I tried this: https://github.com/open-mmlab/mmdetection/issues/445#issue-425739212
But it doesn't work.
Error Traceback (after executing pip install -v -e .)
1 location(s) to search for versions of opencv-python:
* https://pypi.org/simple/opencv-python/
Fetching project page and analyzing links: https://pypi.org/simple/opencv-python/
Getting page https://pypi.org/simple/opencv-python/
Found index url https://pypi.org/simple
Looking up "https://pypi.org/simple/opencv-python/" in the cache
Request header has "max_age" as 0, cache bypassed
https://pypi.org:443 "GET /simple/opencv-python/ HTTP/1.1" 304 0
  Skipping link: none of the wheel's tags match: cp27-cp27m-macosx_10_10_intel, cp27-cp27m-macosx_10_10_x86_64, cp27-cp27m-macosx_10_6_intel, cp27-cp27m-macosx_10_9_intel, cp27-cp27m-macosx_10_9_x86_64: https://files.pythonhosted.org/packages/e3/09/6d175ca54a4fd3072b4575e971140ced19a96fffe75828358c72b29c6147/opencv_python-<IP_ADDRESS>-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl#sha256=d316c9660ed5f2625a69425c52d391dedd4df3fef11f2ed50dcce97f3e567827 (from https://pypi.org/simple/opencv-python/)
  Skipping link: none of the wheel's tags match: cp27-cp27m-manylinux1_i686: https://files.pythonhosted.org/packages/42/64/fabb6af7858c346cf946deccbadee9b7fb2bb38d18d1b7de12b87675ce50/opencv_python-<IP_ADDRESS>-cp27-cp27m-manylinux1_i686.whl#sha256=4ba17d5ce5c7f3ad39c50ce46c3c6af0bf2d77df6382840310352e0d9dd9be8a (from https://pypi.org/simple/opencv-python/)
.
.
.
Given no hashes to check 0 links for project 'opencv-python': discarding no candidates
ERROR: Could not find a version that satisfies the requirement opencv-python>=3 (from mmcv>=0.2.15->mmdet==1.0rc1+f96e57d) (from versions: none)
Cleaning up...
  Removing source in /tmp/pip-install-8jmtj1ww/mmcv
  Removing source in /tmp/pip-install-8jmtj1ww/terminaltables
Removed build tracker: '/tmp/pip-req-tracker-fs0w87le'
ERROR: No matching distribution found for opencv-python>=3 (from mmcv>=0.2.15->mmdet==1.0rc1+f96e57d)
Exception information:
Traceback (most recent call last):
  File "/home/users/student21/.conda/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
    status = self.run(options, args)
  File "/home/users/student21/.conda/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 331, in run
    resolver.resolve(requirement_set)
  File "/home/users/student21/.conda/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/home/users/student21/.conda/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/home/users/student21/.conda/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 281, in _get_abstract_dist_for
    req.populate_link(self.finder, upgrade_allowed, require_hashes)
  File "/home/users/student21/.conda/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 249, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/home/users/student21/.conda/envs/open-mmlab/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 927, in find_requirement
    'No matching distribution found for %s' % req
pip._internal.exceptions.DistributionNotFound: No matching distribution found for opencv-python>=3 (from mmcv>=0.2.15->mmdet==1.0rc1+f96e57d)

Thanks!

opencv-python is a dependency of MMCV. If you are sure you want to ignore opencv-python, you may comment these lines and install MMCV again.

Cool, thanks a lot!


Delete User with default avatar fails

Gitea version (or commit ref): 1.1.0
Git version: 2.1.4
Operating system: Debian Jessie
Database (use [x]):

[ ] PostgreSQL
[x] MySQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[X] No
[ ] Not relevant

Log gist:  [...uters/admin/users.go:256 DeleteUser()] [E] DeleteUser: Fail to remove /home/user/gitea/data/avatars/4: remove /home/user/gitea/data/avatars/4: no such file or directory

Description
When I try to delete a user i get the error above. The user i wanted to delete didnt changed avatar. The user have the default avatar when logged in. (This blue thing).

maybe you mean v1.0.1?

Oops my fault. I mean 1.0.0. I updated the Issue.

But I think this have been resolved by 1.0.1. Could you try that version?

I updated to 1.0.1. It has been fixed. Thanks a lot. I installed 1.0.0 because the docs say 1.0.0.

wget -O gitea https://dl.gitea.io/gitea/1.0.0/gitea-1.0.0-linux-amd64

It's a document problem. Let's discuss there.

Dropped the bug and milestone because it have already been fixed.


[Bug]: Syntax Error: "(" unexpected

Bug description
Steps to reproduce the problem:

install puppeteer
wait
try to start (through jest if that helps)

Puppeteer version
<EMAIL_ADDRESS>Node.js version
v19.2.0
npm version
8.19.3
What operating system are you seeing the problem on?
Linux
Configuration file
(unmodified)

Relevant log output
Failed to launch the browser process!
    /home/nathan/.cache/puppeteer/chrome/linux-1069273/chrome-linux/chrome: 1: Syntax error: "(" unexpected

    TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md

      at onClose (node_modules/puppeteer-core/src/node/BrowserRunner.ts:328:9)
      at Interface.<anonymous> (node_modules/puppeteer-core/src/node/BrowserRunner.ts:314:16)

@Lightning00Blade could you try on a linux?

I am on linux (linux -> ssh -> raspberry pi)

nathan@nathanpi:~ $ script
Script started, output log file is 'typescript'.
nathan@nathanpi:~ $ ls /home/nathan/.cache/puppeteer/chrome/linux-1069273/chrome-linux/chrome
/home/nathan/.cache/puppeteer/chrome/linux-1069273/chrome-linux/chrome
nathan@nathanpi:~ $ ls /home/nathan/.cache/puppeteer/chrome/linux-1069273/chrome-linux/
chrome                   chrome_sandbox  libEGL.so             locales                nacl_irt_x86_64.nexe  v8_context_snapshot.bin
chrome_100_percent.pak   chrome-wrapper  libGLESv2.so          MEIPreload             product_logo_48.png   vk_swiftshader_icd.json
chrome_200_percent.pak   ClearKeyCdm     libvk_swiftshader.so  nacl_helper            resources             xdg-mime
chrome_crashpad_handler  icudtl.dat      libvulkan.so.1        nacl_helper_bootstrap  resources.pak         xdg-settings
nathan@nathanpi:~ $ exit
exit
Script done.

ah actually it seems to be this issue https://github.com/puppeteer/puppeteer/issues/550

also possibly related https://github.com/puppeteer/puppeteer/issues/2924

#550 is it not downloading, but it looks like it is getting downloaded for me. #2924 looks more accurate, but I am getting a (, not a |.

We don't support arm64 on linux because Chromium project does not provide binaries for that architecture on Linux at the moment.

I just had the same issue on Apple Silicon, people on StackOverflow seem to suggest to turn on Rosetta emulation inside Docker: https://stackoverflow.com/a/76093840

I am using Amazon Linux arm64. if isnt compatible, is something I could do or I just need to use some other library for my PDF creation on backend?

I am using Amazon Linux arm64. if isnt compatible, is something I could do or I just need to use some other library for my PDF creation on backend?

Yeah, I'm facing the same issue, and puppeteer docs say it's not compatible with arm64
https://pptr.dev/troubleshooting/


feat(typed-headers): add support for passing custom ValidationPipe

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

7 of 7 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
149

Relevant Lines:
149

💛  - Coveralls


Test button Fix

Resolves #310
https://drive.google.com/file/d/1oGuO0m9hw3y7er4sJTaQBEg4xTMw4bwH/view?usp=drive_link

I think it should be pointed to testnet when we're on mainnet and back to mainnet when we're on testnet (the icon could also be changed on testnet to a production related icon like the contruction hat or something - ⚙ )

Now what's left is the icon changes

How about the same reflection on the gateway?


server: Get vm network/disk statistics and update database per host

Description
This PR fixes an issue when collect vm network or disk statistics from hosts.
If cloudstack fails to get vm statistics from a host, the whole process will be terminated, and it will not get vm statistics from remaining hosts.
With this pr, the host with problem will be ignored, and cloudstack will continue on remaining hosts.

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[X] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[X] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?

@ustcweizhou do you plan to address above comments?
@blueorangutan package

@ustcweizhou do you plan to address above comments?
@blueorangutan package

@shwstppr a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@shwstppr a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✖centos8 ✔debian. JID-2587

Packaging result: ✔centos7 ✖centos8 ✔debian. JID-2587

@DaanHoogland thanks for review. modified some debug message
@shwstppr code is changed.

@DaanHoogland thanks for review. modified some debug message
@shwstppr code is changed.

@blueorangutan package

@blueorangutan package

@shwstppr a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@shwstppr a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos7 ✖centos8 ✔debian. JID-2591

Packaging result: ✔centos7 ✖centos8 ✔debian. JID-2591

@blueorangutan test

@blueorangutan test

@shwstppr a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@shwstppr a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

This may require testing on different hypervisors too
@blueorangutan test centos7 vmware-67u3

This may require testing on different hypervisors too
@blueorangutan test centos7 vmware-67u3

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

Trillian test result (tid-3412)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39607 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4601-t3412-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_07_deploy_kubernetes_ha_cluster
Failure
3615.86
test_kubernetes_clusters.py

test_08_deploy_and_upgrade_kubernetes_ha_cluster
Failure
0.04
test_kubernetes_clusters.py

test_09_delete_kubernetes_ha_cluster
Failure
0.04
test_kubernetes_clusters.py

ContextSuite context=TestKubernetesCluster>:teardown
Error
103.09
test_kubernetes_clusters.py

Trillian test result (tid-3412)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39607 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4601-t3412-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Smoke tests completed. 82 look OK, 1 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_07_deploy_kubernetes_ha_cluster
Failure
3615.86
test_kubernetes_clusters.py

test_08_deploy_and_upgrade_kubernetes_ha_cluster
Failure
0.04
test_kubernetes_clusters.py

test_09_delete_kubernetes_ha_cluster
Failure
0.04
test_kubernetes_clusters.py

ContextSuite context=TestKubernetesCluster>:teardown
Error
103.09
test_kubernetes_clusters.py

Trillian test result (tid-3414)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 42587 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4601-t3414-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Smoke tests completed. 81 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_03_deploy_and_upgrade_kubernetes_cluster
Failure
252.62
test_kubernetes_clusters.py

test_07_deploy_kubernetes_ha_cluster
Failure
3612.82
test_kubernetes_clusters.py

test_08_deploy_and_upgrade_kubernetes_ha_cluster
Failure
0.05
test_kubernetes_clusters.py

test_09_delete_kubernetes_ha_cluster
Failure
0.05
test_kubernetes_clusters.py

ContextSuite context=TestKubernetesCluster>:teardown
Error
120.22
test_kubernetes_clusters.py

test_02_create_template_with_checksum_sha1
Error
5.11
test_templates.py

test_03_create_template_with_checksum_sha256
Error
5.11
test_templates.py

test_04_create_template_with_checksum_md5
Error
5.10
test_templates.py

Trillian test result (tid-3414)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 42587 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4601-t3414-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Smoke tests completed. 81 look OK, 2 have error(s)
Only failed tests results shown below:

Test
Result
Time (s)
Test File

test_03_deploy_and_upgrade_kubernetes_cluster
Failure
252.62
test_kubernetes_clusters.py

test_07_deploy_kubernetes_ha_cluster
Failure
3612.82
test_kubernetes_clusters.py

test_08_deploy_and_upgrade_kubernetes_ha_cluster
Failure
0.05
test_kubernetes_clusters.py

test_09_delete_kubernetes_ha_cluster
Failure
0.05
test_kubernetes_clusters.py

ContextSuite context=TestKubernetesCluster>:teardown
Error
120.22
test_kubernetes_clusters.py

test_02_create_template_with_checksum_sha1
Error
5.11
test_templates.py

test_03_create_template_with_checksum_sha256
Error
5.11
test_templates.py

test_04_create_template_with_checksum_md5
Error
5.10
test_templates.py

3 checksum checks that failed on vmware. not related but very disturbing!?!


🐛 code-coverage-backend: Not working with new backend system due to enforced auth

Plugin Name
code-coverage-auth
📜 Description
Hey everyone,
Seems like the code-coverage-backend plugin it's broken when used with the new backend system. We started to get "error":{"name":"AuthenticationError","message":"Missing credentials"} when trying to upload coverage reports after updating to the new backend, I'm assuming because all backend endpoints are meant to be behind auth according to the docs
👍 Expected behavior
Coverage reports are uploaded to Backstage
👎 Actual Behavior with Screenshots
"error":{"name":"AuthenticationError","message":"Missing credentials"}
👟 Reproduction steps
curl     --header "Content-Type:text/xml" \             -X POST \             -d @coverage.xml \             "https://<URL>/api/code-coverage/report?entity=component:default/<YOUR-ENTITY>&coverageType=cobertura"
📃 Provide the context for the Bug.
No response
👀 Have you spent some time to check if this bug has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
No, but I'm happy to collaborate on a PR with someone else

Closing since we realized we would need to use a service token moving forward.


#101: Fixes invalid call-back for buffers targetted by vkCmdCopyQuery…

…PoolResults()
This addresses #101

-1. It's still not clear to me that this isn't required. The spec says: "VK_BUFFER_USAGE_TRANSFER_DST_BIT indicates that the buffer can be used as the destination of a transfer command." as well as "vkCmdCopyQueryPoolResults is considered to be a transfer operation" which implies (thought its not explicitly stated) that this should be used.

This got to be required - it shouldn't be any different than other copy (or transfer) commands in this respect.
This condition isn't stated in the Valid Usage section for vkCmdCopyQueryPoolResults, but I suspect we just forgot and that we should add "dstBuffer must have been created with VK_BUFFER_USAGE_TRANSFER_DST_BIT usage flag" there.
Do you agree? If so, shall we create a spec bug instead?

Don't mind either way! If that's what your driver needs to work fine, I'm all for getting this requirement added to the valid usage section.
The only reason I put up this MR is that I wanted to pinch someone for a reaction ;) Apparently, it worked!

As probably obvious from my comment I agree with Jan Harald and can propose that change unless you want to do so.

Yes, please, that would be very helpful.

https://gitlab.khronos.org/vulkan/vulkan/merge_requests/1213

closing


major rework of HA backup doc

@deniseschannon: replies to your general comments:

changed it to code blocks.
I adamantly disagree. A) I think most people won't put that together without frustration first. B) Just from a UX perspective, I think people will appreciate both commands there. If you really want me to change it, just ping me in slack :)
Yes, I am going to point to this file in both HA install docs. That's next on my list.
I agree with this comment. I updated all the commands with rke in front.


Expansion of @ tokens suffers from ID collisions

The expansion of @ tokens (e.g. @1 into urn:ivcap:account:fbbddd13-c9a3-472f-a776-d4ffd6c7b9be) happens differently depending on your request history. For example, if you run ivcap service list and ivcap artifact list, and there exists both a service and an artifact with token @1, then in a subsequent command @1 will expand to whichever was run last:

Note the different error messages depending on which list was requested last.
The cause of this is that the CLI saves the history of @ expansions in ~/.config/ivcap/history.yaml, but doesn't save them by resource type, only by @ token. See func GetHistory in cmd/root.go.

Yes, that's how it was designed. I agree that it would be often more convenient to do what you suggest, but to further avoid confusion, we would very likely have to expand the history token to also include an indication of type, such as @s1, @o1, ... as having two o@1 in a command, both referring to different resources may be rather confusing. Unfortunately, we now have (a)artifact and (a)aspect.
Another approach would be to a) reuse the history token if it's in the history and b) to use a largish (~99) history 'ring'. Meaning we reuse old tokens when we have used up all available tokens.
Interested to hear what folks think. What other choices do we have?

Can the CLI always know what type the thing it's expanding /should/ have? I.e. in my order create example above, it knows that the first @1 should be a service, and can probably also infer that the second one should be an artifact. So history.yaml can contain maps with pairs of strings as keys, e.g. ("artifact", "@1"): "urn:ivcap:artifact:...".

Another option is we don't use numbered tokens, and instead do what git does, with truncations of the hashes. E.g. urn:ivcap:account:fbbdd13-c9a3-472f-aa776-d4ffd6c7b9be becomes 7b9be.

Another option is we don't use numbered tokens, and instead do what git does, with truncations of the hashes. E.g. urn:ivcap:account:fbbdd13-c9a3-472f-aa776-d4ffd6c7b9be becomes 7b9be.

I was thinking this as well... I didnt know how hard it would be to implement... so I didnt mention it...
It would be handy to  type in the first few characters of the hexidecimal hash and either  complete it... or trust that you typed in an adequate number of characters to define a unique hash (the CLI could throw an error if you didnt type in enough characters for it to be unique)
I also think the expansion of @ tokens is neat... it is easier to look up and read a number than a hexidecimal hash (of 5-6 characters). Linux shells offer a similar concept using the ! character. From the bash man page:
The two or three characters which control history expansion  and  tokenization  (see
HISTORY  EXPANSION  below).  The first character is the history expansion character,
the character which signals the start of a history  expansion,  normally  `!'.

This is effectively what max has implemented... You can invoke an old command by finding its position in the history and issuing !<number>. Many (most?) shells support it... it is a cool feature if you arent using it already.
If we had infinite resources... it would be cool to implement both :-P

It would be handy to type in the first few characters of the hexidecimal hash and either complete it... or trust that you typed in an adequate number of characters to define a unique hash (the CLI could throw an error if you didnt type in enough characters for it to be unique)

I played around with that for a bit but I couldn't get the auto-completion working like that in bash.

Can the CLI always know what type the thing it's expanding /should/ have?

That is an interesting thought. It definitely should work for the 'primary' entities - service, order, artifact. Need to check if there are issues with 'aspect'. Still not sure how confusing having two identical @x in a single command where each one refers to something different.

Another option is we don't use numbered tokens, and instead do what git does, with truncations of the hashes.

How exactly would that work? Would we display the full urn, but then can use the abbreviated version in future calls? Also, remember not everything follows the urn:...:_uuid_ format.

I also think the expansion of @ tokens is neat...
What exactly do you mean by that - expansion ?

Still not sure how confusing having two identical @x in a single command where each one refers to something different.

I think that is a valid critique.
A user who understands the interface could learn to keep that context in their head... but it isn't immediately obvious. Could you make that difference explicit by prepending each @x with the 'primary entity'?
So the following:
ivcap order create -n "test image order" @1 msg ="Hello IVCAP" background-img=@1
becomes something like:
ivcap order create -n "test image order" service@1 msg ="Hello IVCAP" background-img=artifact@1

I like @asherbender's suggestion to add the type to the @ syntax. Could we combine this with type inference where available, so that adding the type is optional? The user could use these interchangeably:
ivcap order create -n "foo" @service:1 msg="Hello IVCAP" background-img=@artifact:1

ivcap order create -n "foo" @1 msg="Hello IVCAP" background-img=@1

If the type inference doesn't work then the CLI can suggest the user disambiguate with the full @ syntax.


#issue v3 Demo Audio Along with Screen Sharing : audio is not two way ,

hello , we need something like this demo but  its not working well .
in description it says:
All users can NOT only view organizer's screen; but also they can talk with him.
but its not working .
the organizer can speak and everyone can hear it .
but organizer cannot hear anyone .
and its requies camera from everyone .
we change the require camera settings and now , its
html code  added:
'

'
            // ..................RTCMultiConnection Code.............

            var connection = new RTCMultiConnection();
            connection.socketURL = '/';
            connection.socketMessageEvent = 'wtest';

            connection.session = {

				audio: 'two-way',
				screen: true,
                                oneway: true
            };

            connection.sdpConstraints.mandatory = {
                OfferToReceiveAudio: true,
                OfferToReceiveVideo: false
            };

	     connection.mediaConstraints = {
                audio: true,
                video: false
            };

            connection.videosContainer = document.getElementById('videos-container');
            connection.audiosContainer = document.getElementById('audios-container');

            connection.onstream = function(event) {

				var width = event.mediaElement.clientWidth ||
                                connection.videosContainer.clientWidth;
				var mediaElement = getMediaElement(event.mediaElement, {
                                title: event.userid,
                                buttons: ['full-screen'],
                                width: width,
                               showOnMouseEnter: false
                });

				if(event.stream.isAudio) {
                    connection.audiosContainer.appendChild(mediaElement);
                }
				if(event.stream.isScreen) {
                    connection.videosContainer.appendChild(mediaElement);
                }

                setTimeout(function() {
					mediaElement.media.play();
                }, 5000);

				mediaElement.id = event.streamid;
            };

            // Using getScreenId.js to capture screen from any domain
            // You do NOT need to deploy Chrome Extension YOUR-Self!!
            connection.getScreenConstraints = function(callback, audioPlusTab) {
                if (isAudioPlusTab(connection, audioPlusTab)) {
                    audioPlusTab = true;

                }

                getScreenConstraints(function(error, screen_constraints) {
                    if (!error) {
                        screen_constraints = connection.modifyScreenConstraints(screen_constraints);
                        callback(error, screen_constraints);
                    }
                }, audioPlusTab);
            };
            connection.onstreamended = function(event) {
                var mediaElement = document.getElementById(event.streamid);
                if(mediaElement) {
                    mediaElement.parentNode.removeChild(mediaElement);
                }
            };

so our code is the same we just take audio along with screen sharing and manipulate it but it is the same
organizator can send screen and voice
but noone cant talk with him only can hear
we need to everyone talk with organizator and organizator talk with everyone
basicly : one to many audio and screen topology
but we might have problems
connection.session = {
			audio: 'two-way',
			screen: true,
                            oneway: true
        };

on session if we delete oneway , everyone can talk the organizator but they have to also send their screens , audio : 'two-way' only working if we dont have oneway: true part of our code .
oneway:true overloading the audio so audio : two way or true or anything going to useless ,
thats our problem how can we fix it ?

solved this problem, i have the same error.

i have same error please solve this problem.


Incorrect popper placement when using Lightning Web Components on Safari/Firefox

Hi there,
I've been having some issues where the transform: translate() would have some incorrect calculations on the popper element when using Safari/Firefox (but would be ok with chrome). Specifically, the popper would open up when a button is clicked but would be near the bottom of the page (Y-axis would be calculated WAY off only on Safari or Firefox).
I know this is a shot in the dark, but does anyone have an idea what might be causing this?
unfortunately, I am having issues trying to reproduce the issue with Codepen as I am using Lightning Web Components.
Thanks!

Sales Force implements its own DOM interface and it's not always up to spec. We introduced workarounds to overcome its issues in the past but I don't plan to do anything like that anymore.


RDF Vocabulary without class for domain of properties

Dublin Core Metadata Terms (dcterms is a very widely used RDF vocabulary. One reason it is so widely used is that it does not declare any class to be the domain for the properties -- so they can be used on any class.
On importing dcterms into DESM, it does not seem possible to select properties without first selecting a class as the domain, thus most dcterms properties cannot be imported. => we need a way to select any property regardless of whether the file imported declares a domain for them.

@philbarker - I think this is an enhancement since this use case was never identified previously.

@philbarker have you tried uploading the DESM schema?

@jbaird123 OK, that's fair enough.
There is a workaround which is simply to edit the schema to add a domain to the properties.
@jeannekitchens I'm not sure what you mean by DESM schema?

Sorry...I meant DCMI:)

Suggested solution

[ ] for RDF vocabularies, always allow a "None - properties with no domain declared" option for the domain alongside the list of classes.
[ ] for any vocabulary, allow an "All" option to import all properties.

@excelsior relevant part of workflow is on schema upload, after inputting schema details and hitting the big red button, for this file
https://www.dublincore.org/specifications/dublin-core/dcmi-terms/dublin_core_terms.ttl I see

Most properties do not have any of those classes in their domain.

Yup, this looks right and behaves as expected. Thank you.


Allow 'Exceptions' For Kill Count Notifier

Checklist

[X] I've searched the issues and pull requests for similar looking suggestions.
[X] I've checked the Unreleased section of the changelog for newly added features that sound like my suggestion.

Describe your Suggestion
Allow users to enter exceptions that doesn't follow the Kill Count Interval value; similar to Ignored Region IDs in the Death setting. Whether this setting would just send a notification for every time the boss is killed or if possible allow the user to also define what the Kill Count Interval would be for that exception.
Reasoning
This would be useful in situations where a group has their Kill Count Interval set to 100 to avoid spam notifications for the average boss but being able to set Kill Count Interval to say 5 for Zuk/Colosseum would be a nice to have.

Possible syntax
Boss:interval - e.g., Nex:5
If :interval suffix is missing, assume :1
Longer example
Hespori:20
Nex
Vorkath:100
Zulrah:1

but if the same boss is listed twice with different intervals, it's unclear which to use

This setting may be too complicated to accurately explain to users
Since you have programming experience, I would recommend setting your interval to 1 and write code for a custom webhook handler to forward the notification if your boss-specific interval is met

This setting may be too complicated to accurately explain to users
Since you have programming experience, I would recommend setting your interval to 1 and write code for a custom webhook handler to forward the notification if your boss-specific interval is met

Thanks for the information. This may be a good opportunity for me to explore plugin development! Do you have any recommended guides/docs for creating custom webhooks?

To clarify, rather than writing runelite plugin code, you can write a HTTP request handler that accepts Dink's POST requests and performs custom logic. If you set your kc interval to 1, every kc would hit your logic, which can selectively forward the notification to discord webhook URLs depending on whether the kc is divisible by arbitrary boss-specific values.
To host your webhook handler for free, you can use Cloudflare Workers or AWS Lambda or a localhost webserver.
To parse Dink requests, I recommend you read these docs: https://github.com/pajlads/DinkPlugin/blob/master/docs/json-examples.md#structure

To clarify, rather than writing runelite plugin code, you can write a HTTP request handler that accepts Dink's POST requests and performs custom logic. If you set your kc interval to 1, every kc would hit your logic, which can selectively forward the notification to discord webhook URLs depending on whether the kc is divisible by arbitrary boss-specific values.
To host your webhook handler for free, you can use Cloudflare Workers or AWS Lambda or a localhost webserver.
To parse Dink requests, I recommend you read these docs: https://github.com/pajlads/DinkPlugin/blob/master/docs/json-examples.md#structure

Thank you for the clarification. I'm making good progress! I was wondering if there was some documentation for boss names? I.e. will Dink send me Inferno vs Tzkal-Zuk or Fortis Colosseum vs Sol heredit?

Thank you for the clarification. I'm making good progress! I was wondering if there was some documentation for boss names? I.e. will Dink send me Inferno vs Tzkal-Zuk or Fortis Colosseum vs Sol heredit?

Glad to hear! For the kill count plugin, it uses the exact name used in the chat message. The main exception to this is (Corrupted) Gauntlet: we report the NPC name instead (i.e., Crystalline Hunllef or Corrupted Hunllef). For your two examples, TzKal-Zuk and Sol Heredit are correct

Thank you for the clarification. I'm making good progress! I was wondering if there was some documentation for boss names? I.e. will Dink send me Inferno vs Tzkal-Zuk or Fortis Colosseum vs Sol heredit?

Glad to hear! For the kill count plugin, it uses the exact name used in the chat message. The main exception to this is (Corrupted) Gauntlet: we report the NPC name instead (i.e., Crystalline Hunllef or Corrupted Hunllef). For your two examples, TzKal-Zuk and Sol Heredit are correct

Appreciate the information! I am getting a bit stuck on creating the img url to where when it sends to the webhook it posts the screenshot from Dink. Do you have any tips on how to tackle this? Does the img actually have to be hosted somewhere?

Ignoring the heresy that is my snake case, this is my node.js running express + multer to handle dink requests. I've stripped out the surrounding code, this should hopefully point you in the right direction.
This link has a lot of great documentation: https://expressjs.com/en/resources/middleware/multer.html
``
//Require express and multer (for multi-part webhook/API body handling)
const express = require("express");
const multer = require("multer");
//Initialize express as webhook_handler and define a port
const webhook_handler = express();
const PORT = 1234;
//Start express on the defined port
webhook_handler.listen(PORT, () => console.log(log_green,🚀 Webhook Listening-Server running on port ${PORT}.));
webhook_handler.use(express.json());
//Multer diskstorage for image up/down load to local
const storage = multer.diskStorage({
destination: './webhook_files',
filename: function (req, file, cb) {
cb(null, path.parse(file.originalname).name + "-" + Date.now() + path.extname(file.originalname));
}
});
//Multer file filter for extention
const file_filter = function (req, file, cb) {
const allowed_mimes = ['image/jpeg', 'image/jpg', 'image/png', 'image/gif'];
if (allowed_mimes.includes(file.mimetype)) {
cb(null, true);
} else {
cb(null, false);
}
};
//upload called when file receieved on webhook, file size limit and filter
//maximum file size is 15Mb
const upload = multer({
storage: storage,
limits: { fileSize: 15 ** 7 },
fileFilter: file_filter
}).single('file');
``

We define standard express server, define port, begin listening
We define the storage function: using multer we define the parameters for storing the file locally. For me this is on my destination ./webhook_files
We ensure that the name of the file is consistent naming wise and uses the same extension provided initially
We defined the file_filter function which is a security catch to ensure we are not saving anything outside of the desired file types.
We define the upload function which includes the previous functions, and also defines single file mode (avoids multiple files, again a minor security concern that we prevent).
Following this, I have some reasonably complex code that would be difficult to share here, but you can initialize a call with something like I do (calling to another file) using the originally defined webhook_handler and upload:
webhook_handler.post(webhook.route_param, upload, (...args) => another_file.execute(bot, sqlcon, upload, ...args));
In the separate file you will want to consider validation/other security to ensure you are handling "bad" requests or malicious requests.
You will want to consider then routing different things to different channels or servers depending on your use case (different GIMs/discords/whatever)

I would think this approach would work for me as well but something as simple as -
await fetch(WH_URL, {      method: "POST",      headers:  { 'content-type':'multipart/form-data; boundary=--------------------------807394529256768815026859'},      body: request, })
but I'm having no luck. I'm leaning towards the issue being with the body and how Discord is expecting a multipart/form-data post to look like.
I've tried with Dink sending the request and with Postman
POST / HTTP/1.1
User-Agent: PostmanRuntime/7.39.0
Accept: */*
Postman-Token: 7c33dadb-b80f-42ad-992e-729050ff32de
Host: <IP_ADDRESS>:3000
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Type: multipart/form-data; boundary=--------------------------740985452248073530185399
Cookie: __dcfduid=896cb03a1d2511ef85d3aa593567cc5c; __sdcfduid=896cb03a1d2511ef85d3aa593567cc5c8e91ab58643ba6ce1b7b7810faf449fea093a0b42c4ffd728b108ee2134e15bb
Content-Length: 501747

----------------------------740985452248073530185399
Content-Disposition: form-data; name="payload_json"
{"type":"KILL_COUNT","playerName":"LSx Swap","accountType":"NORMAL","dinkAccountHash":"","seasonalWorld":false,"extra":{"boss":"Sarachnis","count":691,"gameMessage":"Your Sarachnis kill count is: 691.","time":"PT32S","isPersonalBest":false},"content":"**LSx Swap** has defeated **Sarachnis** with a completion count of **691**","embeds":[]}
----------------------------740985452248073530185399
Content-Disposition: form-data; name="file"; filename="Untitled.png"
<Untitled.png>
----------------------------740985452248073530185399--

HTTP/1.1 200 OK
content-type: text/plain; charset=utf-8
content-length: 2
Date: Tue, 28 May 2024 19:31:43 GMT
Connection: keep-alive
Keep-Alive: timeout=72

don't think you can pass fastify's Request object as a fetch body. might be easier to yield http code 307 to trigger client-sided redirect: reply.code(307).redirect('WH_URL')

oh actually code 307 won't work because runelite uses an ancient okhttp version (3.14.9), and 307/308 POST are not supported until 4.6.0

if you're willing to wait, we can add logic to dink to workaround that okhttp bug

don't think you can pass fastify's Request object as a fetch body. might be easier to yield http code 307 to trigger client-sided redirect: reply.code(307).redirect('WH_URL')

I went ahead and started looking down this approach but testing with Postman for the time being. It does work however, if an image is sent with the redirect I get error Error: You cannot pipe to this stream after the outbound request has started.. No matter the flow of redirects. I.e. -
No img redirect -> No img redirect -> img redirect -> error
img redirect -> posts to Discord -> img redirect -> error
img redirect -> posts to Discord -> No img redirect -> posts to Discord
Pretty clear the image has something to do with it but I can't seem to find any similar situations online.

@jdanthdavis could you share your full js code? this problem is outside the realm of dink (can't replicate using a cloudflare worker), but maybe one of us can spot where things are going wrong

@jdanthdavis sample cloudflare worker handler https://gist.github.com/iProdigy/53654691a76679221a74fd442dea068a

Here is my current Cloudflare Worker - https://gist.github.com/jdanthdavis/647ab73453373f2381f357c9652b58b7
After reviewing yours I was able to get Postman to work flawlessly! Of course, an attempt with Dink didn't succeed but I'm sure your fix address that. Here is the Cloudflare log of the Dink post in case it could be helpful for you. At this point I think we just wait til the fix is published. I really appreciate your help @iProdigy and @APKiwi! It was my first time working with webhooks and workers and it was awesome to learn.
{
  "outcome": "ok",
  "scriptVersion": {
    "id": "redacted"
  },
  "scriptName": "redacted",
  "diagnosticsChannelEvents": [],
  "exceptions": [],
  "logs": [],
  "eventTimestamp":<PHONE_NUMBER>827,
  "event": {
    "request": {
      "url": "redactedCloudflareUrl",
      "method": "POST",
      "headers": {
        "accept-encoding": "gzip, br",
        "cf-connecting-ip": "redacted",
        "cf-ipcountry": "US",
        "cf-ray": "redacted",
        "cf-visitor": "{\"scheme\":\"https\"}",
        "connection": "Keep-Alive",
        "content-length": "556",
        "content-type": "multipart/form-data; boundary=bde7ef43-32e6-4137-babc-40c8e9e2b0f1",
        "host": "redactedCloudflareUrl",
        "user-agent": "RuneLite/<IP_ADDRESS>-b22f15e (Dink/1.x)",
        "x-forwarded-proto": "https",
        "x-real-ip": "redacted"
      },
      "cf": {
        "clientTcpRtt": 18,
        "longitude": "redacted",
        "httpProtocol": "HTTP/2",
        "tlsCipher": "redacted",
        "continent": "NA",
        "asn": 209,
        "clientAcceptEncoding": "gzip",
        "country": "US",
        "verifiedBotCategory": "",
        "tlsClientAuth": {
          "certIssuerDNLegacy": "",
          "certIssuerSKI": "",
          "certSubjectDNRFC2253": "",
          "certSubjectDNLegacy": "",
          "certFingerprintSHA256": "",
          "certNotBefore": "",
          "certSKI": "",
          "certSerial": "",
          "certIssuerDN": "",
          "certVerified": "NONE",
          "certNotAfter": "",
          "certSubjectDN": "",
          "certPresented": "0",
          "certRevoked": "0",
          "certIssuerSerial": "",
          "certIssuerDNRFC2253": "",
          "certFingerprintSHA1": ""
        },
        "tlsExportedAuthenticator": {
          "clientFinished": redacted,
          "clientHandshake": redacted,
          "serverHandshake": redacted,
          "serverFinished": redacted
        },
        "tlsVersion": "TLSv1.3",
        "city": redacted,
        "timezone": redacted,
        "colo": "BNA",
        "tlsClientHelloLength": "385",
        "edgeRequestKeepAliveStatus": 1,
        "postalCode": redacted,
        "region": redacted,
        "latitude": redacted,
        "requestPriority": "weight=16;exclusive=0;group=0;group-weight=0",
        "regionCode": redacted,
        "asOrganization": redacted,
        "metroCode": redacted,
        "tlsClientExtensionsSha1": redacted,
        "tlsClientRandom": redacted
      }
    },
    "response": {
      "status": 200
    }
  },
  "id": 0
}

@jdanthdavis glad to hear!
in terms of code review:

extra.playerName should be payload.playerName
nit: if you plan to handle other notification types in the future, i wouldn't store the kc-specific extra fields to variables before checking the notification type (but js is a loose language so your code is fine too)
nit: bossMap.has(bossName.toUpperCase()) can just be bossInterval !== undefined
THEATRE OF BLOOD HARD MORE has a typo (MORE should be MODE)
for your "special occasion" handling, bossName === 'SOL HEREDIT' and bossName === 'TZKAL-ZUK' will always evaluate to false since you don't have bossName.toUpperCase()

lastly, since you're on cf workers now, if you don't want to wait for the okhttp patch, you can switch back to return await fetch(isPb ? PB_URL : KC_URL, request);

@iProdigy thanks for the feedback; some really good catches their as well! I went ahead and made the adjustments - https://gist.github.com/jdanthdavis/647ab73453373f2381f357c9652b58b7
It is fully working now as intended. However, I did receive this 500 error stating This ReadableStream is disturbed (has already been read from), and cannot be used as a body. The solution was to change
const form = await request.formData(); to
const form = await request.clone().formData();
Again, greatly appreciate all the help with this!

@iProdigy Quick question, we ran into an instance where the special occasion logic triggered and we saw some unexpected results. A player got their first KC but it also was their personal best since it's their first completion. However, the player double-deathed with the boss. What's interesting is that we had the message for a personal best but it was sent to our KC URL.
I've setup a fiddle with manipulated data and it works as intended. Did we see this because Dink would be sending two different payloads for this scenario? I.e it sent the personal best payload first and then it would have sent the kc payload after? I'm just confused on how we received the dink message for a personal best but the return await fetch(!isPb ? KC_URL : PB_URL, request); went to the PB_URL.
Fiddle

Which aligns with what I intended but we received the personal best message from Dink. Shouldn't Dink have sent 2 payloads in this scenario?

It's a single kill so dink sends 1 kc notif
The forwarded message is missing the specific kc because the user has modified the PB Notification Message to not include %COUNT%

I see. We achieved a PB so dink sends the PB message but we modified our message to not include %COUNT% which by default is included in the Dink message.
So, to achieve the logic/messages I want I would have to actually modify the message from Dink once I've done my logic? Which won't be possible until your PR is merged if I'm not mistaking.

you don't need the 307/308 pr to fix this
The easiest approach is to just have users update the value of PB Notification Message. For example, folks could just append  at a completion count of **%COUNT%**. If your users are doing ::dinkimport, you can update the json to make this easier on users
Alternatively, you can use the values in the extra object to construct the message on your server instead, and POST directly to the discord url. This is more involved and folks typically utilize a discord webhook library to create the embed (but it seems like you're not doing embeds, so prolly easy enough without a lib)

The easiest approach is to just have users update the value of PB Notification Message. For example, folks could just append  at a completion count of %COUNT%. If your users are doing ::dinkimport, you can update the json to make this easier on users

100% is the easiest solution for this. This is definitely a super picky scenario (plus learning, ya know?)

Alternatively, you can use the values in the extra object to construct the message on your server instead, and POST directly to the discord url. This is more involved and folks typically utilize a discord webhook library to create the embed (but it seems like you're not doing embeds, so prolly easy enough without a lib)

I am able to deconstruct everything from extra that I need and build specific messages depending on our logic in utils.js. However, what I am struggling with is creating the correct structure for the payload to the webhook. From reading the official Discord docs it seems not utilizing embeds is a bit of an odd approach these days so the documentation is subpar for constructing a payload in this manner. I'm finding mixed information between needing content, message instead of content, or even using payload_json for the body(?).
My next approach will be to review the logs of my worker to see what the request from Dink looks like. Cloudflare is a bit lame in the fact that you need to pay for their upgraded plan to actual have logs that are within your worker and not just the real time logs that only shows the status, trigger, url, ect.

yeah the discord docs may not be so clear:

You should use FormData to prepare the POST body. The first part should be called payload_json. The second optional part can be named file, which would contain the screenshot (when available)
For the json part, you just need {"content": "your message here"}

cloudflare's real-time logs do report console.log calls, so you can just leave that running until a notification comes through (and avoid needing to upgrade to a paid worker)

Happy to say with your help we're making wonderful progress. We're getting successful posts with Dink with the custom messages. Albeit, I'm currently passing a random blob for the img since I'm issues grabbing the Dink img. Would that be from:
const file = await request.file;

I also couldn't find the property names of file in the Dink docs either. request.file.name(?)

you should use const file = form.get("file") since you already have const form = await request.clone().formData();
then:
const formData = new FormData();
formData.append("payload_json", JSON.stringify({"content": "your message here"}));
if (file !== null) {
    formData.append("file", file)
}

I really appreciate your help! With mock data we're looking good. Getting each customer message sent to it's corresponding URL. If you have the time I'd love some feedback on my approach.
https://gist.github.com/jdanthdavis/647ab73453373f2381f357c9652b58b7

I really appreciate your help! With mock data we're looking good. Getting each custom message sent to it's corresponding URL. If you have the time I'd love some feedback on my approach.
https://gist.github.com/jdanthdavis/647ab73453373f2381f357c9652b58b7

yw! I'd add a null check before appending file (and make sure it can be reused across multiple requests)

Made those adjustments! We did just have an instance where a PB was obtained in game but for the time we posted PT21M35S which is just straight from extra.time. Does Dink handle the formatting here and applies it to the gameMessage before sending the payload? If that's the case and I'm not overlooking something I guess I'll have to do some cleaning of that string.
In the case I do need to clean the string myself; do you have an example of what extra,time will look like when the player has the precise timing in game enabled? I.e. PT21M35S10MS?

yeah we send extra.time as a ISO-8601 duration
with precise timing, it'd look like PT21M35.01S
one trick is to remove PT prefix and add a space after every letter (e.g., via regex)

Seems to be working flawlessly with mock data! Went with this solution -
let sanitizedTime = time.replace("PT", "").replace("S", "").replaceAll(regex, ":") for our preferred formatting. Thank you again, it's been a pleasure learning!

Hey @iProdigy, noticed something a bit weird with the payload for HMT PBs. We got a 5 man PB but I got isPersonalBest: false. Below are screenshots of the exact in-game message and the payload from that exact raid.

Could you screenshot the game tab where it says new personal best (rather than clan tab)

Could you screenshot the game tab where it says new personal best (rather than clan tab)

Sure!

Dink uses the last time posted in chat, so it's reporting 18:25.80, which technically wasn't a pb


[Addition]:

Hotspot b58 Addresses
11gRVmUxETzjQv3kVpALAusCkM3Wb1xgwBTSZQzCBkNJuKvS1NT
112QyJZzYBayzs7oSoovdz5ecewQxEsuTATxDxkGYiF6q68Megxd
11CfCnw7epNTUsykK1HwbrbyQxZbFnn9Djh5Gx6go7zkvdnNRPb
112T7TcbfYLsS231HDhazgtV3WqzG6ysWczn1qDR37EAbnXqyXzp
Discord Handle
berk s#7790
Reason(s)
When I check their nearest hotspots can't earn like them and nearest hopspots don't have withnesses like them.
They are new in their area and they earn multible times then their nearest hotprots. Most probably they are usind amplifier to hear hotspots. As usual 1.2dbi and 0m height antenna settings they have who are using amplifier.
Would you check and add them to deny list?

You might need to report those hs aswell because all in same place even beaconing with each others this is same person hs are Helium network reputation is hurt an seems like devs not care about it  in this project not need to amp or another things this is middle man project as you can find on github taking all witnesses through on online you can go Belguim with any hs down below and you can still witness in istanbul hs i know sound funny but it is  you will be suffer with 0.0004 hnt a day but the others will take 10 hnt a day. So do your packet forwarder as and duck the system who care
Happy Mulberry Grasshopper
Sneaky Pecan Opossum
Young Steel Gorilla
Short Mercurial Raccoon
Nice Eggshell Lemur
Powerful Ash Ram
Odd Infrared Urchin**


Updated peerDependencies to support react 17

Can someone tell me the command the install this specific PR into my project?


Initial Prototype

This PR is here to track feedback on the initial prototype so far. API, possible suggestions, etc are very welcome! The aim is to make this library a temporary companion to arrow-core until context receivers are fully released on every platform (so that there are no pre-release binaries). Pointers to APIs in Arrow or Kotlin Stdlib that would benefit from contexts are welcome as well.

Using kotest to test context receiver Arrow functions doesn't feel like Arrow is a first-class citizen.
When we had Either<Error,A> as a return value we could simply call methods and use shouldBeRight() or shouldBeLeft().
With context receivers we need to setup a context to get the test compiling/executing, and need to capture the error path to validate it; happy-path result verification is cleaner as it's simply the return value.
This is one mechanism to make this work (see below for ideas on improvement):
   test("UUID converter works") {
            either {
                val uuid = UUID.randomUUID()
                converter.convert(ResolvedNodes.string(uuid.toString()), type, conversionContext())
                    .shouldBe(uuid)
            }.shouldBeRight()

            either {
                converter.convert(ResolvedNodes.string(""), type, conversionContext())
            }.shouldBeLeft()
                .shouldBeInstanceOf<ConversionError.TypeConversionError>()
                .message()
                .shouldBe(
                    "Error converting '' into 'java.util.UUID': IllegalArgumentException: Invalid UUID string: "
                )
        }

...with a few helper shims its a bit cleaner:
 test("UUID converter works") {
            shouldBeRight {
                val uuid = UUID.randomUUID()
                converter.convert(ResolvedNodes.string(uuid.toString()), type, conversionContext())
                    .shouldBe(uuid)
            }

            shouldBeLeft {
                converter.convert(ResolvedNodes.string(""), type, conversionContext())
            }.shouldBeInstanceOf<ConversionError.TypeConversionError>()
                .message()
                .shouldBe(
                    "Error converting '' into 'java.util.UUID': IllegalArgumentException: Invalid UUID string: "
                )
        }

        public inline fun <Error, A> shouldBeRight(@BuilderInference block: Raise<Error>.() -> A): A = either(block).shouldBeRight()
        public inline fun <Error, A> shouldBeLeft(@BuilderInference block: Raise<Error>.() -> A): Error = either(block).shouldBeLeft()

It would be ideal if tests using Arrow/Either were symmetrical with raw Kotlin/ code w/ exceptions, specifically:
a) There would be a mechanism to execute code in a context w/o having to explicitly (and repeatedly) establish that context;
b) As with Kotest exception handling in tests, any errors that are raised fail the test;
c) Some form of shouldRaise<ErrorType>, similar to shouldThrow<ExceptionType> to handle the error path.
Perhaps parts of this exist already and my evolving Arrow journey hasn't taken me there yet.

@cloudshiftchris Very insightful. I've added shouldRaise and shouldSucceed to the test code and they seem to clean things up well.
Those functions should be eventually added to kotest-extensions-arrow, but I'd like to try them out here first before I suggest them

@kyay10 thanks for the quick turnaround!
Dropped those implementations in to compare against what we've stubbed out internally - comparison w/ commentary on a few actual tests below.
Don't think we're there yet - the mixing of "providing a context" with "checking assertions" is somewhat problematic.
This is syntactically ideal - separating context receiver from assertion - with unknowns (for me) on how to have kotest provide a context receiver:
        // doesn't compile
        // this would <somehow> provide a context for the test *and* ensure that no errors were raised
        // (that otherwise weren't handled in the block, for example, by `shouldRaise`)
        // this would eliminate the need for the `shouldSucceed`
        context(Raise<ConfigurationError>)
        test("sample magic kotest / arrow voodoo") {
           // uses context receiver provided above
            converter.convert(ResolvedNodes.boolean(true), type, conversionContext()).shouldBe(true)

            // symmetrical to `shouldThrow<ExceptionType>`
            // uses the context provided above - only responsibility is verifying error type
            shouldRaise<ConversionError.TypeConversionError> {
                converter.convert(ResolvedNodes.string("abc"), type, conversionContext())
            }
                .message()
                .shouldBe(
                    "Error converting 'abc' into 'kotlin.Boolean': Expected value to be in [true, t, 1, yes] or [false, f, 0, no]"
                )

            // symmetrical to `shouldThrowAny`
            // uses the context provided above - only responsibility is verifying error raised
            shouldRaiseAny {
                converter.convert(ResolvedNodes.string("abc"), type, conversionContext())
            }
        }

Comparison:
          // using our internal `shouldSucceed` implementation
            // here it's used as a means to provide a context for the enclosed block, and verify that no errors were raised.
            shouldSucceed {
                converter.convert(ResolvedNodes.boolean(true), type, conversionContext()).shouldBe(true)
                converter.convert(ResolvedNodes.boolean(false), type, conversionContext()).shouldBe(false)
                converter.convert(ResolvedNodes.string("true"), type, conversionContext()).shouldBe(true)
                converter.convert(ResolvedNodes.string("false"), type, conversionContext()).shouldBe(false)
            }

            // could also use it to further chain assertions
            shouldSucceed {
                converter.convert(ResolvedNodes.boolean(true), type, conversionContext())
            }.shouldBe(true)

            // using arrow-context `shouldSucceed` implementation
            // here it's used as a means to provide a context for the enclosed block, and verify that no errors were raised.
            // this is functionally equivalent to the above, when used solely to provide a context
            arrow.context.shouldSucceed {
                converter.convert(ResolvedNodes.boolean(true), type, conversionContext()).shouldBe(true)
                converter.convert(ResolvedNodes.boolean(false), type, conversionContext()).shouldBe(false)
                converter.convert(ResolvedNodes.string("true"), type, conversionContext()).shouldBe(true)
                converter.convert(ResolvedNodes.string("false"), type, conversionContext()).shouldBe(false)
            }

            // could also use it to further chain assertions
            arrow.context.shouldSucceed {
                converter.convert(ResolvedNodes.boolean(true), type, conversionContext())
            }.shouldBe(true)

            // not loving this overload; it seems problematic to combine `shouldBe` comparison with the
            // context provided by `shouldSucceed` - why `shouldBe` and not other assertions?
            // does this imply that only a single assertion should be made within the context?
            // perhaps we leave validation of the result to the caller, and only provide the context?
            arrow.context.shouldSucceed(true) {
                converter.convert(ResolvedNodes.boolean(true), type, conversionContext())!!
            }

            // using our internal `shouldRaise` implementation
            // this requires two type arguments (perhaps someone with better generics-fu can improve on this)
            // one argument for the type of error expected, and one (supertype) for the context to execute within
            // aside from the second type argument this is equivalent to `shouldThrow<ExceptionType>` for non-fp code
            // as with `shouldThrow` we can further chain assertions on the raised error (which is smart-casted to the expected type)
            shouldRaise<ConversionError.TypeConversionError, ConfigurationError> {
                    converter.convert(ResolvedNodes.string("abc"), type, conversionContext())
                }
                .message()
                .shouldBe(
                    "Error converting 'abc' into 'kotlin.Boolean': Expected value to be in [true, t, 1, yes] or [false, f, 0, no]"
                )

            // using arrow-context `shouldRaise` implementation
            // this doesn't compile as the expected error type is a subtype of the context type
            // nor does this `shouldRaise` implementation check that the raised error is of the expected type
            arrow.context
                .shouldRaise<ConversionError.TypeConversionError> {
                    converter.convert(ResolvedNodes.string("abc"), type, conversionContext())
                }
                .message()
                .shouldBe(
                    "Error converting 'abc' into 'kotlin.Boolean': Expected value to be in [true, t, 1, yes] or [false, f, 0, no]"
                )

            // using arrow-context `shouldRaise` implementation
            // this compiles but is not equivalent to `shouldThrow` - its the same behaviour as `shouldThrowAny`
            // this isn't equivalent to our internal `shouldRaise` implementation, as it doesn't check that the raised error is of the expected type
            // (this could be done with an additional .shouldBeInstanceOf check, but the goal was symmetric with `shouldThrow<ExceptionType>`
            arrow.context
                .shouldRaise {
                    converter.convert(ResolvedNodes.string("abc"), type, conversionContext())
                }
                .message()
                .shouldBe(
                    "Error converting 'abc' into 'kotlin.Boolean': Expected value to be in [true, t, 1, yes] or [false, f, 0, no]"
                )

            // not loving this overload, for similar reasons as noted above for `arrow.context.shouldSucceed`
            // providing an instance for `shouldBe` isn't always desired - in this case, it's problematic to create an instance of the expected error type
            // would prefer to leave further validation of the error to the caller
            // (this doesn't compile)
            arrow.context
                .shouldRaise(ConversionError.TypeConversionError(...)) {
                    converter.convert(ResolvedNodes.string("abc"), type, conversionContext())
                }
                .message()
                .shouldBe(
                    "Error converting 'abc' into 'kotlin.Boolean': Expected value to be in [true, t, 1, yes] or [false, f, 0, no]"
                )

            public inline fun <Error, A> shouldSucceed(@BuilderInference block: Raise<Error>.() -> A): A =
                recover(block) { throw AssertionError("Expected to succeed, but raised $it") }

            public inline fun <reified ExpectedError : Error, Error : Any> shouldRaise(
                @BuilderInference block: Raise<Error>.() -> Unit
            ): ExpectedError {
                recover(block) { error ->
                    if (error is ExpectedError) return error
                    throw AssertionError(
                        "Expected raise of ${ExpectedError::class.bestName()} but ${error::class.bestName()} was raised: $error"
                    )
                }
                throw AssertionError("Expected raise of ${ExpectedError::class.bestName()} but no error was raised.")
            }

This is closer - adding withContext around the whole test removes the need for shouldSucceed; haven't yet figured out how to reduce shouldRaise to a single type parameter.
 test("Boolean converter works") {
            withContext {
                converter.convert(ResolvedNodes.boolean(true), type, conversionContext())
                    .shouldBe(true)
                converter.convert(ResolvedNodes.boolean(false), type, conversionContext())
                    .shouldBe(false)
                converter.convert(ResolvedNodes.string("true"), type, conversionContext())
                    .shouldBe(true)
                converter.convert(ResolvedNodes.string("false"), type, conversionContext())
                    .shouldBe(false)

                shouldRaise<ConversionError.TypeConversionError, ConfigurationError> {
                    converter.convert(ResolvedNodes.string("abc"), type, conversionContext())
                }
                    .message()
                    .shouldBe(
                        "Error converting 'abc' into 'kotlin.Boolean': Expected value to be in [true, t, 1, yes] or [false, f, 0, no]"
                    )
            }
        }

        public inline fun <Error> TestScope.withContext(@BuilderInference block: Raise<Error>.() -> Unit): Unit =
            recover(block) { throw AssertionError("Expected to succeed, but raised $it") }

        public inline fun <reified ExpectedError, Error : Any> Raise<Error>.shouldRaise(
            @BuilderInference block: Raise<Error>.() -> Unit
        ): ExpectedError {
            recover(block) { error ->
                if (error is ExpectedError) return error
                throw AssertionError(
                    "Expected raise of ${ExpectedError::class.bestName()} but ${error::class.bestName()} was raised: $error"
                )
            }
            throw AssertionError("Expected raise of ${ExpectedError::class.bestName()} but no error was raised.")
        }

Check the new commit! Thank you for the great suggestions. Hopefully, kotest can support something similar to ExtendWith from junit. I don't think there's a way to reduce shouldRaise to one type parameter. The reasons are complex, but Raise should never support a recover method that reuses an out-there Raise instance. In other words, every Raise method must create its own Raise instance if it wishes to grab an Error value out of it. Thus, shouldRaise must create its own instance, and thus it needs knowledge of the outer error type.

@kyay10 amazing.  thank you for the context (pun intended) around Raise - had dug into it a bit and figured it was special.
On that same topic (of requiring a specific Raise instance) - not sure if this is solvable but ran into a few challenges (of my own making) that took a bit to figure out, perhaps Arrow/compiler could help somehow.
This is a working test:
 test("Boolean converter works") {
            withContext {
                converter
                    .convert(ResolvedNodes.boolean(true), type, conversionContext())
                    .shouldBe(true)

                shouldRaise<ConversionError.TypeConversionError, ConfigurationError> {
                    converter.convert(ResolvedNodes.string("abc"), type, conversionContext())
                }
                    .message()
                    .shouldBe(
                        "Error converting 'abc' into 'kotlin.Boolean': Expected value to be in [true, t, 1, yes] or [false, f, 0, no]"
                    )
            }
        }

...however, I struggled to get it working, having accidentally done this (note the second type parameter on shouldRaise):
        test("Boolean converter works") {
            withContext {
                converter
                    .convert(ResolvedNodes.boolean(true), type, conversionContext())
                    .shouldBe(true)

                shouldRaise<ConversionError.TypeConversionError, ConversionError> {
                    converter.convert(ResolvedNodes.string("abc"), type, conversionContext())
                }
                    .message()
                    .shouldBe(
                        "Error converting 'abc' into 'kotlin.Boolean': Expected value to be in [true, t, 1, yes] or [false, f, 0, no]"
                    )
            }
        }

...because the types are all in the same hierarchy the compiler never blinked; at runtime, where shouldRaise would expect to fail, the internals never tripped as the block being executed was done so in the outer context (as the inner context was not of the correct type).  Took a bit to figure out.  Understand the need for individual Raise blocks - perhaps there is a way to enforce that the blocks must be of the same type.
New code looks great!  A small amount of code that goes a long way.
Only a minor naming consistency item, if we desire symmetry with kotest's shouldThrowAny, then this should be shouldRaiseAny: inline fun <OuterError> shouldRaise(block: context(Raise<OuterError>) () -> Any?): OuterError
I've asked on the #kotest slack channel around context-receiver options: https://kotlinlang.slack.com/archives/CT0G9SD7Z/p1705370189026969.

@cloudshiftchris with the risk of slightly derailing the conversation, I want to point out that you can define a Raise subclass, mark it with a @DslMarker annotation, and use it in withContext and shouldRaise. This would prevent outer calls to the withContext instance from within shouldRaise. Also note that then you can define shouldRaise inside of that Raise subclass, and make its type parameter invariant, and hence you won't face this issue, and you'll have a shouldRaise that only takes 1 type argument.

@kyay10 thanks for the guidance & commit showing how that would work.
Back-ported for Arrow 1.2 / Kotlin 1.9.x - works well, will a minor annoyance of failure messages being nested due to two levels or Raise (DefaultRaise from recover and custom TestingRaise) in withContext2, likely due to my uncertainty on how to use the custom Raise w/o something like a RaiseResolver.
java.lang.AssertionError: Expected to succeed, but raised TypeConversionError(Error converting 'abc' into 'kotlin.Boolean': AssertionError: Expected to succeed, but raised TypeConversionError(Error converting 'abc' into 'kotlin.Boolean': Expected value to be in [true, t, 1, yes] or [false, f, 0, no]))

public inline fun <Error> TestScope.withContext2(
    @BuilderInference block: (@TestingRaiseDsl TestingRaise<Error>).() -> Unit
): Unit {
    recover<Error, Unit>({ block(TestingRaise()) }) { fail("Expected to succeed, but raised $it") }
}

@Target(AnnotationTarget.CLASS, AnnotationTarget.TYPE)
@DslMarker
public annotation class TestingRaiseDsl

public class TestingRaise<Error> : Raise<Error> {
    override fun raise(r: Error): Nothing = fail("Expected to succeed, but raised $r")

    public inline fun shouldRaiseAny(block: (@TestingRaiseDsl Raise<Error>).() -> Any?): Error {
        val result = recover(block) { error -> return error }
        fail("Expected to raise an error, but instead succeeded with $result")
    }

    public inline fun <reified ExpectedError : Any> shouldRaise(
        block: (@TestingRaiseDsl Raise<Error>).() -> Any?
    ): ExpectedError =
        shouldRaiseAny(block).shouldBeInstanceOf<ExpectedError>()

    public inline fun shouldRaise(
        expected: Error,
        block: context((@TestingRaiseDsl Raise<Error>)) () -> Any?
    ): Error =
        shouldRaiseAny(block) shouldBe expected
}

withContext2's body can simply be: block(TestingRaise())

🤦 yep, overthought the heck out of that one ;)  thank you.
That wasn't actually the issue, however: the messages are still nested - this likely due to my code raising an error internally, which is immediately turned into an AssertionError, however, that is in a catch( { ... code that raises ... } { raise an error here }
Not seeing a clean way around that, short of catch ignoring AssertionErrors.

This addresses it - leaving raised errors intact until the end of the context scope (i.e. letting them bubble up as they normally would), then (if present) fail the test.

public inline fun <Error> TestScope.withContext(
    @BuilderInference block: (@TestingRaiseDsl TestingRaise<Error>).() -> Unit
): Unit {
    recover({ block(TestingRaise(this)) }) { error ->
        fail("Expected to succeed, but raised $error")
    }
}

@Target(AnnotationTarget.CLASS, AnnotationTarget.TYPE)
@DslMarker
public annotation class TestingRaiseDsl

public class TestingRaise<Error>(private val outerRaise: Raise<Error>) : Raise<Error> {
    override fun raise(r: Error): Nothing = outerRaise.raise(r)

    public inline fun shouldRaiseAny(block: (@TestingRaiseDsl Raise<Error>).() -> Any?): Error {
        val result =
            recover(block) { error ->
                return error
            }
        fail("Expected to raise an error, but instead succeeded with $result")
    }

    public inline fun <reified ExpectedError : Any> shouldRaise(
        block: (@TestingRaiseDsl Raise<Error>).() -> Any?
    ): ExpectedError = shouldRaiseAny(block).shouldBeInstanceOf<ExpectedError>()

    public inline fun shouldRaise(
        expected: Error,
        block: (@TestingRaiseDsl Raise<Error>).() -> Any?
    ): Error = shouldRaiseAny(block) shouldBe expected
}


Wms tutorial

Completed the Web Map Service Automated Tutorials.
It is tested on Windows and Linux both.
As of now, the cursor is visible in Screen Recordings in Windows but in Linux, the cursor is not visible in the tutorials recording (But if screenrecorder.py is run separately, it works in Linux too), I don't understand the reason.
Therefore, in Linux, I have made the cursor fixed in a particular position on the screen.
((The pyautogui.position() in screenrecorder.py in line 115 was throwing runtime error when the tutorial_wms.py is executed on Linux.))

Tried with only one screen, have you an idea what gets wrong?
Exception :'select to open control' button/option not found on the screen.

Exception : 'Server\Layers' button/option not found on the screen.

Exception : 'WMS URL' editbox button/option not found on the screen.

Exception : 'Get capabilities' button/option not found on the screen.

Exception : 'Divergence Layer' option not found on the screen.

Exception : 'Layer filter editbox' button/option not found on the screen.
Process Process-2:
Traceback (most recent call last):
  File "/home/reimar/miniconda3/envs/risehr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/reimar/miniconda3/envs/risehr/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/reimar/PycharmProjects/2021/risehr/MSS/tutorials/tutorial_wms.py", line 174, in automate_waypoints
    if x is not None and y is not None:
UnboundLocalError: local variable 'x' referenced before assignment

Perhaps this is similiar that it relays on an US Keyboard
https://github.com/asweigart/pyautogui/issues/46

Perhaps this is similiar that it relays on an US Keyboard
asweigart/pyautogui#46

By changing my german keyboar to an US Keyboard it works.

please use the template header on all created py files

Okay. I will do that.

Perhaps this is similiar that it relays on an US Keyboard
asweigart/pyautogui#46

By changing my german keyboar to an US Keyboard it works.

Ohh. Yes, I guess it works only for US keyboard layout.


Fix billboard on matrix scaled nodes

could you fix the name of the PR ? :)

hehe it's ok now?

yep :) is there an example using it ?

Yep, Texts use billboarding, also I've been using this in production to draw billboards. The problem is that it was not respecting  MatrixTransforms scale.
I noticed it when developing my next PR about AutoTransform node. As it was not respecting the scale when  autoscaling to screen.
Also in that PR text supports characterSizeModes (http://trac.openscenegraph.org/documentation/OpenSceneGraphReferenceDocs/a00873.html#a80fb5ff01377d8ac78935653752fee82) and the example is updated.
If you prefer I can do it all in the same PR, and close this one.

Yeah if we can only have one PR for this stuff that would be great

oks, closing this one then.


Data associations failing

pydent version 1.0.3
plan = session.Plan.new(name="Associations Test")
plan.create()
plan.associate("foo", "bar")

produces
Traceback (most recent call last):
  File "test_associations.py", line 7, in <module>
    plan.associate("foo", "bar")
  File "/usr/local/lib/python3.8/site-packages/pydent/models/data_associations.py", line 60, in associate
    return self.session.utils.create_data_association(
  File "/usr/local/lib/python3.8/site-packages/pydent/interfaces.py", line 306, in create_data_association
    if data_association.id not in [da.id for da in model_inst.data_associations]:
AttributeError: 'NoneType' object has no attribute 'id'

This happens for associations on Plan or Operation.

Update: I get this error when I run point at my laptop installation (v2.8.1), or the Duke sandbox (v2.9.0), but not Nursery (v2.7.3)

And I am actually running pydent version 1.0.4

OK, my hypothesis is that this has something to do with the nginx
configuration
On Fri, Dec 11, 2020 at 12:48 PM Devin Strickland<EMAIL_ADDRESS>wrote:

Update: I get this error when I run point at my laptop installation
(v2.8.1), or the Duke sandbox (v2.9.0), but not Nursery (v2.7.3)
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/aquariumbio/pydent/issues/92#issuecomment-743417743,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABJZ3PJJQSZTJRIQ5BNDZITSUKADVANCNFSM4UXANH3Q
.

--
Ben Keller
Aquarium Project Lead
Klavins Lab
Department of Electrical and Computer Engineering
University of Washington
<EMAIL_ADDRESS>        206-685-6726

This was an Aquarium bug in the JSON controller.
Fixed by https://github.com/aquariumbio/aquarium/commit/2c980e48a9be63d55be74229e2216f31ca1e744c


[BUG]

Describe the bug
java.lang.ClassNotFoundException: com.nvidia.spark.SQLPlugin when trying SparkSession build in local mode.
Steps/Code to reproduce bug
spark = SparkSession.builder
.config("spark.driver.extraClassPath", classpath)
.config("spark.executor.extraClassPath", classpath)
.config("spark.sql.repl.eagerEval.enabled", True)
.config("spark.sql.repl.eagerEval.maxNumRows", 10)
.config("spark.sql.execution.arrow.enabled", "true")
.config("spark.driver.memory", f"{spark_driver_size}g")
.config("spark.driver.maxResultSize", f"{max_result_size}g")
.config("spark.rapids.sql.enabled", "true")
.config("spark.rapids.sql.explain", "ALL")
.config("spark.jars", "path/to/jar/rapids-4-spark_2.12-24.10.1.jar")
.config("spark.plugins", "com.nvidia.spark.SQLPlugin")
.getOrCreate()
Expected behavior
Looking to get the RAPIDS Accelerator For Apache Spark for interaction use with S3 data sources.
Environment details (please complete the following information)

Environment location: Standalone/Local
Running on Sagemaker Jupyter Notebook
Scala V 2.12.15
Spark V 3.3.0

Additional context
Have added the .jar file to the class path and am able pick it up within a  spark.conf print dump.
What am I doing wrong in this situation? Seems like everything should play well together with us being within the compatibility range.

Double check the location of the jar. In local mode when we do not need to optimize RapidsShuffleManager we do not require the plugin jar to be on extraClassPath. You can use --jars or --packages.
The equivalent config is spark.jars and spark.jars.packages, respectively.   You can check this example notebook https://github.com/NVIDIA/spark-rapids-examples/blob/branch-24.12/examples/SQL%2BDF-Examples/tpcds/notebooks/TPCDS-SF10.ipynb
Specifically
.config('spark.jars.packages', f"com.nvidia:rapids-4-spark_{scala_version}:{rapids_version}")

I followed the eaxample you provided, and I took the jar file off the class path (made another dir to hold it).
spark_jar_path = "SageMaker/kernels/nvidia-rapids-py310/lib/python3.10/site-packages/sagemaker_pyspark/extra_jars/rapids-4-spark_2.12-24.10.1.jar"
scala_v = '2.12'
rapids_v = '24.10.1'
spark = SparkSession.builder \
    .config("spark.driver.extraClassPath", classpath) \
    .config("spark.executor.extraClassPath", classpath) \
    .config("spark.sql.repl.eagerEval.enabled", True) \
    .config("spark.sql.repl.eagerEval.maxNumRows", 10) \
    .config("spark.sql.execution.arrow.enabled", "true") \
    .config("spark.driver.memory", f"{spark_driver_size}g") \
    .config("spark.driver.maxResultSize", f"{max_result_size}g") \
    .config("spark.rapids.sql.enabled", "true") \
    .config("spark.rapids.sql.explain", "ALL") \
    .config("spark.plugins", "com.nvidia.spark.SQLPlugin") \
    .config("spark.jars", spark_jar_path) \
    .config("spark.jars.packages", f"com.nvidia:rapids-4-spark_{scala_v}:{rapids_v}") \
    .getOrCreate()

When I run, I get the following runtime error "RuntimeError: Java gateway process exited before sending its port number"
:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
Exception in thread "main" java.lang.RuntimeException: [unresolved dependency: com.nvidia#rapids-4-spark_2.12;24.10.1: not found]
at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1456)
at org.apache.spark.util.DependencyUtils$.resolveMavenDependencies(DependencyUtils.scala:185)
at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:308)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:901)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Just adding that I have been following the initial install and configuration steps from this guide:
https://docs.nvidia.com/spark-rapids/user-guide/latest/getting-started/on-premise.html#local-mode

You do not need both at the same time: rapid-4-spark added via  spark.jars  and  via spark.jars.packages. If added via spark.*.extraClassPath then no need to use spark.jars
If you want to overcome this
Exception in thread "main" java.lang.RuntimeException: [unresolved dependency: com.nvidia#rapids-4-spark_2.12;24.10.1: not found]

since it looks like there is no default Maven repo configure in your environment some repo via spark.jars.repositories such as https://repo1.maven.org/maven2
if you confirm the location of the jar (maybe via !ls location in the notebook  you could resort to configuring extraClassPatrh. Does the path work relative to PWD of the kernel?


Add info on HTTP method types sent from the Client lib. Change the Cheatsheet default tab.

Codecov Report

Merging #132 into master will increase coverage by 2.12%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #132      +/-   ##
==========================================
+ Coverage   28.16%   30.28%   +2.12%
==========================================
  Files         102      114      +12
  Lines        2805     4200    +1395
==========================================
+ Hits          790     1272     +482
- Misses       2015     2928     +913

@adamwardecki are we waiting for something?


Finalize Early Adopter Workshop agenda for early January

Describe the issue
We have two Early Adopter teams currently scheduled to begin work in our compliant Landing Zone, starting in earnest in January. The Health Team (Freshworks - COVID Traveler Screening app) has requested a workshop on December 15. We've set expectations that the Landing Zone and Cloud Pathfinder support team won't be fully available in January, and their requirements for the workshop are minimal as they've already been building and running this app in AWS for several months. Therefore, we need an agenda tuned to this context for December 15
Which Sprint Priority is this issue related to?
1
Additional context
Add any other context, attachments or screenshots
Definition of done

collaborate with Julian on building the agenda and supporting slide deck
review with Gen and incorporate her input
review with Pam Smith and incorporate her input

Update Health to Jan 6 or 7

Still need to schedule Mines

Done, presented to Health today and have date lined up for Mines right after their release

Done, presented to Health today and have date lined up for Mines right after their release


EDM-291: Update CLI to handle OIDC login

Update kind deployment to allow auth
This PR updates auth logic to allow loggin in to OIDC based providers (keycloak). It also adds a new web flow where user logins via browser.
OIDC login via browser
login_oidc.webm
OCP login via browser
login_ocp.webm

followup https://github.com/flightctl/flightctl/pull/429


Add the pull to refresh for iOS

Summary

At this moment there's no way of having the pull to refresh feature on both iOS and Android (excluding wrapping WebView around ScrollView). This PR implements PTR for iOS.
It's quite difficult to do the same for Android without changing lots of code, though I'm trying to bring SwipeRefreshLayout but it will take some time.
For now, I think it makes sense to add this feature at least for iOS.
Test Plan

What's required for testing (prerequisites)?
Add the pullToRefreshEnabled property to WebView
What are the steps to reproduce (after prerequisites)?
Set the property to true
Compatibility

OS
Implemented

iOS
✅

Android
❌

Checklist

[X] I have tested this on a device and a simulator
[X] I added the documentation in README.md
[ ] I mentioned this change in CHANGELOG.md
[X] I updated the typed files (TS and Flow)
[ ] I added a sample use of the API in the example project (example/App.js)

really waiting for this feature. Any timeline for merge?

@svbutko Can you resolve conflicts and address @xilin's comment?

@svbutko Can you resolve conflicts and address @xilin's comment?

Yes, sure, I'll resolve it today a little bit later

Looks like we have conflicts again, sorry @svbutko !

@jamonholmgren, @xilin, @Titozzz resolved the conflicts, sorry that took so long 😔

Don't be sorry for doing open source stuff 🥰 thank you for your contribution


Tenth Korean l10n work for release 1.18

Translate concepts/extend-kubenetes/service-catalog.md in Korean (#23233)
Translate concepts/configuration/secret.md into Korean (#23110)
Translate tasks/extend-kubernetes/setup-extension-api-server.md into … (#22574)
Update outdated files in dev-1.18-ko.10 branch (#23013)

Co-authored-by: Seokho Son<EMAIL_ADDRESS>Co-authored-by: Jerry Park<EMAIL_ADDRESS>Co-authored-by: santachopa<EMAIL_ADDRESS>

/assign

/lgtm
/approve


Revert pr 3480

Add account id parameter back into curl commands
Create generic steps to pull Harness CI images from a private registry (separate from the STO topic).

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app


utility: change stale interrupted time from 48 to 6 hours

Signed-off-by: Jonathan Holloway<EMAIL_ADDRESS>Description
Change stale interrupted build time from 48 hours to 6 hours.
Fixes # (issue)
Type of change
What is it?

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Documentation update
[ ] Tests update
[x] Refactor

Checklist:

[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[x] I run go fmt ./... to check that my code is properly formatted
[x] I run go vet ./... to check that my code is free of common Go style mistakes

Test results are available in Ibutsu. The test run IDs are:
5659b05e-2712-4594-a434-4a2d3ff50b3b
8740b0bf-92af-4cc8-9f01-d99b0a6addc8

Codecov Report

Merging #985 (aa7b96a) into main (4dbe08e) will increase coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #985      +/-   ##
==========================================
+ Coverage   30.81%   30.84%   +0.03%
==========================================
  Files          45       45
  Lines        5475     5475
==========================================
+ Hits         1687     1689       +2
+ Misses       3549     3548       -1
+ Partials      239      238       -1

Impacted Files
Coverage Δ

pkg/services/devices.go
60.61% <0.00%> (+0.41%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4dbe08e...aa7b96a. Read the comment docs.

Test results are available in Ibutsu. The test run IDs are:
2ceef792-5478-4ff0-ac18-0f07a11fe074
c1faaf22-bfb7-446d-b20f-016b14239a53

/retest

Test results are available in Ibutsu. The test run IDs are:
5425ea7e-721e-4cd3-96c5-12d819af3090
89cd80d6-cfd3-4972-bffd-bd4eaa043165

/retest

Test results are available in Ibutsu. The test run IDs are:
42b3c4da-544e-4a1d-8565-0cafa3922d4e
8a2bc203-f5c7-4f7c-a5d9-887f22dabe28


Obtain & Harvest: request_ID argument case mismatched.

Related Bugs:
#97 Changed request_ID to request_id to match spec
#82 harvest/getrecords request_ID paramater capitalization doesn't match spec
#85 Harvest/GetRecord - Returns OK for invalid request IDs.
#83 fix for issue #82

Description:
The argument "request_ID" appears to still have an error. The related bugs, listed above, are all in reference to the different implementation of the argument in regards to Obtain and Harvest. One is using "request_ID" while the other is using "request_id".
In reviewing both I find that they remain mismatched after bug fixes were checked in. Examples below.
Could this be due their descriptions in the Spec? Consider this from the Spec:
API
     GET <node-service-endpoint-URL>/harvest/getrecord?request_id=<id>
     &by_doc_ID=<T|F>
     &by_resource_ID=<T|F>

Arguments (GET)
    "request_ID":		ID,
    "by_doc_ID":		bool
    "by_resource_ID":	bool

If one were to use the API description they would implement "request_id". If they were instead to  implement what is in the Argument section they would use "request_ID". This is true for Obtain as well. The value of "request_ID" is the only one that differs from the API section to the Arguments section which may be the cause of this bug continuing one way or the other.
Examples:
obtain?request_ID=1234&by_doc_ID=true
    --Returns multiple docs

obtain?request_id=1234&by_doc_ID=true
    --Returns the single doc.

harvest/getrecord?request_ID=1234&by_doc_ID=true"
    --Returns the single doc.

harvest/getrecord?request_id=1234&by_doc_ID=true"
    --Returns {"request": {"HTTP_request": "", "verb": "getrecord"}, "OK": false, "responseDate": "2011-10-19T21:16:43.467918Z", "error": "badArgument"}

This was covered in a previous status meeting and will be addressed in a future version of the spec


[DS-6499][WorkerServer] report busy state when worker is overload

this pr close #6499

Purpose of the pull request

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:

Codecov Report

Merging #6512 (dddc08b) into dev (db04a5b) will decrease coverage by 0.16%.
The diff coverage is 3.24%.

@@             Coverage Diff              @@
##                dev    #6512      +/-   ##
============================================
- Coverage     38.58%   38.41%   -0.17%
  Complexity     3205     3205
============================================
  Files           645      645
  Lines         25756    25819      +63
  Branches       2791     2792       +1
============================================
- Hits           9938     9919      -19
- Misses        14907    14991      +84
+ Partials        911      909       -2

Impacted Files
Coverage Δ

.../org/apache/dolphinscheduler/common/Constants.java
75.00% <ø> (ø)

...pache/dolphinscheduler/common/utils/HeartBeat.java
0.00% <0.00%> (ø)

...server/master/dispatch/host/CommonHostManager.java
0.00% <0.00%> (ø)

...r/master/dispatch/host/LowerWeightHostManager.java
0.00% <0.00%> (ø)

...e/dolphinscheduler/server/worker/WorkerServer.java
0.00% <ø> (ø)

.../server/worker/processor/TaskExecuteProcessor.java
0.00% <0.00%> (ø)

...r/server/worker/registry/WorkerRegistryClient.java
0.00% <0.00%> (ø)

...uler/server/worker/runner/WorkerManagerThread.java
0.00% <0.00%> (ø)

...phinscheduler/service/registry/RegistryClient.java
1.34% <0.00%> (-0.05%)
:arrow_down:

...olphinscheduler/server/registry/HeartBeatTask.java
27.58% <27.27%> (-25.36%)
:arrow_down:

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db04a5b...dddc08b. Read the comment docs.