Request body gzip compression

Hi,
Do you have a built-in mechanism for gzip compressing request body? So far I've tried the following:
def gzipCompressBytes(dataBytes: Array[Byte]): Array[Byte] = {
  val baos = new ByteArrayOutputStream
  val gzos = new GZIPOutputStream(baos)
  gzos.write(dataBytes)
  gzos.finish()
  gzos.close()
  baos.close()
  baos.toByteArray
}

val body: JValue = JArray(objs.toList)
val compressedBody = gzipCompressBytes(body.show.getBytes())

val baseRequest = basicRequest
 .header(ContentEncoding, "gzip")
 .header(ContentType, "application/json")
 .body(compressedBody)
 .header("accept", "application/json")

However this approach didn't seem to work - is there any recommendation from your end how to achieve request body compression? Thanks! 🙏

So far there's no built-in mechanism, but you are right that this is a good candidate for a feature (just as we have auto-decompression).
Until this is done, your "manual" approach looks fine. I tested it with https://postman-echo.com/post and got back correct results. Can you describe what problems do you encounter, and which backend you are using?

#2148


make -C docker release_run LOCAL_USER=1 is failing

I have cloned the tensorRT-LLM repo and have tried make -C docker release_run LOCAL_USER=1
but getting the following error:
pull access denied for tensorrt_llm/release, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
Full logs:
docker build --progress auto --build-arg BASE_IMAGE_WITH_TAG=tensorrt_llm/release:latest --build-arg USER_ID=0 --build-arg USER_NAME=root --build-arg GROUP_ID=0 --build-arg GROUP_NAME=root --file Dockerfile.user --tag tensorrt_llm/release:latest-root ..
Sending build context to Docker daemon  1.567GB
Step 1/9 : ARG BASE_IMAGE_WITH_TAG
Step 2/9 : FROM ${BASE_IMAGE_WITH_TAG} as base
pull access denied for tensorrt_llm/release, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
make: *** [Makefile:118: release_run] Error 1
make: Leaving directory '/home/jaya_kommuru/TensorRT-LLM/docker'

Hi @jayakommuru , you need to build the docker image tensorrt_llm/release:latest first, please try:
make -C docker release_build LOCAL_USER=1
make -C docker release_run LOCAL_USER=1

@QiJune Ok, now make -C docker release_build LOCAL_USER=1 is throwing the following error. Any idea?
------
 > [wheel 10/10] RUN --mount=type=cache,target=/root/.cache/pip --mount=type=cache,target=/root/.cache/ccache     python3 scripts/build_wheel.py --clean --trt_root /usr/local/tensorrt --python_bindings --benchmarks:
#33 0.474 fatal: not a git repository (or any of the parent directories): .git
#33 0.475 Traceback (most recent call last):
#33 0.475   File "/src/tensorrt_llm/scripts/build_wheel.py", line 387, in <module>
#33 0.475     main(**vars(args))
#33 0.475   File "/src/tensorrt_llm/scripts/build_wheel.py", line 84, in main
#33 0.476     build_run('git submodule update --init --recursive')
#33 0.476   File "/usr/lib/python3.10/subprocess.py", line 526, in run
#33 0.477     raise CalledProcessError(retcode, process.args,
#33 0.477 subprocess.CalledProcessError: Command 'git submodule update --init --recursive' returned non-zero exit status 128.
------
executor failed running [/bin/bash -c python3 scripts/build_wheel.py ${BUILD_WHEEL_ARGS}]: exit code: 1
make: *** [Makefile:64: release_build] Error 1

This is resolved, I had to initialise and update the submodules before building. Thanks

@QiJune Ok, now make -C docker release_build LOCAL_USER=1 is throwing the following error. Any idea?
------
 > [wheel 10/10] RUN --mount=type=cache,target=/root/.cache/pip --mount=type=cache,target=/root/.cache/ccache     python3 scripts/build_wheel.py --clean --trt_root /usr/local/tensorrt --python_bindings --benchmarks:
#33 0.474 fatal: not a git repository (or any of the parent directories): .git
#33 0.475 Traceback (most recent call last):
#33 0.475   File "/src/tensorrt_llm/scripts/build_wheel.py", line 387, in <module>
#33 0.475     main(**vars(args))
#33 0.475   File "/src/tensorrt_llm/scripts/build_wheel.py", line 84, in main
#33 0.476     build_run('git submodule update --init --recursive')
#33 0.476   File "/usr/lib/python3.10/subprocess.py", line 526, in run
#33 0.477     raise CalledProcessError(retcode, process.args,
#33 0.477 subprocess.CalledProcessError: Command 'git submodule update --init --recursive' returned non-zero exit status 128.
------
executor failed running [/bin/bash -c python3 scripts/build_wheel.py ${BUILD_WHEEL_ARGS}]: exit code: 1
make: *** [Makefile:64: release_build] Error 1

Hi, how can I solve this problem?

#33 0.477 subprocess.CalledProcessError: Command 'git submodule update --init --recursive' returned non-zero exit status 128.
The git command failed, it seems to be a network issue when running the docker script.

The submodule cmd did't work, so I clone the submodules manually.


fix(create-hops-app): rename bin to "create-hops-app" for stable version

Bors merge bot cheat sheet
We are using bors-ng to automate merging of our pull requests. The following table provides a summary of commands that are available to reviewers (members of this repository with push access) and delegates (in case of bors delegate+ or bors delegate=[list]).

Syntax
Description

bors merge
Run the test suite and push to master if it passes. Short for "reviewed: looks good."

bors merge-
Cancel an r+, r=, merge, or merge=

bors try
Run the test suite without pushing to master.

bors try-
Cancel a try

bors delegate+
Allow the pull request author to merge their changes.

bors delegate=[list]
Allow the listed users to r+ this pull request's changes.

bors retry
Run the previous command a second time.

This is a short collection of opinionated commands. For a full list of the commands read the bors reference.

bors merge


Setting a valid datacenter ID in the blob ID

This commit provides a way of assigning IDs to datacenters so that this information is correctly embedded inside the blob ID. In effect, this will provide information on which DC the blob was created in.
This information could be applied in many ways. Two immediate examples:

Cross colo GET requests - Know exactly which DC to proxy to to get the blob
which wasn't found in the local datacenter.
Track workload - how often is a blob fetched in a DC it was not created in?

Codecov Report

Merging #794 into master will decrease coverage by 0.03%.
The diff coverage is 81.35%.

@@             Coverage Diff              @@
##             master     #794      +/-   ##
============================================
- Coverage     66.39%   66.35%   -0.04%
- Complexity     4021     4026       +5
============================================
  Files           347      347
  Lines         26204    26231      +27
  Branches       3285     3287       +2
============================================
+ Hits          17398    17406       +8
- Misses         7894     7912      +18
- Partials        912      913       +1

Impacted Files
Coverage Δ
Complexity Δ

...java/com.github.ambry/config/ClusterMapConfig.java
100% <ø> (ø)
1 <0> (ø)
:arrow_down:

...ub.ambry/clustermap/HelixBootstrapUpgradeTool.java
0% <ø> (ø)
0 <0> (ø)
:arrow_down:

...thub.ambry.clustermap/CompositeClusterManager.java
69.36% <0%> (-2.6%)
18 <0> (ø)

...ry/clustermap/HelixClusterWideAggregationTool.java
0% <0%> (ø)
0 <0> (ø)
:arrow_down:

...m.github.ambry.clustermap/HelixClusterManager.java
94.68% <100%> (+0.37%)
34 <3> (+1)
:arrow_up:

....github.ambry.clustermap/StaticClusterManager.java
77.62% <100%> (+0.66%)
81 <1> (+3)
:arrow_up:

...n/java/com.github.ambry.clustermap/Datacenter.java
88.33% <100%> (+0.4%)
26 <3> (+1)
:arrow_up:

.../com.github.ambry.clustermap/HelixParticipant.java
75.75% <100%> (+0.37%)
14 <0> (ø)
:arrow_down:

...b.ambry.clustermap/StaticClusterAgentsFactory.java
63.15% <100%> (ø)
4 <0> (ø)
:arrow_down:

...ub.ambry.clustermap/HelixBootstrapUpgradeUtil.java
70.84% <90.9%> (+0.55%)
45 <10> (ø)
:arrow_down:

... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ce0496c...5b3899d. Read the comment docs.

@pnarayanan @dharju Reminder

Sorry for the spurious reminder @pnarayanan - @nsivabalan is the other reviewer.


Flow promotion and developer training program

Flow promotion and developer training program
Grant category
Please select one or more of:

[ ] Open source maintenance
[ ✓] Developer tools / services
[✓ ] Educational material

Description
Problem statement
In the Asia-Pacific Chinese developer community, Flow has a low brand awareness, low developer attention, and few developers proficient in cadence language. The number of Web3 projects developed based on Flow is small.
Currently, entrepreneurship based on Web3 is very popular in the Asian-Pacific Chinese community. A large number of teams, computer majors in colleges, and Web2 developers are exploring Web3 entrepreneurship, but there is a lack of Web3 courses suitable for Chinese developers.
Through Google Trends comparison and research on enterprises and some Web2 practitioners, we found that the most used front-end framework is vue in the Chinese community. We believe that Flow is suitable for Web3 entrepreneurship in the Asia-Pacific Chinese community. But this is not a familiar front-end framework, and it will also lead to high learning costs and reduced development efficiency.
Proposed solution

We hope to develop a cadence development course which is suitable for the Asian-Pacific Chinese community and college graduates.
Complete the construction of a cadence+vue3+elementui scaffolding, and complete an nft casting and trading platform based on the scaffolding for course case explanation.
Cadence+vue3+elementui can enrich the scaffolding case library of the cadence community and reduce the learning cost of Chinese community developers and students. Elementui is based on the vue component library, which greatly improves the development efficiency of dapp.
Promote this course to college developers and Web2 developers by our "Web3 Accelerator", "2022 China University Web3 Acceleration Camp", "China Entrepreneurs Conference".
Promote this course to Web2 developers in conjunction with the Chinese developer community (CSDN,  SegmentFault) and developer education platforms (Geek Time, Tencent Classroom, Netease Classroom), YouTube, bilibili, Baidu Video, etc.

Course Details
course1：Learn about Flow and Cadence.

Learn about Block-chain and Its Basic Vocabulary
Learn about Flow
Learn about Cadence
Install the Flow CLI
Learn about Playground
Complete the First Smart Contract Development: Hello Cadence
What Web2.0 Practitioners Need to Prepare for Learning Web3.0

course2：Cadence Basic Grammar

Values and Types
Struct & Resource
Control Flow
Function

course3: Engineering Development

Local environment build, Environment configuration and Simulator
Accounts
Transactions
Events
Interface
Capability-based Access Control
Contract Upgrade Specifications and Standards

course4：FCL

Learn about FCL
FCL Integrated
The Use of FCL Common Functions
Testnet Deployment

course5： Practical Operation 1

Teach You to Develop DAPP quickly
Introduction to the Engineering Structure
Introduction to Vue3 + Elementui + Fcl Scaffolding
Interact with the Front-end,Store and mint NFT

course6： Practical Operation 2

NFT Sales
NFT Transaction
Testnet Deployment

Qualifications
We created the "WEB 3.0 Accelerator" and launched the "2022 Chinese University Web3 Acceleration Camp". We held 9 "China Entrepreneurs Conference" and "China University Youth Entrepreneurship Conference" in the Chinese community. We cooperated with the Chinese developer community (CSDN, Sifu) and developer education platforms (Tencent Classroom, Netease Classroom, Geek Time, etc.) established a course promotion partnership. We have our own columns on Chinese video platforms (Baidu Video, blibli) and Youtube.
links:
http://9.cyz.cn/
http://8.cyz.cn/
https://ke.qq.com/course/5271218#term_id=105445645
https://space.bilibili.com/1781314512?spm_id_from=333.1007.0.0
https://www.youtube.com/channel/UC4G60PmMwvgBGRf9a73o4lw
https://haokan.baidu.com/author/1727801298887666
https://www.8btc.com/college/item/548
https://haokan.baidu.com/author/1727801298887666
https://segmentfault.com/u/metatribe_623d0e914e484
https://www.zhihu.com/people/da-peng-72-69-64/zvideos
Impact

In what ways does this benefit the broader Flow developer ecosystem?
1、Developed courses suitable for Chinese community colleges and Web2.0 developers.
2、Course promotion covers the Chinese developer community, education platform and video platform. The cumulative exposure exceeded 100,000.
3、The number of participants in the course is not less than 1,500, and 300-500 qualified Flow developers are trained.
4、Building brand influence in the Chinese developer community.

Milestones and funding

Note: You can leave the USD proposal empty / "TBD". Please consider adoption and/or maintenance milestones at the end of your project.

Milestone
Deliverables
Timeline
Risks
USD proposal

1-course1
Recording and Editing
1 weeks
-
2500

2-course2
Recording and Editing
1 weeks
-
2500

3-course3
Recording and Editing
1 weeks
-
2500

4-course4
Recording and Editing
1 weeks
-
2500

5-course5
Recording and Editing
1 weeks
-
2500

6-course6
Recording and Editing
1 weeks
-
2500

Web3 Acceleration Camp I
Recruit 500-1000 students
4 weeks
-
10000

Web3 Acceleration Camp II
Recruit 500-1000 students
4 weeks
-
10000

online promotion
Online more than 5 publicity channels, accumulated learning and watching 50000
2 weeks
-
5000

online promotion
Online more than 10 publicity channels, accumulated learning and watching 100000
2 weeks
-
5000

Team

Name
Role
Bio
Contact

Schiele
Project leader
6 years of experience in crypto community market and operation, 10 years of experience in IT market
<EMAIL_ADDRESS>

Kaka
Senior development engineer, lecturer
years of experience in development and system architecture
<EMAIL_ADDRESS>

Seven
Senior Web3 Researcher
10 years experience in course development and promotion, Senior researcher in Web3 and Crypto
<EMAIL_ADDRESS>

Moya
Senior development engineer
5 years experience in block-chain project development and system architecture
<EMAIL_ADDRESS>

Hi @dapeng1215 we reviewed your proposal in detail and decided to pass because it is unclear if the impact on the developer ecosystem will be high enough.
Thanks for taking the time to submit your proposal and feel free to reach me directly on Discord in case you have further questions: alx-flw.find#6198.

Thank you for your reply, we will provide data for review at each stage @alxflw

@dapeng1215 to be clear, we've passed on this grant proposal. you are very welcome to work on this project of course.


Update README.md

Add titles to the different way to install perfect-scrollbar and add the Rails gem link

Absolutely stellar work @zedtux, I think this will greatly increase the usage of the library.


Appium Support Error

idk why this is happening
Traceback (most recent call last):
  File "F:\Zen\App\zen_app\browserstack_test\app_test.py", line 44, in <module>
    driver = Remote(
  File "C:\Users\t-adubey\AppData\Roaming\Python\Python310\site-packages\appium\webdriver\webdriver.py", line 274, in __init__
    super().__init__(
  File "C:\Users\t-adubey\AppData\Roaming\Python\Python310\site-packages\selenium\webdriver\remote\webdriver.py", line 269, in __init__
    self.start_session(capabilities, browser_profile)
  File "C:\Users\t-adubey\AppData\Roaming\Python\Python310\site-packages\appium\webdriver\webdriver.py", line 369, in start_session
    response = self.execute(RemoteCommand.NEW_SESSION, parameters)
  File "C:\Users\t-adubey\AppData\Roaming\Python\Python310\site-packages\selenium\webdriver\remote\webdriver.py", line 425, in execute
    self.error_handler.check_response(response)
  File "C:\Users\t-adubey\AppData\Roaming\Python\Python310\site-packages\selenium\webdriver\remote\errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Appium Flutter Driver support is available on Appium version 1.16 and above. Appium version 1.16.0 and above are not supported on "0.0". To know more about the supported versions and defaults refer to - https://www.browserstack.com/app-automate/capabilities

Appium Doctor -
info AppiumDoctor Appium Doctor v.1.16.0
info AppiumDoctor ### Diagnostic for necessary dependencies starting ###
info AppiumDoctor  ✔ The Node.js binary was found at: C:\Program Files\nodejs\node.EXEinfo AppiumDoctor  ✔ Node version is 16.14.2WARN AppiumDoctor  ✖ ANDROID_HOME environment variable is NOT set!WARN AppiumDoctor  ✖ JAVA_HOME environment variable is NOT set!
WARN AppiumDoctor  ✖ adb, android, emulator could not be found because ANDROID_HOME or ANDROID_SDK_ROOT is NOT set!
WARN AppiumDoctor  ✖ Cannot check %JAVA_HOME% requirements since the environment variable itself is not set
info AppiumDoctor ### Diagnostic for necessary dependencies completed, 4 fixes needed. ###
info AppiumDoctor
info AppiumDoctor ### Diagnostic for optional dependencies starting ###
WARN AppiumDoctor  ✖ opencv4nodejs cannot be found.
WARN AppiumDoctor  ✖ ffmpeg cannot be found
WARN AppiumDoctor  ✖ mjpeg-consumer cannot be found.
WARN AppiumDoctor  ✖ bundletool.jar cannot be found
WARN AppiumDoctor  ✖ gst-launch-1.0.exe and/or gst-inspect-1.0.exe cannot be found
info AppiumDoctor ### Diagnostic for optional dependencies completed, 5 fixes possible. ###
info AppiumDoctor
info AppiumDoctor ### Manual Fixes Needed ###
info AppiumDoctor The configuration cannot be automatically fixed, please do the following first:
WARN AppiumDoctor  ➜ Make sure the environment variable ANDROID_HOME is properly configured for the Appium process. Refer https://github.com/appium/java-client/blob/master/docs/environment.md for more details.
WARN AppiumDoctor  ➜ Make sure the environment variable JAVA_HOME is properly configured for the Appium process. Refer https://github.com/appium/java-client/blob/master/docs/environment.md for more details.
WARN AppiumDoctor  ➜ Manually configure ANDROID_HOME and run appium-doctor again.
WARN AppiumDoctor  ➜ Set %JAVA_HOME% environment variable to the root folder path of your local JDK installation
info AppiumDoctor
info AppiumDoctor ### Optional Manual Fixes ###
info AppiumDoctor The configuration can install optionally. Please do the following manually:
WARN AppiumDoctor  ➜ Why opencv4nodejs is needed and how to install it: https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/image-comparison.md
WARN AppiumDoctor  ➜ ffmpeg is needed to record screen features. Please read https://www.ffmpeg.org/ to install it
WARN AppiumDoctor  ➜ mjpeg-consumer module is required to use MJPEG-over-HTTP features. Please install it with 'npm i -g mjpeg-consumer'.
WARN AppiumDoctor  ➜ bundletool.jar is used to handle Android App Bundle. Please read http://appium.io/docs/en/writing-running-appium/android/android-appbundle/ to install it. Also consider adding the ".jar" extension into your PATHEXT environment variable in order to fix the problem for Windows
WARN AppiumDoctor  ➜ gst-launch-1.0.exe and gst-inspect-1.0.exe are used to stream the screen of the device under test. Please read https://appium.io/docs/en/writing-running-appium/android/android-screen-streaming/ to install them and for more details
info AppiumDoctor
info AppiumDoctor ###
info AppiumDoctor
info AppiumDoctor Bye! Run appium-doctor again when all manual fixes have been applied!
info AppiumDoctor

Appium Version  - 1.22.3
PATH env. var. is also up to date

selenium.common.exceptions.WebDriverException: Message: Appium Flutter Driver support is available on Appium version 1.16 and above. Appium version 1.16.0 and above are not supported on "0.0". To know more about the supported versions and defaults refer to - https://www.browserstack.com/app-automate/capabilities

I don't know of the error message. Was this browserstack thing? Then, please ask the support team first.


Fix: coffescript to js conversion leftover

There is no need for the return. It looks like a leftover from a conversion from Coffee to JS.

@felippenardi thank you for contributing!


visdiff: Add more startup logging

:eyeglasses: @keybase/react-hackers

:mag_right: The commits 707333bac884...fdc922bbee49 introduced some visual changes on linux:

Changed: New_Proof_Post-Facebook (before) (after)


added GPU-accelerated image processing on the TUD HPC cluster

First draft of GPU-accelerated image processing on the TUD HPC cluster.
Now as a new branch on BiAPoL/blog
Feedback very welcome :)

@haesleinhuepf @jo-mueller thanks a lot for your excellent feedback :)
I have two remaining questions:

should we publish the blog article now or wait for the part about moving data?
after the discussion here (and also the discussion with Melissa, Jessica and Fabian) it seems to me that it would be sensible to make the singularity-devbio-napari repository public. Is that o.k./possible with the math gitlab? It will be difficult to migrate the CI workflow to github, and neither github.com nor gitlab.com will allow us to store tens of gigabytes of singularity images (at least not without paying for the storage). The most feasible alternative might be the TU Chemnitz gitlab. But I would ask them if they are o.k. with us storing tens of gigabytes of image files in their container registry first.

should we publish the blog article now or wait for the part about moving data?

I would wait until opening images from the fileserver works. Otherwise the blog post is a toothless tiger 😉

2. make the singularity-devbio-napari repository public

Yes, do it! (If not yet done)
Love to see progress here! Cool projects ahead 🤓

@haesleinhuepf I just added the section: Transfer Data to/from the HPC cluster :-)
Since I am neither PI nor contact person on our HPC project, I cannot see the links and user interface needed to do this. Could you please elaborate in Step 1?

@haesleinhuepf I just added the section: Transfer Data to/from the HPC cluster :-)
Since I am neither PI nor contact person on our HPC project, I cannot see the links and user interface needed to apply for storage on the HPC cluster. Could you please elaborate in Step 1?

Hi @thawn ,
maybe what you are looking for is this link?

You can apply for a fileserver / project space in the ZIH self-service portal. Then, you need to ask HPC support via email to mount it on the transfer nodes ...

@haesleinhuepf I got the part about how to get the fileserver space.
What I could not figure out was how to apply for the project space on the cluster (i.e. how did we get /projects/p_bioimage/? Do you also apply for that on the self-service portal, or does that come automatically with the HPC project?

No, afaik, you need to send an email to hpc-support.

What I could not figure out was how to apply for the project space on the cluster (i.e. how did we get /projects/p_bioimage/?)

I think this may be what you are looking for

@haesleinhuepf here is a small status update:
Here is a short summary from my test session with Conni:

Unfortunately, the link to join our HPC project did not work for Conni, so we had to use my account. Therefore, we could not test all steps.
With my account, Conni managed to open and save a test image from our fileserver space.
we discovered a bug in the synchronization function
I rewrote some sections that were to detailed and were missing concrete examples.

For the next steps I suggest the following:

Help Conni (and Hella) write an application for a HPC project for the CMCB LMF. That way, I will learn how this works and can add more detailed instructions to the blog article.
Work with some actual data from one of Conni's users.

@haesleinhuepf The blog posts are ready for another round of review ;-)


Librespot 20.0.130 Kodi

Hi,
I'm still using Librespot 20.0.130 on a Kodi system. Reason is that from version 20.5 and later 21.1.1. Discovery mode isn't working anymore and it stops after the album has been finished. With 20.0.130 this all works fine. However the artwork of an album has been zoomed in and isn't clearly visible. Is there a way to solve this? I only have this with this addon. And already tried to change the system resolution. But unfortunately without any luck.
Hope someone can help with this matter.

Those version numbers are meaningless here I'm afraid, we don't know anything at all about the Kodi plugin. You would need to get support from the plugin maintainer for a start. If they can't help with that (they should) then you need to return here with the version of the librespot library you're using and also a librespot debug log file.

@kingosticks first of all thanks for your reply. This is what I could find regarding the version number.
[2024-09-08T08:09:03Z INFO  librespot] librespot 0.3.1 0989b9836a (Built on 2022-04-28, Build ID: ILkusV0o, Profile: release)

0.3 is very very old and not supported. Please request the kodi plugin maintainer updates to the latest version. Technically that's v0.4 but it'll very soon be v0.5

librespot doesn't handle artwork at all, does it? This has likely nothing to do with librespot, but is a Kodi integration/implementation issue.

dev provides some logic to get the URL, I think. But indeed, this is Kodi and an old version, so closing.


Correctly handle bad characters

Probably due to line truncation:
Traceback (most recent call last):                                                                                File "python/lean_step.py", line 421, in <module>
    create_datasets(**vars(opts))                                                                                File "python/lean_step.py", line 385, in create_datasets
    for json_line in json_file_handle:
  File "/opt/conda/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 0: unexpected end of data

Becomes:
BAD FILE: data/raw_data/shard_268.json LINE: 14080: EXCEPTION: 'utf-8' codec can't decode byte 0xf0 in position 0: unexpected end of data

Merging


capture network requests

Is your feature request related to a problem? Please describe.
Playwrite allows show the network traffic (which requests from browser are done) via page.route

https://playwright.dev/python/docs/api/class-page#page-route
<EMAIL_ADDRESS>
Unfortunately there is no keyword implemented in robotframework-browser lib to get the network traffic.
Describe the solution you'd like
Implement new keywords in robotframework-browser lib like this:
page = browser.new_page()
page.route("**/*", lambda route: capturedRequests.add(route))

Describe alternatives you've considered
Add a keyword AttachRoute and delegate to Page.route in playwrite: https://playwright.dev/python/docs/api/class-page#page-route

@allcontributors please add @DirkRichter for ideas.

You can do this pretty easily with JS based extension.
The example shows a python version of this.
Also we do allow recording network traffic as har file. Could you @DirkRichter explain more how would you benefit from a Robot Framework keyword for this?
I feel there is a cost in adding keywords as it makes the lib harder to learn.

Sse New Context recordHar for the current way


Return of the WSOD

Describe the bug
User ID 21891 had a WSOD and could not log in. I deleted her states, and the WSOD went away. This was her states:
{"filters": {}, "incident": 208, "incidents": {"208": {"filters": {}, "showingMap": true, "mapViewPort": {"_northEast": {"lat": 30.420700230527785, "lng": -87.60961532592775}, "_southWest": {"lat": 30.33510204068193, "lng": -87.78676986694336}}, "showingTable": false, "appliedFilters": {}}, "210": {"filters": {}, "showingMap": true, "mapViewPort": {"_northEast": {"lat": 30.208224042915713, "lng": -93.22757184505464}, "_southWest": {"lat": 30.205544430826226, "lng": -93.23310792446138}}, "showingTable": false, "appliedFilters": {}}, "undefined": {"filters": {}, "showingMap": true, "mapViewPort": {"_northEast": {"lat": 42.42345651793833, "lng": -68.59863281250001}, "_southWest": {"lat": 24.44714958973082, "lng": -106.47949218750001}}, "showingTable": false, "appliedFilters": {}}}, "showingMap": true, "mapViewPort": {"_northEast": {"lat": 30.208224042915713, "lng": -93.22757184505464}, "_southWest": {"lat": 30.205544430826226, "lng": -93.23310792446138}}, "showingTable": false, "showingLegend": false, "appliedFilters": {}}
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

See if you are able to reproduce this when https://github.com/CrisisCleanup/crisiscleanup-3-web/commit/5a9ccdf5da632b61a399c4bf59d4348acde68e0a is pushed into live.
May have resolved this problem.

Agreed. Likely fixed. Marking closed.


[Bug Report] Stable Diffusion interactive demo is hanging on N150

Describe the bug
Stable Diffusion interactive demo : pytest models/demos/wormhole/stable_diffusion/demo/demo.py::test_interactive_demo is hanging.
I have tested with both FWs: <IP_ADDRESS> and <IP_ADDRESS>.
It best it runs up to 98%. Then crashes: see attached.
To Reproduce
Steps to reproduce the behavior:

Go to main branch
Follow the build instructions from main and build metal.
run: pytest models/demos/wormhole/stable_diffusion/demo/demo.py::test_interactive_demo
the demo hangs and I lose connection to the VM.
Expected behavior
the interactive demo is used for the web demo as well. We need this issue resolved to enable the SD web demo.

Screenshots

Please complete the following environment information:

OS: [e.g. Ubuntu 20.04]

On my latest test from main (commit: 78075c68b6f25399a4c27e5c5082bafc2a350bc1):
It did not hang but failed with an error at 12%.

                 Always | WARNING  | Circular buffer size 262144 B exceeds allocated L1 buffer bank size of 245760 B. This may allow this circular buffer to write outside the allocated buffer space.
                 Always | WARNING  | Circular buffer size 262144 B exceeds allocated L1 buffer bank size of 245760 B. This may allow this circular buffer to write outside the allocated buffer space.
                 Always | FATAL    | Statically allocated circular buffers in program 503 clash with L1 buffers on core range [(x=0,y=0) - (x=7,y=7)]. L1 buffer allocated at 941952 and static circular buffer region ends at 942848
 12%|████████████▌                                                                                | 6/51 [02:50<21:20, 28.45s/it]
malloc(): unsorted double linked list corrupted
Fatal Python error: Aborted

@mywoodstock  to help with first level triage. Removing other owners.

@dvartaniansTT @mbahnasTT Is this issue non-deterministic in where it hangs/fails?

@dvartaniansTT @mbahnasTT Is this issue non-deterministic in where it hangs/fails? Since this is a regression, we should to identify the commit that broke it and revert it.

@mywoodstock @dvartaniansTT Looking at the last 2 weeks of pipelines, it seems like the failure present about 50% of the time. I'm going to try and determine if it is machine-dependant or truly ND.

I am getting a deterministic failure right at the beginning (after the prompt). Narrowed down this regression to this PR: https://github.com/tenstorrent/tt-metal/pull/15394
The commit before this PR was merged is good: 69870bdeaf1c9270e325810249def6a3e9f38fb4
After this PR we see the failing case -- the new/updated condition was added in the above PR -- either the check is buggy, or there was an OP bug that was previously hidden? cc: @tt-aho

The pr was made of 2 commits. Do you know if it was the first or second commit? The first commit enables correct asserts/warnings but doesn't change functionality of any op/infra.
The second commit fixes 2 ops to properly size cbs. Could be something was undersized previously (concat?)?

@mywoodstock what is the deterministic failure you see? Is it the same as the one mentioned in this issue?

@tt-aho The failure is different from the one mentioned in this issue:
                     Op | DEBUG    | shard_shape: (1, 16)
                     Op | DEBUG    | CB 0 :: npages = 4, pagesize = 2048
                     Op | DEBUG    | CB 16 :: npages = 4, pagesize = 2048
                     Op | DEBUG    | CB 17 :: npages = 264, pagesize = 64
                     Op | DEBUG    | CB 1 :: npages = 1, pagesize = 64
                 Always | FATAL    | Cannot set to globally allocated buffer. Circular buffer size 1536 B exceeds allocated L1 buffer bank size of 32 B
...
E       RuntimeError: TT_ASSERT @ /localdev/asarje/tt-metal.wh/tt_metal/impl/buffers/circular_buffer_types.cpp:104: false
E       info:
E       Cannot set to globally allocated buffer. Circular buffer size 1536 B exceeds allocated L1 buffer bank size of 32 B
E       backtrace:
E        --- /localdev/asarje/tt-metal.wh/ttnn/ttnn/_ttnn.so(+0x19b080b) [0x77d0c2fb080b]
E        --- void tt::assert::tt_assert<unsigned int, unsigned int>(char const*, int, char const*, bool, char const*, fmt::v11::basic_format_string<char, fmt::v11::type_identity<unsigned int const&>::type, fmt::v11::type_identity<unsigned int const&>::type>, unsigned int const&, unsigned int const&)
E        --- tt::tt_metal::v0::CircularBufferConfig::set_globally_allocated_address(tt::tt_metal::v0::Buffer const&)
E        --- ttnn::operations::data_movement::detail::untilize_with_halo_multi_core_v2(tt::tt_metal::v0::Program&, tt::tt_metal::Tensor const&, unsigned int, unsigned int, unsigned int, tt::tt_metal::Tensor const&, tt::tt_metal::Tensor const&, tt::tt_metal::Tensor const&, bool, bool, tt::tt_metal::Tensor&)
E        --- ttnn::operations::sliding_window::halo::HaloDeviceOperation::create_program(std::__1::vector<tt::tt_metal::Tensor, std::__1::allocator<tt::tt_metal::Tensor>> const&, std::__1::vector<tt::tt_metal::Tensor, std::__1::allocator<tt::tt_metal::Tensor>>&) const

Probably its an "oversizing" issue here:
https://github.com/tenstorrent/tt-metal/blob/main/ttnn/cpp/ttnn/operations/data_movement/untilize_with_halo_v2/device/untilize_with_halo_v2_program_factory.cpp#L148
The CB size should be per core, right?

Yes, it should be size per core. Right now it only asserts in debug mode, and will log a warning in release. Will be changed to assert in release once existing tests are fixed.

@esmalTT I just tested it now from main branch and it's still failing
There are a lot of those concerning warning messages before it crashed:

Also, the test_demo is disabled by default and the message says it's due to the di/dt issue.

Even the basic demo is failing with segfault:

@mbahnasTT Thanks - I'll take a look today.

I have made a small fix in https://github.com/tenstorrent/tt-metal/pull/15746 to address the crash reported by @dvartaniansTT and @mbahnasTT.
There is a larger cleanup that needs to be done here, which I am currently working on.

Addressed in 917d168aa339f37547414b9d09ae850f16eca9bf - re-open if you are still seeing the same issue.

Reopening because something is messed up with the image quality

@mbahnasTT Let's close this issue in favour of this follow-on issue in #16057.


[FR]: support vuepress to generate website

Issue validation

[X] My issue is not a question relative to Material Mkdocs nor Mkdocs (You can open a discussion to get more help!)
[X] I checked the issue to prevent duplicate

Workflow
Obsidian plugin 💠
Is your feature related to a problem ?
No response
What solution do you want to see ?
I imagine the publishing plugin for obsidian is very similar to what you're doing now, but mkdoc doesn't look very prerfect.
So I think using vuepress as a static blog generator would be a better choice because:

theoretically support all features of mkdoc, and now except folder note and hover preview are supported
besides, obsidian chart plugin, obsidian echart plugin, even advanced slide etc. are also supported, but the syntax is different.
most importantly, there is a lot of optimization for blogs, such as seo, article list , timeline, rss and so on

You can check vuepress theme hope to get all features.
In order to adapt obsidian syntax in vuepress, I am going to write a vuepress plugin —— vuepress-plugin-obsidian. The general framework is ready and is expected to be completed in the next few weeks
My general idea is as follows:

I would appreciate it if you could do an adaptation of vuepress, depending on your idea of how to do it.
Finally, I can translate this obsidian plugin into Chinese(Currently the plugin does not have i18n, I can also add if you do not mind).
Thanks for any reply!!!
Describe the alternative you've considered
No response
OS
Windows
Anything else?
No response

Hello !
First, I will love to saw a translation of the plugin in different language, as I search how i can do it.
About vuepress... Well, JS-TS is not my primary langage. I choosen mkdocs because i know python so i can write some plugin to add some feature i wanted. Vuepress seems pretty cool, so if you want to create your template, you can ! I will update the doc to add it.
Github Publisher can add any obsidian markdown file to any repo, so the plugin is "template agnostic".
I don't think i will wrote a New template for the moment but i will love to saw a vuepress version !

If you want the same tree/hierarchy in github and obsidian, you can use the "Obsidian path" option!

If you want the same tree/hierarchy in github and obsidian, you can use the "Obsidian path" option!

Wow, that's wonderful.
At the same time, I would like to make a settings page dedicated to vuepress. Do you allow PR?

Obviously! Go ahead here : https://github.com/obsidianMkdocs/obsidian_mkdocs_publisher_docs to make a settings page dedicated !
If you need, you can create a new folder or simply a page for github publisher with updating the configuration example page!

Since I needed to add a lot of features to the plugin and I was not familiar with web programming, I pulled a partner. He suggested that I recreate a plugin that would improve development efficiency as well as greater freedom, and I will therefore support vuepress on the new plugin.
I'm sorry we couldn't work together!

oh :( Sad! I hope saw your plugin in the future!
Also, please, bump me when you publish your vuepress plugin, as I can saw if I can add compatibility with it :).

I mainly based on two points of consideration.

China's network environment is special, some people can't login to github (it doesn't affect me much), many people will build blogs on gitee
the existence of vuepress and mkdoc at the same time will bother new users. Too many settings and too many commands are problems.All my settings are deeply bound to vuepress.

I'll share the plugin with you when I'm done, Thank you and have a nice day!😆😆😆


Add volume group handling

Description
Motivation:
Some applications needs an specific volume controller configuration. For example to get a optimal performance or it is necessary to get an certificate.
Therefore admins needs a opportunity to add an volume to certain controller.
Implementation:
Decided to add a VolumeGroup class as an abstract object to manage this feature.
A volume group is a triple of vm id, volume id and group number.

group number == bus number
groups are optional, user can work like before
groups can only created via api
during the attach the group will be create
if vm have got mixed volumes with and without groups,
attach order is: root vol, vol with groups(order by disk seq), vols without groups(order by disk seq).
for import unmanaged VMs user have got three option:
1. can do like before (no groups) => import with current conf, after restart volumes
will be attached on the first controller until max and then second controller and so on.
(applied after restart)
2. add vol groups to disk offering list => add groups like user want it (applied after restart)
3. set useControllerConfiguration = true => read current controller conf of unmanaged vm
and add automatic volumegroups for each volume

SQL  for Update:
DROP TABLE IF EXISTS `cloud`.`volume_group`;
CREATE TABLE IF NOT EXISTS `cloud`.`volume_group`(
    `id` bigint unsigned PRIMARY KEY AUTO_INCREMENT COMMENT 'Primary Key',
    `vm_id` bigint unsigned NOT NULL COMMENT 'foreign key to vm table',
    `volume_id` bigint unsigned NOT NULL COMMENT 'foreign key to volume table',
    `group_number` int unsigned NOT NULL COMMENT 'group number of volume',
    FOREIGN KEY(vm_id) REFERENCES vm_instance (id),
    FOREIGN KEY(volume_id) REFERENCES volumes (id)
);

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[x] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[x] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[x] Major
[ ] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?

like the idea, is this (meant to be) vmware only?

In the future also for KVM  but for now only for vmware.

Hi @DaanHoogland, @nvazquez,
I have a question, what I have to do to add follow code
def importUnmanagedInstance(self, command, method="GET"):
         response = importUnmanagedInstanceResponse()
         response = self.connection.marvinRequest(command, response_type=response, method=method)
         return response

to
cloudstack/tools/marvin/marvin/cloudstack/cloudstackAPIClient.py

I added in my env, but it seems to be automated generated or ?

Hi @DaanHoogland, @nvazquez, I have a question, what I have to do to add follow code
cloudstack/tools/marvin/marvin/cloudstack/cloudstackAPIClient.py

I added in my env, but it seems to be automated generated or ?

yes, it should be auto generated.

Hi @DK101010 - just extending @DaanHoogland's answer. It should be generated by building marvin: mvn -P developer -pl :cloud-marvin and upgrade the packages pip install --upgrade tools/marvin/dist/Marvin-*.tar.gz

Ahh, ok a case of self confusion :D. Thanks you two

Soo, I guess I'm finished and ready for review :)

Soo, I guess I'm finished and ready for review :)

you mean you are taking it out of draft @DK101010 ?

Soo, I guess I'm finished and ready for review :)

you mean you are taking it out of draft @DK101010 ?

Japp, correct.

Soo, I guess I'm finished and ready for review :)

you mean you are taking it out of draft @DK101010 ?

Japp, correct.

:grin: than do so ;)

Soo, I guess I'm finished and ready for review :)

you mean you are taking it out of draft @DK101010 ?

Japp, correct.

😁 than do so ;)

Oh, how I can do it? :D I can only editing the description and the headline.

Soo, I guess I'm finished and ready for review :)

you mean you are taking it out of draft @DK101010 ?

Japp, correct.

grin than do so ;)

Oh, how I can do it? :D I can only editing the description and the headline.

You don't see the Ready for review button below here?

also @DK101010 , if you go to files you should be able to commit multiple suggestions in one commit.

also @DK101010 , if you go to files you should be able to commit multiple suggestions in one commit.

Thanks for the hint, I will try it next time. Or I could squash the commits if there are too many.

@blueorangutan package

@blueorangutan package

@nvazquez a Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 3186

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Hi @DK101010 there are some conflicts on this PR, can you please fix?

Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 3217

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6012 (SL-JID-1454)

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6012 (SL-JID-1457)

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 4230

Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 4234

@DK101010 you have conflicts, please have a look

@DK101010 you have conflicts, please have a look

@DaanHoogland Merge was tricky but it should be fine. I could not test it in my own environment because I have problems with cs upgrade. Maybe it will be a good idea to wait with the merge until tested it in my environment.

I converted to draft @DK101010 , please let me know when ready.

btw @DK101010 , yet another conflict. Hope you keep up the spirit ;)

btw @DK101010 , yet another conflict. Hope you keep up the spirit ;)

I hope so too ;). Perhaps you could help me. Currently is the main branch makes problem regards initialization of system vm's.

agent.zip, cloud-scripts.tgz, patch-sysvms.sh in /usr/share/cloudstack-common/vms are not there but Cloudstack need it.
id_rsa.cloud is created but Cloustack want id_rsa without postfix .cloud
patched.sh and /opt/cloud/bin/keystore*  is  not there in newest systemvm template (4.17) but is needed

I could solve all problems manually except for the keystore problem. I have no idea what these keystore files are or where they come from. Do you have any idea how I can solve the problem?

@DK101010 how are you building?
mvn clean install?
none of these problems seem normal for a regular build. do you use -Ddeveloper, -D noredist, -Dsimulator -Pdeveloper, etc.
Do you use a self build systemvm or do you download it during install?

@DK101010 how are you building? mvn clean install? none of these problems seem normal for a regular build. do you use -Ddeveloper, -D noredist, -Dsimulator -Pdeveloper, etc. Do you use a self build systemvm or do you download it during install?

@DaanHoogland Japp, mvn clean install -Ddeveloper, -D noredist,  -Pdeveloper. I have a real environment(vmware) behind, therefore the simulator is not need it.
I can create zones, pods, networks etc. and can attach/detach primary and secondary storages, API works also fine. Only the system vms drive me crazy.
I guess I need a 4.18 system template :/

@DK101010 how are you building? mvn clean install? none of these problems seem normal for a regular build. do you use -Ddeveloper, -D noredist, -Dsimulator -Pdeveloper, etc. Do you use a self build systemvm or do you download it during install?

@DaanHoogland Japp, mvn clean install -Ddeveloper, -D noredist, -Pdeveloper. I have a real environment(vmware) behind, therefore the simulator is not need it.
I can create zones, pods, networks etc. and can attach/detach primary and secondary storages, API works also fine. Only the system vms drive me crazy. I use the official templates from cs site.
I guess I need a 4.18 system template :/

I think you don´t need a new systemvm template. It looks like you don´t have the systemvm.iso. You should build it locally with -Pdeveloper,systemvm . I don´t understand you keystore issue in relation to whatever else you are saying. it should be inserted in the systemvm.iso on runtime.

Hi @DaanHoogland @DK101010 actually the systemvm.iso is deprecated but afaik the systemVM template should be downloaded and included when the systemvm profile is activated, as Daan says. Can you try mvn clean install -Dnoredist -P developer,systemvm?

Hi @DaanHoogland @DK101010 actually the systemvm.iso is deprecated but afaik the systemVM template should be downloaded and included when the systemvm profile is activated, as Daan says. Can you try mvn clean install -Dnoredist -P developer,systemvm?

@DaanHoogland @nvazquez sure, I can try it. But how to use this iso? Normally, I adapted the metadata.ini file, create zone and storages etc. and enable the zone. After this  CS use metadata file to download the the os template and create the system vms automatically.

@DK101010 the systemvm iso has been deprecated so no need to use it anymore. I think the approach you are taking is fine, only one comment is that CloudStack would examine the system VM template checksum and compare it to the ones on the .ini file and in case they do not match it will attempt to download a new template

@DK101010 the systemvm iso has been deprecated so no need to use it anymore. I think the approach you are taking is fine, only one comment is that CloudStack would examine the system VM template checksum and compare it to the ones on the .ini file and in case they do not match it will attempt to download a new template

@nvazquez @DaanHoogland
It is all fine, after the update failed I removed my whole environment and build a clean cs instance to prevent such things.
I checked the agent.zip and cloud-scripts.tgz to find the needed files but no luck. I'm still the opinion that a new template is needed or the agent initializing is wrong, but in that case we have bad source code in the main branch and I don't believe that.
I would feel better if I could test it again in my own environment but this PR have extensive test to get sure that this feature works. so theoretically it could be merged or we move the merge to 4.19 :-/

I would feel better if I could test it again in my own environment but this PR have extensive test to get sure that this feature works. so theoretically it could be merged or we move the merge to 4.19 :-/

@DK101010 I can not believe there is a big issue in your PR. Maybe it is in the build procedure you use.  It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4467

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian Build Failed (tid-5138)

Trillian Build Failed (tid-5146)

@DK101010 you have conflicts.
Can you also have a look at these code smells

SonarCloud Quality Gate failed.
  0 Bugs   0 Vulnerabilities   0 Security Hotspots   16 Code Smells
 4.4% Coverage  0.0% Duplication

@DK101010 there are still some comments like
https://github.com/apache/cloudstack/pull/6012#discussion_r999075541 and
https://github.com/apache/cloudstack/pull/6012/files#r849150798 (and similar)
I think these are easy refactorrings for a lot clearer code
the two sonar issues where coplexity is 16 instead of allowed 15 are both due to an extra condition that could be extracted. It would be cleaner to extract those. 72 or 41 I can not blame on you of course so feel free to extract anything there or not.

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4502

@DK101010 there are still some comments like #6012 (comment) and https://github.com/apache/cloudstack/pull/6012/files#r849150798 (and similar) I think these are easy refactorrings for a lot clearer code
the two sonar issues where coplexity is 16 instead of allowed 15 are both due to an extra condition that could be extracted. It would be cleaner to extract those. 72 or 41 I can not blame on you of course so feel free to extract anything there or not.

@DaanHoogland After the merge and not being able to test it in my environment, I'm worried that the feature won't work anymore, so I'd rather not to change the methods.

@DaanHoogland After the merge and not being able to test it in my environment, I'm worried that the feature won't work anymore, so I'd rather not to change the methods.

ok, if you still have problems with your env better focus on that first. This is still in draft ;)

I have no more ideas except to wait until 4.18 comes out.
I have built up my environment again with 4.16. That worked without any problems.

I have no more ideas except to wait until 4.18 comes out.
I have built up my environment again with 4.16. That worked without any problems.

Did you report all the issues you are facing?

I have no more ideas except to wait until 4.18 comes out.
I have built up my environment again with 4.16. That worked without any problems.

Did you report all the issues you are facing?

ohm nope, make it sense to report it when it occur in  the main? And when yes where I should report it?

If there are obvious errors in any branch report them in the issues. We can then discuss and fix or help if it is environmental.

https://github.com/apache/cloudstack/issues/6837

so it was only on main, not on 4.17?

so it was only on main, not on 4.17?

I have only main tested with main and my last stable base dev branch was 4.16

@blueorangutan package

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@GutoVeronezi @weizhouapache can you review?

Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 4873

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4985

@blueorangutan test

@DaanHoogland a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

Trillian Build Failed (tid-5531)

Trillian Build Failed (tid-5555)

@DK101010 the smoke test runs fail to start the system VMs due to
2022-12-19 09:37:50,085 WARN  [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-2:ctx-48cb44d7 job-4/job-30 ctx-f8c501d2) (logid:d647b5b8) Unable to orchestrate start VM instance {"id":2,"instanceName":"s-2-VM","type":"SecondaryStorageVm","uuid":"47535f32-97f4-4ad0-9393-64154953ac78"} due to [DB Exception on: com.mysql.cj.jdbc.ClientPreparedStatement: SELECT volume_group.id, volume_group.vm_id, volume_group.volume_id, volume_group.group_number FROM volume_group WHERE volume_group.vm_id = 2  AND volume_group.volume_id = 2  ORDER BY RAND() LIMIT 1].
com.cloud.utils.exception.CloudRuntimeException: DB Exception on: com.mysql.cj.jdbc.ClientPreparedStatement: SELECT volume_group.id, volume_group.vm_id, volume_group.volume_id, volume_group.group_number FROM volume_group WHERE volume_group.vm_id = 2  AND volume_group.volume_id = 2  ORDER BY RAND() LIMIT 1
        at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:424)
        at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:360)
        at com.cloud.utils.db.GenericDaoBase.findOneIncludingRemovedBy(GenericDaoBase.java:897)
        at com.cloud.utils.db.GenericDaoBase.findOneBy(GenericDaoBase.java:906)
        at com.cloud.storage.dao.VolumeGroupDaoImpl.findByVmAndVolume(VolumeGroupDaoImpl.java:67)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
        at com.sun.proxy.$Proxy230.findByVmAndVolume(Unknown Source)
        at com.cloud.hypervisor.HypervisorGuruBase.lambda$addGroupsToDisks$0(HypervisorGuruBase.java:313)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
        at com.cloud.hypervisor.HypervisorGuruBase.addGroupsToDisks(HypervisorGuruBase.java:312)
        at com.cloud.hypervisor.HypervisorGuruBase.toVirtualMachineTO(HypervisorGuruBase.java:256)
        at com.cloud.hypervisor.KVMGuru.implement(KVMGuru.java:116)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1205)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5333)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
        at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5457)
        at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:620)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:568)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.sql.SQLSyntaxErrorException: Table 'cloud.volume_group' doesn't exist
        at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
        at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
        at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
        ... 48 more

can you have a look?

@DK101010 are you actively pursuing this? I will move it to the next milestone if not.

@DK101010 are you actively pursuing this? I will move it to the next milestone if not.

@DaanHoogland please move it, I'm currently working on other projects.

@DK101010 can you please address the merge conflicts? Should we consider this for <IP_ADDRESS>?

@shwstppr I'm sorry, but I don't have time for that at the moment. I would rather see the 4.20

@DK101010 thanks for the update. Moving it to <IP_ADDRESS> milestone


Organization Page UI/UX

📝 Description
The card size should remain constant regardless of the content inside it. Additionally, the hover color of the button needs to be updated to a different shade for better visual feedback.
📷 Screenshot or Visual Reference (optional)

🐛 Bug Report (if applicable)

Steps to Reproduce:

Add content to the card of varying lengths.
Hover over the button to observe the color.
Observe how the card size or button color behaves.

Expected behavior: The card size remains fixed irrespective of the content length.
Actual behavior: The card size fluctuates based on the content length.

💡 Enhancement / Feature Request (if applicable)
It would be beneficial to ensure that the card maintains a uniform size for a consistent layout. Also, changing the hover color of the button will improve user interaction feedback and overall UX.

This is a great suggestion, and ensuring the card size remains fixed will definitely improve the layout consistency. Adjusting the hover color for better visual feedback is also a smart enhancement for user interaction. Go ahead with the changes, and feel free to keep contributing—your efforts are valuable to the project!
Keep up the great work! 🚀

?

Yes, This is looks great
Go ahead with the changes, and feel free to keep contributing

retained the animation and also modified the layout for more professional look

does this work?

Yess this will Work
In your given image, it shows that you have changed the layout and added animation, but in your PR, it’s not reflecting those changes. Could you please check and ensure that the layout and animation modifications are included in the PR as well?

yup ill raise the pr again


[docs] Improve English and fix 404 link

This PR hopefully closes #1814 partially.

I think we should mention not only ‘minTime’ and minDateTime  but also max analogues.

Answering:

No, we do not need to link API for each component, because they are mostly identical. In v3 KeyboardDatePicker and DatePicker had different props
Totally right, shame on me 👎
It is because an idea to prerender time on the server is wrong. Because of timezones conflict, it can be useful for some edge-cases like a calendar. But I think just nobody using SSR with picker content (like calendar or clock)

But I think just nobody using SSR with picker content (like calendar or clock)

Yeah, I guess it's a much smaller segment of the market 😆.


[v12] Pull fixes from upstream

Changes:

fix: Incorrect currency precision
fix(prettyDate): Convert datetime to user's timezone first
fix(pdf): Use non-pretty version of HTML for pdf generation
fix: Total Row in Reports

Pull Request Test Coverage Report for Build 1028

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-49.4%) to 0.0%

Totals

Change from base Build 1025:
-49.4%

Covered Lines:
0

Relevant Lines:
0

💛  - Coveralls


[20.09] routinator: 0.7.1 -> 0.8.0

Motivation for this change
Backport of #99999
Upgrade to v0.8.0.
Fixes CVE-2020-17366.
(cherry picked from commit b52ed6a1fe3dc5efaf76a03cb6a88412b6556742)
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Cherry-picking from your local master branch only works as long as the commits go in fast-forward, that's why we usually wait with creating the stable port until it has hit upstream master.
Result of nixpkgs-review pr 101066 1

1 package built:

routinator


Use TRAVIS_REPO_SLUG instead of hard-coded path.

Changes to the core

[x] Followed hacking.md.

Thanks!


TypeScript branch

I forked the Aurelia skeleton-navigation, created a new master-typescript branch from it, then merged in the TypeScript version of the app from jdanyow/aurelia-typescript-skeleton and brought it up to date with the latest in the master.
You can find the changes here: https://github.com/ctoran/skeleton-navigation/tree/master-typescript
I wanted to make a pull request but I don't want it to go to the master branch. My request is, while the build process for ES6 and TypeScript are not unified, let's at least keep them in parallel branches in the same repository. This makes it much easier to compare branches and keep them in sync.
I also suggest we do the same for other samples like app-contacts.
I've already signed your CLA, if you create a master-typescript branch, I will make a pull request to that.
Thanks.

I also added all typings, so with the TypeScript compiler option of noImplicitAny: true it compiles with no errors. Right now this is the cleanest port of skeleton-navigation to TS out there. Please let me know if you are interested in this stuff.

We are going to deprecate this skeleton for v1 in place of a new, easier way to start projects. I wouldn't spend a lot of time on this. The new getting started infrastructure will let you select what options you want, including TypeScript.

The new getting started infrastructure will let you select what options you want, including TypeScript.
I like that. Thanks.

Wonderful!
This sounds like a good way to go. The modularity I see in Aurelia is one of the factors why I keep an eye on it.

@EisenbergEffect Can you give some details about this 'getting started infrastructure'? i want to start with the beta1 skeleton but typescript is not in there... how do i start out with the combi skeleton+typescript?

@vip32 the mentioned getting started packages are found here http://aurelia.io/docs.html#/aurelia/framework/1.0.0-beta.1.0.2/doc/article/getting-started

yes, those I have found. that is great starting material. but was more looking for the whole TS skeleton including the gulp tasks for TS

found this: https://github.com/eriklieben/skeleton-navigation-typescript
seems like it is a port of the skeleton-navigation to typescript with also the gulp stuff updated.


Think about reducing the complexity of the advance_players() system.

If the complexity of advance_players' function signature grows any further, it could very well become unmaintainable or bevy may eventually give up on supporting queries with types as complex as what we have. Due to the fact that ALL player actions (that each need their specific data from the queries) start within the same advance_players() system, the system has to fetch input data for all possible actions. If we assume that we want to potentially have dozens of types of player actions, we can easily see how after implementing 3-4 more actions, the complexity of the advance_players() system's function signature may become unmaintainable. Because of this, it's better to think about how to refactor this system so that we don't stick all of the possible datatypes into a single system right now, rather than when it does become unmaintainable.
The purpose of this refactor:

To declutter advance_players() system's function signature;
To do this before bevy refuses to recognize the function as a system because of how complex its queries are.

Unfortunately, this has now started blocking #7 from implementation due to the current advance_players() implementation conflicting with Rust's borrowing rules.
The task becomes:

To rewrite existing player action handling into an event-based system where the advance_players() sends out specific player action events on FixedUpdate (let's say once a second), and then specific event handling action executors work on Update schedule (that is, every frame in parallel) to handle the pending actions from the advance_players() system.


[ACVP][ERROR]: Refreshed + retried, HTTP transport fails. curl rc=400

Hi,
I have successfully ran some offline tests for few days.
But today, I keep getting this error. [ACVP][ERROR]: Refreshed + retried, HTTP transport fails. curl rc=400
Could you please guide me how to solve this?
Thank you
Jenna
[ACVP]: Sending responses for vector set 2675493
[ACVP][WARNING]: JWT authorization has timed out, curl rc=401. Refreshing session...
[ACVP]: Logging in...
[ACVP]: POST Login...
Status: 200
Url: https://demo.acvts.nist.gov:443/acvp/v1/login
Resp: Recieved
[ACVP]: Login successful
[ACVP]: Refresh successful, attempting to continue...
HTTP PUT:
[{"acvVersion":"1.0"},{"vsId":2675493,"algorithm":"KDF","testGroups":[{"tgId":1,"tests":[{"tcId":1,"keyOut":"951B460544FEC2B7569DF8DDDAC94F7D","fixedData":"000000000000000000000000000000"},{"tcId":2,"keyOut":"3705B645EDFF317C4CBD502A09D918E3","fixedData":"000000000000000000000000000000"}]},{"tgId":2,"tests":[{"tcId":3,"keyOut":"6030F8C6DA2E5CFA1D8282C81FCADD764EA1FE99C19F5FC5A220F41CA596DCE1","fixedData":"000000000000000000000000000000"},{"tcId":4,"keyOut":"7421E5833DA36AEE56AB53A1673264E48B581892DA2637F19054D7D89EF7B144","fixedData":"000000000000000000000000000000"}]},{"tgId":3,"tests":[{"tcId":5,"keyOut":"038A45A63213EACFD43B899BCABB099A","fixedData":"000000000000000000000000000000"},{"tcId":6,"keyOut":"BA5F2A0C02EEE57DD209BB8E57046AC7","fixedData":"000000000000000000000000000000"}]},{"tgId":4,"tests":[{"tcId":7,"keyOut":"13C67989065685833945A337B253D91A3741797670C8358441A5A320A1C02583","fixedData":"000000000000000000000000000000"},{"tcId":8,"keyOut":"162265618503F90A0F4CEE23E6E7F9A2B6574939DF863690CD1C06AFED1E6771","fixedData":"000000000000000000000000000000"}]}]}]
[ACVP][ERROR]: Refreshed + retried, HTTP transport fails. curl rc=400
[ACVP]: POST Response Submission...
Status: 400
Url: https://demo.acvts.nist.gov:443/acvp/v1/testSessions/560521/vectorSets/2675493/results
Resp:
[
{
"acvVersion": "1.0"
},
{
"error": "Validation error(s) on JSON payload.",
"context": [
"Vector set not in \u0027failed\u0027 or \u0060error\u0060 status.",
"Unable to submit results when vector set in \u0027passed\u0027 status."
]
}
]
[ACVP][ERROR]: 400 error received from server. Message:
[ACVP][ERROR]: [
{
"acvVersion": "1.0"
},
{
"error": "Validation error(s) on JSON payload.",
"context": [
"Vector set not in \u0027failed\u0027 or \u0060error\u0060 status.",
"Unable to submit results when vector set in \u0027passed\u0027 status."
]
}
]
[ACVP][ERROR]: Failed to submit test results for vector set - skipping...
[ACVP]: Tests complete, checking results...
[ACVP]: GET Vector Set Result...
Status: 200
Url: https://demo.acvts.nist.gov:443/acvp/v1/testSessions/560521/results
Resp:
[
{
"acvVersion": "1.0"
},
{
"passed": true,
"results": [
{
"vectorSetUrl": "/acvp/v1/testSessions/560521/vectorSets/2675493",
"status": "passed"
}
]
}
]
[ACVP]: Passed all vectors in test session!

Hi,
The server error is indicating that you cannot re-submit results for test sessions that have already passed. The log messaging is a bit misleading there and I will make a note to clean that up.
Thanks,
Andrew

Hi, Andrew
Thank you for the help


fix(README): Adding link to video demo

:tada: This PR is included in version 5.0.2 :tada:
Your semantic-release bot :package::rocket:


Custom headers

Tackles issue #242.
Adds the following command line parameters:

--content-disposition
--content-language
--cache-control

Tested by hand. If you want a unit test, I'm happy to write one but given this relies upon azblob and azfile functionality and is 99.9% new variables and setting them, I don't see the point.

We should add unit tests for the ste module. But that can be done later.


Can you provide an example using spring boot?

Describe the bug
Please provide an example using spring boot.
I tried to apply this plugin but when I run gradle buildNativeImage, the task never finish.
To Reproduce
Steps to reproduce the behavior:

Create a spring boot project with gradle
apply plugin

apply plugin: 'com.github.ayltai.spring-graalvm-native-plugin'
nativeImage {
    mainClassName = 'ec.com.smx.base.BaseSpringBootApplication'

    traceClassInitialization    = true
    traceClassInitialization    = false
    removeSaturatedTypeFlows    = false
    reportExceptionStackTraces  = true
    printAnalysisCallTree       = false
    enabledAllSecurityServices  = false
    staticallyLinked            = false
    warnMissingSelectorHints    = false
    removeUnusedAutoConfig      = true
    verbose                     = false
    springNativeVerbose         = false
    removeYamlSupport           = true
    removeXmlSupport            = true
    removeSpelSupport           = true
    removeJmxSupport            = true
    maxHeapSize                 = '6G'
    verify                      = false
    //springNativeMode            = 'feature'  //agent|feature|functional
    //dumpConfig                  = '/tmp/dump.txt' // dumps the configuration to the specified file.
    //initializeAtBuildTime       = //Unsupported options
}

add dependencies
compile 'org.springframework.experimental:spring-graalvm-native:0.7.1'
annotationProcessor 'org.springframework.experimental:spring-graalvm-native:0.7.1'
compileOnly 'org.springframework:spring-context-indexer'
annotationProcessor 'org.springframework:spring-context-indexer'
Run task 'gradle buildNativeImage'

Expected behavior
Task completed
Screenshots

Desktop (please complete the following information):

OS:  macOS
Version: 10.15.5

Thanks for the feedback. Yes, I'm going to create a sample project and add more instructions on how to use this based on the sample project.
I'm not sure what caused the issue, and it's probably not related, but we don't need to add any extra dependencies (spring-graalvm-native or spring-context-indexer). This plugin handles it for you.
What are the computer configurations used for this build? CPU and RAM?

Please refer to this sample project:
https://github.com/ayltai/spring-graalvm-native-plugin/blob/master/samples/springmvc-tomcat/build.gradle


[Rebuild] Jupyter Notebook not getting a connection to the kernel

Hi, I really like cling and use it on the command line already. A really great feature would be some sort of autocompletion but it is already great without it. I now tried to run it as a Jupyter notebook but I always get this on startup:

[I 16:44:06.111 LabApp] KernelRestarter: restarting kernel (1/5), new random ports
[I 16:44:09.120 LabApp] KernelRestarter: restarting kernel (2/5), new random ports
[I 16:44:12.131 LabApp] KernelRestarter: restarting kernel (3/5), new random ports
[I 16:44:15.141 LabApp] KernelRestarter: restarting kernel (4/5), new random ports
[W 16:44:18.150 LabApp] KernelRestarter: restart failed
[W 16:44:18.150 LabApp] Kernel 98865579-3a64-46bf-9e78-56d50ab0c640 died, removing from map.

I am on Manjaro Linux

Did you figure this out? I have the same issue.

Same problem here on Arch Linux using a miniconda fresh environment.

No, I never got it working

I just encountered the same problem on Fedora -- for me, xeus-cling segfaults immediately.
Could you try to run xeus-cling from the command line?
We hope that with the conda-forge rebuilds the situation will stabilize. For the moment I'd recommend running xeus-cling on a Ubuntu 18.04 installation.

You are right. cling works from command line but xeus-cling segfaults

Same problem. Arch Linux, fresh new installed miniconda3, and xeus-cling segfaults.

xeus-cling built from source on Arch Linux also segfaults

(we use xeus-cling extensively on binder, OS X and various linux environments, mostlty built from source, without an issue)

I tried rebuilding in a clean environment with xtl 0.5.1, xeus 0.17.0, and xeus-cling built from master. Now when I run xeus-cling, I get
terminate called after throwing an instance of 'nlohmann::detail::parse_error'
  what():  [json.exception.parse_error.101] parse error at 1: syntax error - unexpected end of input; expected '[', '{', or a literal
[1]    7106 abort (core dumped)  xeus-cling --verbose --debug```

Ah so that is the expected behavior ! ;)
In fact, one should not launch xeus-cling manually. xeus-cling needs a connection file argument given by the jupyter notebook, which is the one launching the kernel.

If you want to hop on gitter.im/QuantStack/Lobby I am happy to answer questions in the chat.

@threecgreen we have just release 0.4.10, which now can be started without a connection file argument and display connection parameter upon start.
Starting xeus-cling kernel...

If you want to connect to this kernel from an other client, just copy and paste the following content inside of a `kernel.json` file. And then run for example:

# jupyter console --existing kernel.json

kernel.json

{
    "transport": "tcp",
    "ip": "<IP_ADDRESS>",
    "control_port": 61734,
    "shell_port": 50139,
    "stdin_port": 62536,
    "iopub_port": 65325,
    "hb_port": 54533,
    "signature_scheme": "hmac-sha256",
    "key": "94b779252b98477189ddc9507be4909e"
}

I updated xeus-cling via conda and got essentially the same output as @jackprob :
[D 09:00:22.319 NotebookApp] Requesting kernel info from 85fbd31c-2a7f-44de-a235-c1e11f09cc6a
[D 09:00:22.319 NotebookApp] Connecting to: tcp://<IP_ADDRESS>:46687
[I 09:00:24.558 NotebookApp] KernelRestarter: restarting kernel (1/5), new random ports
[D 09:00:24.559 NotebookApp] Starting kernel: ['/home/cgreen/.conda/envs/cling/bin/xeus-cling', '-f', '/run/user/10499/jupyter/kernel-85fbd31c-2a7f-44de-a235-c1e11f09cc6a.json', '-std=c++14']
[D 09:00:24.565 NotebookApp] Connecting to: tcp://<IP_ADDRESS>:36539
[I 09:00:27.590 NotebookApp] KernelRestarter: restarting kernel (2/5), new random ports
...

I also rebuilt the latest version of master and got the following output when trying to create a new notebook with the xeus-cling-cpp14 kernel.
[D 09:08:55.220 NotebookApp] Requesting kernel info from 105d71b1-3827-4c65-8506-c648bb413c1b
[D 09:08:55.220 NotebookApp] Connecting to: tcp://<IP_ADDRESS>:41085
[I 09:08:58.102 NotebookApp] KernelRestarter: restarting kernel (1/5), new random ports
[D 09:08:58.104 NotebookApp] Starting kernel: ['/home/cgreen/.conda/envs/cling-source/bin/xeus-cling', '-f', '/run/user/10499/jupyter/kernel-105d71b1-3827-4c65-8506-c648bb413c1b.json', '-std=c++14']
/home/cgreen/.conda/envs/cling-source/bin/xeus-cling: /home/cgreen/.conda/envs/cling-source/lib/libuuid.so.1: no version information available (required by /home/cgreen/.conda/envs/cling-source/lib/libxeus.so.1)
[D 09:08:58.108 NotebookApp] Connecting to: tcp://<IP_ADDRESS>:44991
[I 09:09:01.112 NotebookApp] KernelRestarter: restarting kernel (2/5), new random ports
...

With version built from source I get:
Adding library paths from 'LD_LIBRARY_PATH':
cling version 0.5
Looking for C++ headers with:
  LC_ALL=C g++  -O3 -DNDEBUG -xc++ -E -v /dev/null 2>&1 >/dev/null | awk '/^#include </,/^End of search/{if (!/^#include </ && !/^End of search/){ print }}' | GREP_OPTIONS= grep -E "(c|g)\+\+"
Found:
  /home/cgreen/.conda/envs/cling-source/bin/../lib/gcc/../../gcc/include/c++
  /home/cgreen/.conda/envs/cling-source/bin/../lib/gcc/../../gcc/include/c++/x86_64-pc-linux-gnu
  /home/cgreen/.conda/envs/cling-source/bin/../lib/gcc/../../gcc/include/c++/backward
clang version 5.0.0 (tags/RELEASE_500/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir:
Found candidate GCC installation: /../lib/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /../lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /usr/lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Selected GCC installation: /../lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /home/cgreen/.conda/envs/cling-source/bin/../lib/gcc/../../gcc/include/c++
 /home/cgreen/.conda/envs/cling-source/bin/../lib/gcc/../../gcc/include/c++/x86_64-pc-linux-gnu
 /home/cgreen/.conda/envs/cling-source/bin/../lib/gcc/../../gcc/include/c++/backward
 /usr/local/include
 /home/cgreen/.conda/envs/cling/lib/clang/5.0.0/include
 /usr/include
End of search list.
Adding runtime include paths:
  ""
Added include paths:
Added include paths:
  /home/cgreen/.conda/envs/cling-source/include
Setting up system headers with clang:
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /home/cgreen/.conda/envs/cling-source/bin/../lib/gcc/../../gcc/include/c++
 /home/cgreen/.conda/envs/cling-source/bin/../lib/gcc/../../gcc/include/c++/x86_64-pc-linux-gnu
 /home/cgreen/.conda/envs/cling-source/bin/../lib/gcc/../../gcc/include/c++/backward
 /home/cgreen/.conda/envs/cling-source/include
 /usr/local/include
 /home/cgreen/.conda/envs/cling/lib/clang/5.0.0/include
 /usr/include
End of search list.
#include "cling/Interpreter/RuntimeUniverse.h"
namespace cling { class Interpreter; namespace runtime { Interpreter* gCling=(Interpreter*)0x39831e8;}}
extern "C" int __cxa_atexit(void (*f)(void*), void*, void*)  noexcept;
#define __dso_handle ((void*)0x39831e8)
extern "C" int atexit(void(*f)())  throw ()  { return __cxa_atexit((void(*)(void*))f, 0, __dso_handle); }
extern "C++" int at_quick_exit(void(*f)())  throw ()  { return __cxa_atexit((void(*)(void*))f, 0, __dso_handle); }
[1]    7719 segmentation fault (core dumped)  xeus-cling -

@jackprob @stringhamc @threecgreen
We have been working on a rebuild of the xeus stack as part of the "conda-forge rebuild" endeavor which has been going on in the past few month. This should resolve the portability issue of our gcc7 package.
Basically, all packages of conda-forge are being migrated to the latest conda-build infrastructure that uses more modern compilers, and will allow for a continuous release process. The "progress bar" for the rebuild is available on https://conda-forge.org/status/:

You should already be able to try out the new build for xeus-cling on the conda-forge/label/gcc7 channel.
From a clean environment:
conda install xeus-cling notebook -c conda-forge/label/gcc7

When the rebuild is complete, this will be come the official conda-forge channel. Let me know how this goes!

The gcc7 build of xeus-cling works for me. Thanks for your help @SylvainCorlay

Thanks! Closing!

problem jupyter notebook not getting a connection after installing xeus-cling. Runnning on mac os catalina. Any ideas on how to fix this?


Add new dependency update workflows

Summary

Part of #529

[X] This workflow still needs the Bucket upload step added for compiled dependencies - may need some help from @ryanmoran to get this step sorted.

This PR won't be merged in until all of our buildpacks are ready to move off of the dep-server. At which point, the pre-existing dependency update workflow can be removed.
New plan: once the dependency upload step is added (see TODO in code), we can merge this PR and just add this workflow to the .syncignore in every repository until we are ready to use it.
Workflow Steps:

Runs on workflow dispatch or every 12 hours
Retrieve: Runs retrieve which is buildpack-provided code for picking up new versions within buildpack.toml constraints and generating a metadata.json file the with metadata for each new version to be added.
Get Compile and Test: Job to determine if there is user-provided compilation and testing code available.
Compile and/or Test Dependency: Using the output of Get Compile and Test, if either step has related buildpack code, then the job will run. Using a matrix strategy to loop over each metadata.json entry, compilation is run using buildpack-provided compilation action, and the dependency is tested using the buildpack-provided test code. Both compilation and testing can occur without the other step happening.
Upload Compiled Dependency to S3: If compiled, upload the compiled dependency to a Paketo S3 bucket. The dependency will be available at https://artifacts.paketo.io/<dependency-name>/<artifact>
Update Metadata: If compiled, this job updates the metadata.json entry for each new dependency with the compiled dependency SHA256 and the bucket URI where it was uploaded.
Update buildpack.toml: The buildpack.toml is updated by jam update-dependencies and passes in the final metadata.json as a source of metadata for the dependencies to add. A PR will be opened if dependencies are updated.

Use Cases

Checklist

[x] I have viewed, signed, and submitted the Contributor License Agreement.
[x] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
[x] I have added an integration test, if necessary.
[x] I have reviewed the styleguide for guidance on my code quality.
[x] I'm happy with the commit history on this PR (I have rebased/squashed as needed).

Can @paketo-buildpacks/dependencies-maintainers be marked as a CODEOWNER of this new workflow file so that they can iterate it independently?

Once all linked PRs are merged, we can undraft this

Ready for review @paketo-buildpacks/tooling-maintainers

With the changes to the checksum field some of this is going to need to change because its hardcoded to use SHA256 😭

LGTM. It's hard for me to really get a sense of if there's things that won't work in here. I imagine there'll be follow-along PRs as bugs are inevitably discovered and squashed.

It was a suggestion. I'm not a tooling or dependency maintainer, so I'll leave up to those groups to decide.

@fg-j @ryanmoran I tested out the workflow changes (with AWS upload steps removed, and actions pointing to this branch instead of main) and saw the workflow go green in the bundler POC using the checksum field and make appropriate changes to the buildpack.toml (see this commit)


Empty response in key builder

I have a custom key builder like this:
def key_builder(
    func: Callable[..., Any],
    namespace: str = "",
    *,
    request: Optional[Request] = None,
    response: Optional[Response] = None,
    args: Tuple[Any, ...],
    kwargs: Dict[str, Any],
) -> str:
    cache_key = hashlib.md5(  # noqa: S324
        f"{kwargs.get('xyz')}".encode()
    ).hexdigest()
    print(response.body)
    return f"{namespace}:{cache_key}"

and using it on a get method abc using @cache(key_builder=key_builder)
@router.get("/abc")
@cache(key_builder=key_builder)
async def abc(
        *, session: Session = Depends(get_session), xyz: str
):
    return {"hello":"world"}

but the response.body prints nothing, even though the method abc returns a dict.
Am I missing something here. Thanks.

I have browsed some of the relevant code of fastapi-cache and fastapi, and the following is my analysis:
The response parameter in key_builder is just an empty response object, which is generated by this line of code in fastapi.
https://github.com/tiangolo/fastapi/blob/6f6e786979eb3156b85e7c2f44d0b8af5cf0af17/fastapi/dependencies/utils.py#L522-L544
Actually, the response is created here.
https://github.com/tiangolo/fastapi/blob/6f6e786979eb3156b85e7c2f44d0b8af5cf0af17/fastapi/routing.py#L315-L326
Simply put, the response in key_builder is not the actual response; it can only be used to set the response headers, as it appears to be the case at the moment.
I hope this helps you

Thanks for looking.
I dont know if its possible, but having response body would be useful for some cases.


fix #211: EADDRINUSE

If a grip server is already running on the requested port, fail cleanly
instead of dumping the call stack.

@joeyespo I'm not 100% sure I picked a portable exception. Will this error manifest as a socket.error on non-Linuces? I tested on Ubuntu.
+    except socket.error as ex:

It's socket.errno with .errno == errno.EADDRINUSE for Python 2.7 on Windows 7 at least.
I think it should be fine everywhere else, too.

This looks good! Thanks for the fix 😃


Python: added tests for core_plugins

Motivation and Context

WIP
Description

Contribution Checklist

[x] The code builds clean without any errors or warnings
[x] The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
[x] All unit tests pass, and I have added new tests where possible
[x] I didn't break anyone :smile:

Python 3.10 Test Coverage Report • FileStmtsMissCoverMissingsemantic_kernel/core_plugins   conversation_summary_plugin.py16850%37–38, 64–73TOTAL607685186%
Python 3.10 Unit Test Overview

Tests
Skipped
Failures
Errors
Time

1399
1 :zzz:
0 :x:
0 :fire:
15.168s :stopwatch:


Keyboard navigation with tab key broken in Big Sur?

After upgrading to Big Sur, keyboard navigation of native dialogs appears to be broken as far as I can tell. An example of these dialogs appears if you edit a file in another editor while open in MacVim:

I have confirmed that Preferences > Keyboard > Shortcuts > Use Keyboard Navigation is enabled
I haven't seen this happen in any other app after updating.
OSX Version: 11.2.3
MacVim Version: 8.2.2681 (171)

It seems to work for me. (I could press "Tab", and then space bar to confirm) What doesn't work for you? As in, what key sequences are you expecting to work but doesn't?

Very interesting. The tab key wasn't working at all (no blue outline focus change). However turning the setting off and on and doing a full restart seemed to fix the issue. Also note that I had Karabiner installed but disabled it to debug. Not impossible it was causing interference.. At any rate hopefully this can at least be useful to someone experiencing the same issue.


Update menu.md

Ownership for this polyfill has changed to swisnl organization, could you update the link?

Thanks :octocat:

could you update the link?

Well it took a while, but your commit did it to the master branch with 4f213d7 and the change should be deployed in a matter of seconds.

Hehe, did take a little while trash, but thanks anyways for merging :+1


feat: update secret referenced by ingress

Signed-off-by: Ling Samuel<EMAIL_ADDRESS>
Type of change:

Fixes #1190

[x] Bugfix

What this PR does / why we need it:

Codecov Report

Merging #1243 (f8ae028) into master (fcfa188) will decrease coverage by 0.70%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #1243      +/-   ##
==========================================
- Coverage   42.78%   42.08%   -0.71%
==========================================
  Files          73       73
  Lines        6477     6585     +108
==========================================
  Hits         2771     2771
- Misses       3409     3517     +108
  Partials      297      297

Impacted Files
Coverage Δ

pkg/providers/apisix/translation/apisix_ssl.go
0.00% <0.00%> (ø)

pkg/providers/ingress/ingress.go
4.09% <0.00%> (-1.13%)
:arrow_down:

pkg/providers/ingress/provider.go
0.00% <0.00%> (ø)

...providers/ingress/translation/annotations/types.go
100.00% <ø> (ø)

pkg/providers/translation/util.go
15.00% <0.00%> (-3.19%)
:arrow_down:

pkg/providers/utils/ingress_status.go
0.00% <0.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


ciao-launcher: Remove --server command line option

Remove the --server option from the ciao-launcher. The SSNTP
server will be obtained from the certificate. With this change
the launcher no longer needs to create the ciao CNCI config
payload. It also does not need to generate the ciao config iso.
Signed-off-by: Manohar Castelino<EMAIL_ADDRESS>

@markdryan can you review this and let me know if this addresses the issue that you flagged.

@mcastelino Yes, the launcher changes look fine but the payload changes seem to break the payloads unit tests.

@mcastelino  Actually the ciao-launcher/README.md file needs to be updated as well to remove the --server parameter.

Coverage increased (+0.2%) to 60.848% when pulling 1acfcc2bcd87d2ddf8dcab739411282985382e28 on mcastelino:topic/singlevmauto into 88ce618dc1277bcca630d51915712e1727788000 on 01org:master.

@markdryan Addresses the review comments and fixed the payload failure.

Coverage increased (+0.2%) to 60.841% when pulling 4ce385f179a660a70833927652c861e158012bba on mcastelino:topic/singlevmauto into 88ce618dc1277bcca630d51915712e1727788000 on 01org:master.


Allow selecting text inside dm

I can't select text in a dm to try and copy paste something

same thing for android


[#163] Added code for loading Genie config for Python client

Adding code to load configuration for the Genie Python client to use
when submitting jobs.

Changes Unknown when pulling 74823b1809ca5f99ff2d6b6d0af92a6a843d030f on irontable:master into ** on Netflix:develop**.

Changes Unknown when pulling 74823b1809ca5f99ff2d6b6d0af92a6a843d030f on irontable:master into ** on Netflix:develop**.


How to import cjs?

Hi, I have problem with import:
import fastifyApollo from "@as-integrations/fastify";

Error: require() of ES Module ... not supported.
Instead change the require of .... to a dynamic import() which is available in all CommonJS modules.

I was try, but this not helped:
import fastifyApollo from "@as-integrations/fastify/build/cjs/index";
import fastifyApollo from "@as-integrations/fastify/build/cjs";
How can I import cjs?
P.S. I wan't use ESM

sandbox example: https://codesandbox.io/s/flamboyant-lumiere-mgfogd?file=/index.js

I removed type: module from package.json and all work for my project:
https://github.com/Luchanso/apollo-server-integration-fastify/commit/793780ccdb85f750bc6c516b9b92005a7ec580c0

Hi @Luchanso
Thanks for submitting this issue. We need to get this working so I will look into it.
I assume the fix is to remove the "type": "module" like you said but I need to do some research on if this is best.

The package has been updated to support type: "module" and type: "commonjs".

The real issue is in the package.json:
"exports": {
		".": {
			"types": "./build/esm/index.d.ts",
			"import": "./build/esm/index.js",
			"require": "./build/cjs/index.js"
		}
	},

The types property is breaking cjs builds. Simply remove it and everybody it will solve @Luchanso use case while keeping previous use cases working


small tweaks to the metal runtime

Moved the compute pipeline creation to init
Converted the copyin function to use numpys copy, removing realloc of MTLBuffer.
tested on an intel mac

Both copyin and copyout are expected to actually make a copy.
Remove excess comments and spacing and should be good to merge.


Fix npm warning: 'repositories' (plural) Not supported. Please pick one ...

...as the 'repository' field

Thanks @foxdonut!


Search Within Rarity

Just requesting a feature to restrict searches to a specific rarity to help pauper and artisan formats.
Thanks for all the work you've been doing.

Replaced this feature with format legality (more straightforward)
Solved in next release (already available for preview)
You will be able to search for legal:pauper
Magic Arena formats aren't supported yet


License checker and license update

Hi @mcdee
I wanted to get your thoughts on this. Essentially 2 changes:

Add a github action that ensures the expected license is present on all go files (the license is defined in .licenserc.json).
A golang script that will use the template from .licenserc.json and update all go files to match.

Perhaps the second part is more of a generic tool that we want to store somewhere else and use as a binary across multiple repos, but I thought I'd open the change and get your feedback first.
I did run the updater as part of the PR and it's created a lot of changes, but I'm hoping with the github action enforcing new changes then we would only have to run this once per year to update to the current year.

Regarding a license check, there is a goheader linter that can be enabled in .golangci-lint.yml, e.g.:
linters-settings:
  goheader:
    values:
      regexp:
        YEARS: (20\\d\\d|20\\d\\d, 20\\d\\d|20\\d\\d - 20\\d\\d)
    template: |-
      Copyright © {{ YEARS }} Attestant Limited.
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at

          http://www.apache.org/licenses/LICENSE-2.0

     Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License.

I'm also not sure about the update process, because it resets the copyright date to the current year.  Copyright starts with the date of first implementation, with (optionally) additional years added when there are major updates.  I think that as long as we have the linter in place the update isn't necessary.


Wrong 'externals' processing

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
<EMAIL_ADDRESS>doesn't import externals correctly.
If the current behavior is a bug, please provide the steps to reproduce.
Just build project from gist https://gist.github.com/MykhailoMatiiasevych/0561feca555398fcd2fc9279faae518e
dependency1 - works as expected,
dependency2 - imported as window[undefined]
dependency3 - cause Invalid configuration object. error
changing libraryTarget doesn't solve the issue

What is the expected behavior?
dependancy2 should be imported as windows['library2']['module']
dependency3 should be imported somehow without errors
If this is a feature request, what is motivation or use case for changing the behavior?
Please mention other relevant information such as the browser version, Node.js version, webpack version and Operating System.
Node.js: v8.1.4
npm: v5.0.3
webpack: 3.2.0
macOS Sierra

Looks like generic https://github.com/webpack/webpack/issues/5202

I guess it's only implemented for the UMD libraryTarget. You can send a PR adding the same behavior for the other libraryTargets. Best extract this code into a common file.