Azure web job initialization throws exception during deployment

Is this a bug report?
Yes

Can you also reproduce the problem with the latest version?
Yes
Environment

Operating system: Windows 10
Visual Studio version: 15.10.0
Dotnet version: 2.0

Steps to Reproduce
Repro steps
Provide the steps required to reproduce the problem

Create an azure webjob with a function with input binding to Service Bus Trigger.

Deploy the webjob into azure app service using publish as azure webjob from the visual studio.

Sometimes, there is an exception thrown inside the webjob main method.  The operation cannot be performed because the entity has been closed or aborted. during deployment of the azure webjob. The exception is thrown from azure function before the start of executing the function.
The full detail of the exception is provided below.

Expected behavior
No exception should be thrown.
Actual behavior
The azure function throws an exception.
  The operation cannot be performed because the entity has been closed or aborted.,  InnerException: System.ServiceModel.CommunicationObjectAbortedException: Internal Server Error: The server did not provide a meaningful reply; this might be caused by a premature session shutdown. TrackingId:1c4173c9-deba-4e80-9294-01cdde970d7a, Timestamp:1/11/2019 4:29:45 AM    at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)    at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.EndRequest(IAsyncResult result)    at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory1.RequestSessionChannel.RequestAsyncResult.<>c.b__9_3(RequestAsyncResult thisPtr, IAsyncResult r)
at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.StepCallback(IAsyncResult result) --- End of stack trace from previous location where exception was thrown ---    at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)    at Microsoft.ServiceBus.Common.AsyncResult1.End(IAsyncResult asyncResult)
at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory1.RequestSessionChannel.EndRequest(IAsyncResult result)    at Microsoft.ServiceBus.Messaging.Sbmp.RedirectBindingElement.RedirectContainerChannelFactory1.RedirectContainerSessionChannel.RequestAsyncResult.<>c__DisplayClass8_1.b__4(RequestAsyncResult thisPtr, IAsyncResult r)
at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.StepCallback(IAsyncResult result) --- End of stack trace from previous location where exception was thrown ---    at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)    at Microsoft.ServiceBus.Common.AsyncResult1.End(IAsyncResult asyncResult)
at Microsoft.ServiceBus.Messaging.Sbmp.RedirectBindingElement.RedirectContainerChannelFactory1.RedirectContainerSessionChannel.EndRequest(IAsyncResult result)    at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory1.RequestSessionChannel.RequestAsyncResult.<>c.b__9_3(RequestAsyncResult thisPtr, IAsyncResult r)
at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.StepCallback(IAsyncResult result) --- End of stack trace from previous location where exception was thrown ---    at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)    at Microsoft.ServiceBus.Common.AsyncResult1.End(IAsyncResult asyncResult)
at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory1.RequestSessionChannel.EndRequest(IAsyncResult result)    at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.EndReceiveCommand(IAsyncResult result, IEnumerable1& messages)
Details: System.OperationCanceledException: The operation cannot be performed because the entity has been closed or aborted. ---> System.ServiceModel.CommunicationObjectAbortedException: Internal Server Error: The server did not provide a meaningful reply; this might be caused by a premature session shutdown. TrackingId:1c4173c9-deba-4e80-9294-01cdde970d7a, Timestamp:1/11/2019 4:29:45 AM
at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.EndRequest(IAsyncResult result)
at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory1.RequestSessionChannel.RequestAsyncResult.<>c.<GetAsyncSteps>b__9_3(RequestAsyncResult thisPtr, IAsyncResult r)    at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.StepCallback(IAsyncResult result)
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
at Microsoft.ServiceBus.Common.AsyncResult1.End(IAsyncResult asyncResult)    at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory1.RequestSessionChannel.EndRequest(IAsyncResult result)
at Microsoft.ServiceBus.Messaging.Sbmp.RedirectBindingElement.RedirectContainerChannelFactory1.RedirectContainerSessionChannel.RequestAsyncResult.<>c__DisplayClass8_1.<GetAsyncSteps>b__4(RequestAsyncResult thisPtr, IAsyncResult r)    at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.StepCallback(IAsyncResult result)
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
at Microsoft.ServiceBus.Common.AsyncResult1.End(IAsyncResult asyncResult)    at Microsoft.ServiceBus.Messaging.Sbmp.RedirectBindingElement.RedirectContainerChannelFactory1.RedirectContainerSessionChannel.EndRequest(IAsyncResult result)
at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory1.RequestSessionChannel.RequestAsyncResult.<>c.<GetAsyncSteps>b__9_3(RequestAsyncResult thisPtr, IAsyncResult r)    at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.StepCallback(IAsyncResult result)
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
at Microsoft.ServiceBus.Common.AsyncResult1.End(IAsyncResult asyncResult)    at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory1.RequestSessionChannel.EndRequest(IAsyncResult result)
at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.EndReceiveCommand(IAsyncResult result, IEnumerable1& messages)    --- End of inner exception stack trace ---    at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.EndReceiveCommand(IAsyncResult result, IEnumerable1& messages)
at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.ReceiveAsyncResult.<>c.b__12_3(ReceiveAsyncResult thisPtr, IAsyncResult a)
at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.StepCallback(IAsyncResult result) --- End of stack trace from previous location where exception was thrown ---    at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)    at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.OnEndTryReceive(IAsyncResult result, IEnumerable1& messages)
at Microsoft.ServiceBus.Messaging.MessageReceiver.RetryReceiveAsyncResult.<>c.b__15_3(RetryReceiveAsyncResult thisPtr, IAsyncResult r)
at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.StepCallback(IAsyncResult result) --- End of stack trace from previous location where exception was thrown ---    at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)    at Microsoft.ServiceBus.Messaging.MessageReceiver.RetryReceiveAsyncResult.TryReceiveEnd(IAsyncResult r, IEnumerable1& messages)
at Microsoft.ServiceBus.Messaging.MessageReceiver.EndTryReceive(IAsyncResult result, IEnumerable1& messages)    at Microsoft.ServiceBus.Messaging.MessageReceiver.EndReceive(IAsyncResult result)    at Microsoft.ServiceBus.Messaging.MessageReceivePump.PumpAsyncResult.<>c.<GetAsyncSteps>b__7_3(PumpAsyncResult thisPtr, IAsyncResult r)    at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.StepCallback(IAsyncResult result)|
`

I don't think this has anything to do with MassTransit, not even listed in the call stack. Good luck figuring it out.


Request a feature

Hi
Thanks for this useful upload script
Please add a percentage progress bar to the next version when user wants to upload a file

it will be available in the next version
https://youtu.be/OX_fg7UkeSI


Add data channel middlewares

Description
Add datachannels middlewares
WIP

@decanus  @tarrencev  Please take a look of this implementation of the datachannel middlewares this will allow you to set a middleware chain on datachannels,  we can give access to session and it can be helpul to set an admin middleware to remove/silence users paired with an auth middleware, the API is straightforward to use and similar to a http middleware:
The way to add  a middleware chain is as follows:
	s := sfu.NewSFU(conf)
	dc := s.NewDatachannel(sfu.APIChannelLabel)
	dc.Use(datachannel.KeepAlive(5*time.Second), datachannel.SubscriberAPI)
        // Here its possible to fanout the message or act on processed msg or context values
        // If it's wanted we can give access to session to control other peers
        dc.OnMessage(func(ctx context.Context, msg webrtc.DataChannelMessage, in *webrtc.DataChannel, out []*webrtc.DataChannel) {
	})

This is the basic way to define a middleware:
func SubscriberAPI(next sfu.MessageProcessor) sfu.MessageProcessor {
	return sfu.ProcessFunc(func(ctx context.Context, args sfu.ProcessArgs) {
              next.Process(ctx,args)
        }
}

@decanus  @tarrencev  Please take a look of this implementation of the datachannel middlewares this will allow you to set a middleware chain on datachannels,  we can give access to session and it can be helpul to set an admin middleware to remove/silence users paired with an auth middleware, the API is straightforward to use and similar to a http middleware:
The way to add  a middleware chain is as follows:
	s := sfu.NewSFU(conf)
	dc := s.NewDatachannel(sfu.APIChannelLabel)
	dc.Use(datachannel.KeepAlive(5*time.Second), datachannel.SubscriberAPI)
        // Here its possible to fanout the message or act on processed msg or context values
        // If it's wanted we can give access to session to control other peers
        dc.OnMessage(func(ctx context.Context, msg webrtc.DataChannelMessage, in *webrtc.DataChannel, out []*webrtc.DataChannel) {
	})

This is the basic way to define a middleware:
func SubscriberAPI(next sfu.MessageProcessor) sfu.MessageProcessor {
	return sfu.ProcessFunc(func(ctx context.Context, args sfu.ProcessArgs) {
              next.Process(ctx,args)
        }
}


Can we handle index error in kernel's active_dims parameter?

Error handling
While initializing kernels we can pass the active_dims parameter to specify the dimension on which the kernel is applied. If the specified active_dims is out of bounds for data X, tf.gather(.) returns 0 without throwing any error.
If this scenario is handled, maybe by asserting the data dimensions and active_dims value, debugging in some scenarios can be convenient and time-saving.
To reproduce
Minimal, reproducible example

import tensorflow as tf
a = tf.constant([1,2,3,4])
a1 = tf.gather(a,[1,3])
print(a1)
<tf.Tensor: shape=(2,), dtype=int32, numpy=array([2, 4], dtype=int32)>
a2 = tf.gather(a,[5,99])
print(a2)
<tf.Tensor: shape=(2,), dtype=int32, numpy=array([0, 0], dtype=int32)>

Implementation
If sounds reasonable I can raise a PR for it. :+1:

The label should probably be discussion.


Add support and doc for configuration file

In reviewing the 1.3.0 release, the groundwork for supporting the configuration file was in place, but not documented.  I tested it based on a review of the code and have a working configuration with the included code update as documented in the README update.

@aharden thanks Alex, much appreciated.


Make it possible to construct errors with custom description

This is useful if a more meaningful message can be displayed to the
user with Error::exit. For example, if a file is not found, the
converted io::Error will give a message like:
"error: entity not found"

With this, it is possible to replace this message with a more useful
one, like for instance:
"error: configuration file not found"

Coloring is respected. Some duplication in the From::from impls was
reduced.

Please see the contribution instructions for more information.
More specifically, can you change the commit message to follow the format on the link above?
@kbknapp The link in the automatic bot message is pointing to CONTRIBUTING.md in the repo root directory, which is dead link now (it is moved to .github/CONTRIBUTING.md).

More specifically, can you change the commit message to follow the format on the link above?

Sure, I'll rebase after review.

Looks good to me! Thanks for this! @sru I'll change that link, thanks for pointing it out.
@vks once you rebase I'll merge :+1:

Rebased with a reformatted commit message and slightly extended documentation.

@vks since #639 just merged, can you rebase one last time, and I'll merge this today.
Thanks!

Rebased and updated term_size dep in #646


Changed the User-Agent format

Closes #52 again. ;)

@craffit Agreed this is in the right format?


Handle slash in subject name while performing Schema Registry API calls

Description

One of the limitations mentioned in schema registry documentation is that subject name can contain "/" character. However, we have to escape it in the schema registry APIs
This change replaces "/" in the subject name with "%2F" thereby allowing API calls to succeed

Use case

We have one of our schemas referencing other schema containing "/" in its name. Due to the above mentioned limitation, schema_registry_converter::async_impl::proto_decoder::deserialize_with_context method is returning Could not get id from response as the response. After this fix, deserialize_with_context is returning valid data and things are working E2E as expected

Testing

Tested on my confluent schema registry with subject name as "a/b/c.proto"
API call failing with {"error_code":404,"message":"HTTP 404 Not Found"} error before the change and the call has
succeeded after changing subject name to "a%2Fb%2Fc.proto"

@gklijs could you please review this change?

It looks fine. It would be nice to have it validated with at least a unit test. It would also be nice to have the replace in a separate private function.

@saiharshavellanki I think the forked protofish was pushed here by accident? Although I'm interested why. Can you add test and make it a separate function? I can also do it on main, but I like to give you the credits, and I can modify this pr.

@gklijs Since protofish doesn't seem to have recent contributions, we have forked it (https://github.com/fennel-ai/protofish) and performed below changes on top:

Support map type
Allow parsing identifiers that start with underscores

Additionally it requires changes in the current library to point to our forked version of protofish crate. On the side note, we have also forked the current library (https://github.com/fennel-ai/schema_registry_converter) to perform those changes in the interest of time

Can you add test and make it a separate function? I can also do it on main
Sure. I can add test and make it a separate method

I think for protofish it would be nice to create in issue with them/the community. It would be nice to change to an 'official' fork. Just like when Avro was moved to be included in the Apache project. Preferably, the current maintainers give their blessing, and point to the new location. Than for this repo/project it's just a matter of updating the dependency.

@saiharshavellanki Do you have any update? I can't merge the pr like this, as it needs a released version as depednency.

@gklijs Unfortunately this was missed somehow. Thanks for merging the fix 🚀


Q-Dance Network

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting a new site support request
[X] I've verified that I'm running yt-dlp version 2023.06.22 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that none of provided URLs violate any copyrights or contain any DRM to the best of my knowledge
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and am willing to share it if required

Region
Australia
Example URLs

Single stream: https://www.q-dance.com/network/live/149170193

Provide a description that is worded well enough to be understood
Website streams the liveset of music festivals such as Defqon.1 & Qlimax. The stream can be downloaded via m3u8 using extensions that target and download m3u8 files.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['--cookies-from-browser', 'brave', '-vU', 'https://www.q-dance.com/network/live/149170193']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[42f2d40b4] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19044-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 6.0-full_build-www.gyan.dev (setts), ffprobe 6.0-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[Cookies] Extracting cookies from brave
[debug] Extracting cookies from: "C:\Users\User\AppData\Local\BraveSoftware\Brave-Browser\User Data\Profile 5\Network\Cookies"
[debug] Found local state file at "C:\Users\User\AppData\Local\BraveSoftware\Brave-Browser\User Data\Local State"
[Cookies] Extracted 95 cookies from brave
[debug] cookie version breakdown: {'v10': 95, 'other': 0, 'unencrypted': 0}
[debug] Proxy map: {}
[debug] Loaded 1848 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version<EMAIL_ADDRESS>Current version<EMAIL_ADDRESS>Current Build Hash: 0e00f0a64217f271301625b09ae9ed05fe430e5a42391389504ddc01b2e42380
[debug] Downloading _update_spec from https://github.com/yt-dlp/yt-dlp/releases/latest/download/_update_spec
Updating to<EMAIL_ADDRESS>...
[debug] Downloading yt-dlp.exe from https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.exe
[debug] Downloading SHA2-256SUMS from https://github.com/yt-dlp/yt-dlp/releases/latest/download/SHA2-256SUMS
Updated yt-dlp to<EMAIL_ADDRESS>[debug] Restarting: yt-dlp --cookies-from-browser brave -vU "https://www.q-dance.com/network/live/149170193"
[debug] Command-line config: ['--cookies-from-browser', 'brave', '-vU', 'https://www.q-dance.com/network/live/149170193']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[812cdfa06] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19044-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 6.0-full_build-www.gyan.dev (setts), ffprobe 6.0-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[Cookies] Extracting cookies from brave
[debug] Extracting cookies from: "C:\Users\User\AppData\Local\BraveSoftware\Brave-Browser\User Data\Profile 5\Network\Cookies"
[debug] Found local state file at "C:\Users\User\AppData\Local\BraveSoftware\Brave-Browser\User Data\Local State"
[Cookies] Extracted 95 cookies from brave
[debug] cookie version breakdown: {'v10': 95, 'other': 0, 'unencrypted': 0}
[debug] Proxy map: {}
[debug] Loaded 1851 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version<EMAIL_ADDRESS>Current version<EMAIL_ADDRESS>Current Build Hash: 37e7ffe204309357cfd1388b0e2c782a30e293ddd0f2761a9a8f6afa185b3566
yt-dlp is up to date<EMAIL_ADDRESS>[generic] Extracting URL: https://www.q-dance.com/network/live/149170193
[generic] 149170193: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] 149170193: Extracting information
[debug] Looking for embeds
ERROR: Unsupported URL: https://www.q-dance.com/network/live/149170193
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1555, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1631, in __extract_info
  File "yt_dlp\extractor\common.py", line 708, in extract
  File "yt_dlp\extractor\generic.py", line 2568, in _real_extract
yt_dlp.utils.UnsupportedError: Unsupported URL: https://www.q-dance.com/network/live/149170193

I'm offering 5USD for whoever adds yt-dlp support for Q-Dance Network (assuming it's DRM free)

I figured out I can download Q-Dance Network streams with yt-dlp. This involves opening developer tools to filter for the m3u8 in the Networks option and then copying the link address of the m3u8. Then I could just use yt-dlp URL.
That being said, I'm still looking for someone to add direct yt-dlp support for Q-Dance Network content links so I don't have to search for the m3u8 for each different video link. Therefore, the 5USD bounty is still up.

Sure, DM me on discord, same username


java.lang.OutOfMemoryError when the workdir is "/"

Please answer these questions before submitting your issue.

Why do you submit this issue?
[ ] Question or discussion
[x] Bug
[ ] Requirement
[ ] Feature or performance improvement

Bug

Which version of SkyWalking, OS, and JRE?
8.x

java.lang.OutOfMemoryError: Java heap space
at java.io.UnixFileSystem.resolve(UnixFileSystem.java:110) ~[?:?]
at java.io.File.(File.java:263) ~[?:?]
at java.io.File.listFiles(File.java:1226) ~[?:?]
at com.google.common.reflect.ClassPath$DefaultScanner.scanDirectory(ClassPath.java:551) ~[guava-28.1-jre.jar:?]
at com.google.common.reflect.ClassPath$DefaultScanner.scanDirectory(ClassPath.java:562) ~[guava-28.1-jre.jar:?]
at com.google.common.reflect.ClassPath$DefaultScanner.scanDirectory(ClassPath.java:562) ~[guava-28.1-jre.jar:?]
when I run the command
sh /${skywalking_home}/bin/startup.sh
The current dir "/" is added to the classpath by this code

CLASSPATH="$OAP_HOME/config:$CLASSPATH"

Because the CLASSPATH may be empty before.

This is new, we never expect you will run in this way.

This is new, we never expect you will run in this way.

I want to run it in a docker container. I copy the skywalking-home to "/" which causes this problem.

We have officer container image, take a look.

We have officer container image, take a look.

OK, thank u.

我很好奇为什么会这样，我也遇到了，都怀疑人生了


gh-pages: list markers are outside the text block

maybe a ul { padding-left: 40px; } would fix it - I'm not a front-end guy

That's how browser renders <ul> by default.

That's how browser renders <ul> by default.

Shocking news!
This is issue is about normalize.css' website, not CSS code.
https://nicolasgallagher.com/about-normalize-css/

@szepeviktor, I am new to Github, I can solve this issue, can u please  explain in brief what I have to do

I cannot answer. I do not know how normalize.css' website should look.


headObject Bug

https://github.com/RiptideElements/s3fs/blob/master/lib/s3fs.js#L485-L486
If I use callback instead of promise, if success callback is invoked without data!

All the more reason to not use callbacks :grin:!
Thanks for pointing that out, I'll drop a PR for it. Looks like none of the tests for headObject are validating return values so I'll add that so we can catch and regressions.

@diegobent I just published<EMAIL_ADDRESS>to NPM which has this fix in it. Cheers!

Thanks @DavidTPate for the support! :+1:
Congratulation, for your work! very useful and well done library!  good implemetation and very hight code covarage!


Docker Packaging Script Needed

Currently the start.sh does not match what is built in the Dockerfile . We need to create a packaging script that automatically pushes to dockerhub,

When building the Dockerfile after cloning the repository I get this message:
lawn_tractor/docker$ ./build.sh
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /home/matt/ros/tractorsim_ws/lawn_tractor_pr/lawn_tractor/docker/Dockerfile: no such file or directory
Fix:  Move build.sh to root dir.
The container fails to build with the message:
ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
lawn_tractor_navigation: Cannot locate rosdep definition for [geonav_transform]
The command '/bin/sh -c rosdep update &&     rosdep install -y --from-paths . --ignore-src --rosdistro ${ROS_DISTRO} --as-root=apt:false &&     apt-get -qq upgrade &&     rm -rf /var/lib/apt/lists/*' returned a non-zero code: 1

RE: Build Script
I imagined the user to run the script from the root directory. I will improve the documentation.
Example:
vinny@pop-os:~/Code/lawn_tractor_ws/src$ ./docker/build.sh
RE: Container Build Failure
I am not able to replicate but I have the feeling that was caused by not initializing the submodule before building.
I should add in the instructions or in the build script:
git submodule update --init --recursive


WebUI Concepts for cookieplone – research and suggestions

to @davisagli:
@ericof and me discussed with others  on the Plone Beethoven Sprint 2024 options to enhance the user story & experience using cookiecutter and the Plone Distributions chooser forms.
Prerequisite and some Vision
Kickoff Story for me: Some of these aspects are also driven by an ad hoc example that I showed up during a Lightnig Talk at Plonetagung Gießen 2024 and a short exchange with @fredvd about the user experience having cookiecutter replacing the setup dialog for Plone sites as a chat dialog. (I demoed cookiecutter as prompt generator for a RAD LLM during AI Summer Shool 2023 in Cologne using a NiceGUI Chat Frontend. (Examples: Chat AI-Interface
Targeting an appealing UX for wider audiences makes sense and may end up with offering an WebUI to cookiecutter driving the creation processes around Plone with more fun and ease of use, making it easier to get the options and impress during creation with productive workflows and fast results. Answering customer requests with a fast setup of a demo distribution is always impressing and shows up the quality of the toolchain from the first step.
Imagine the current dialog form when creating an initial Plone site after the cookiecutter setup is going to bootstrap the frontend / backen setup in the two distribution flavors Volto and ClassicUI.
Go further and imagine this in a more dialog driven approach similar to an LLM AI chat experience and ending up in documentation the choosen options for reference or later picking up an/or modify a prior setup run.
Expect to find a nice individual documentation similar to the ansible stuff populated in the devops folder precisely mentioning your choosen options and consequences.
Research for existing WebUI Approaches using cookiecutter
@acsr did some research in the past and provide some tagged links
Create web UI tool for cookiecutter · Issue #1067 · cookiecutter/cookiecutter
url:: https://github.com/cookiecutter/cookiecutter/issues/1067
tags:: #[[cookiecutter]], #[[cookiecutter WebUI]], #[[Interface]], #[[Templating]], #[[Terminal]]

First off, I looooove cookiecutter! Thanks for creating it and working on it! ❤ This is a bit of a long-shot, but wasn't sure where else to leave this. I work in a community where not everyone has...

cjolowicz/cookietemple
creators:: programmer: Claudio Jolowicz
url:: https://github.com/cjolowicz/cookietemple
tags:: #[[cookiecutter]], #[[cookiecutter WebUI]], #[[Interface]], #[[Templating]], #[[Terminal]]

A collection of best practice cookiecutter templates for all domains and languages with extensive Github support

samj1912/CookieS
creators:: programmer: Sambhav Kothari
url:: https://github.com/samj1912/CookieS
tags:: #[[cookiecutter]], #[[cookiecutter WebUI]], #[[Interface]], #[[Templating]], #[[Terminal]]

Cookiecutter as a Service
@acsr: This is the 4 years old prototype mentioned in https://github.com/cookiecutter/cookiecutter/issues/1067 (but not continued) -> watch the gifanim to get a taste of the UX Feeling

cookiedough: browser and install GUI for cookiecutter templates
url:: https://fossil.include-once.org/cookiedough/
tags:: #[[cookiecutter]], #[[cookiecutter WebUI]], #[[Interface]], #[[Templating]], #[[Terminal]]

cookiedough 0.1.0 - new GUI frontend for cookiecutter
creators:: author: milki_
url:: www.reddit.com/r/Python/comments/mbigbs/cookiedough_010_new_gui_frontend_for_cookiecutter/

Web-Based Django Cookiecutter Generator - Easy Setup Online
creators:: author: AI SaaS Template
url:: https://aisaastemplate.com/tools/django-cookiecutter/
tags:: #[[cookiecutter]], #[[cookiecutter WebUI]], #[[Interface]], #[[Templating]]

Streamline your Django project setup with our web-based Cookiecutter generator. Avoid command-line errors and easily modify choices through our user-friendly f…

Guillem96/visual-cookiecutter
creators:: programmer: Guillem Orellana Trullols
url:: https://github.com/Guillem96/visual-cookiecutter
tags:: #[[cookiecutter]], #[[cookiecutter WebUI]], #[[Interface]], #[[Templating]], #[[Terminal]]

Bake cookiecutter templates visually (with a Streamlit web UI)

@ericof introduced already recognized tools and approaches:
rjsf-team/react-jsonschema-form
url:: https://github.com/rjsf-team/react-jsonschema-form
tags:: #[[cookiecutter]], #[[Form]], #[[Formbuilder]], #[[JSON]], #[[JSON Schema]], #[[React]]

A React component for building Web forms from JSON Schema.

Textual - Home
url:: https://textual.textualize.io/
tags:: #[[cookiecutter]], #[[Interface]], #[[Templating]], #[[Terminal]], #[[Textualize]]
zotero:: @TextualHome2024

Textual is a TUI framework for Python, inspired by modern web development.

Textualize
url:: https://github.com/Textualize
tags:: #[[cookiecutter]], #[[Interface]], #[[Templating]], #[[Terminal]], #[[Textualize]]

Textualize/rich
url:: https://github.com/Textualize/rich
tags:: #[[cookiecutter]], #[[Interface]], #[[Templating]], #[[Terminal]], #[[Textualize]]

Rich is a Python library for rich text and beautiful formatting in the terminal.
Rich 13.6.0 documentation
url:: https://rich.readthedocs.io/en/stable/

I enhance this further as I go here.

It could be interesting to use https://pyscript.com/ for this.


Defect: allocatable coarrays take too long to allocate on a large number of images

Avg response time

Defect/Bug Report

OpenCoarrays Version: 1.8.2
Fortran Compiler: gfortran/6.3.0
C compiler used for building lib: unknown pre-installed likely gcc/6.3.0
Installation method: pre-installed on supercomputer
Output of uname -a: Linux cheyenne3 3.12.62-60.64.8-default #1 SMP Tue Oct 18 12:21:38 UTC 2016 (42e0a66) x86_64 x86_64 x86_64 GNU/Linux
MPI library being used: MPT/2.15 (vendor:SGI)
Machine architecture and number of physical cores: Cheyenne: Xeon + infiniband, >9k cores tested
Version of CMake: unknown pre-installed likely cmake/3.7.2

Observed Behavior
Allocating a few coarrays (e.g. allocate(coarray(nx,ny,nz)[*])) can take 1/2 hour on a large number of cores.  See tests case and documentation here: https://github.com/gutmann/caf_alloc_test
Expected Behavior
Allocating coarrays should take longer on more cores, but not such that the entire permittable wall clock in a job scheduler expires before allocation completes...
Steps to Reproduce
git clone https://github.com/gutmann/caf_alloc_test
cd caf_alloc_test
make
# if necessary, submit job to your job scheduler here
cafrun -np 360 ./alloc_test

@afanfa Because this issue first showed up on Cheyenne, I'm hoping you won't mind investigating it so I'm assigning this to you.  If you won't have the time, feel free to let me know and we'll try to reproduce a similar result elsewhere.

Please note: With fixing #399 register() will look significantly different and use far less synchronization and MPI_Allocs.

@vehre Do you have anytime to work on this?  I think @gutmann would agree that it severely limits the utility of OpenCorrays at scale.  He discovered this in the process of performing scaling studies to gather data for a paper that is due July 31.

Well, in the register()-routine it is already fixed. But I haven't had time to port the new style to get_by_ref(), which is used in the testcases far more often than before. Because I am in trouble with my current project, I will not have to finish this before July 31.

@vehre Thanks for the quick replies and good luck with your current project.  @sfilippone You might have the MPI expertise to look at these.  I believe @afanfa is out of the office.

Will take a look asap
On Tue, Jul 25, 2017 at 4:02 PM, Damian Rouson<EMAIL_ADDRESS>wrote:

@vehre https://github.com/vehre Thanks for the quick replies and good
luck with your current project. @sfilippone
https://github.com/sfilippone You might have the MPI expertise to look
at these. I believe @afanfa https://github.com/afanfa is out of the
office.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sourceryinstitute/OpenCoarrays/issues/420#issuecomment-317766384,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APX-T2LHTUgUDw-l5Ex1XtF_Dzgd2IK8ks5sRgNpgaJpZM4OhlQg
.

Well, looking again at this. The fix for #399 will not fix this issue, because #399 addresses a gcc7 issue only, but here we are using gcc6.3, i.e. the very old style.

Well, I could probably switch to gcc7 but, IIRC, there is an issue with gcc7 not actually working with opencoarrays and sending data over MPI in the first place... (or something like that, @rouson, do you remember what the issue is?)  Something about the interface for one of the two (gcc or OCA) getting revamped and just having stubs for some routines?
I'm also not sure what it will take for me to install an updated OCA on Cheyenne that will work with the MPT mpi library (though I think openMPI and others are available too). I could see it being quasi trivial, or I could see it being a nightmare of reconfiguring make/cmake/etc scripts to connect to the right libraries. (For some of us that is a nightmare ;) )
For now I just won't test scaling beyond ~3600 cores on gcc, though hopefully we will soon on intel and cray... it just won't look good for OCA when those plots stop at ~3600 :P

Sending data over MPI works as long as it does not affect allocatable components or allocatable objects at the target. But switching to gcc7 will not solve your issue, because the allocate for gcc7 is even more complicated and allocating more objects than the gcc6 one.

Ah that was it, thanks... yes that would be a problem still, thanks!

Sorry for the inconveniences. I am on another project and the moment and my time is limited.

@gutmann, if you need help installing OpenCoarrays against MPT or OpenMPI let me know. As discussed, it sounds like GCC 7 is not the right approach, was this the only reason you wanted to do a new install on Cheyenne?

@zbeekman, thanks!  I'm more thinking that in a few months if/when this is fixed, I (or @afanfa ) will need to install an updated version on Cheyenne.  This can probably be in my home directory, though @afanfa may be able to get it installed for all users which would be preferable.
gcc7 is already available on Cheyenne, and they are pretty good about installing updates to various packages, though I don't know how many are using OCA on Cheyenne, so it may not be a high priority for our computing group.

@vehre, I just built OpenCoarrays 1.9.0-32-g7dbcf92 using GCC 7.1 and the non-developer tests of allocatable components pass:
      Start  7: register_alloc_comp_1
 7/43 Test  #7: register_alloc_comp_1 ............   Passed    0.09 sec
      Start  8: register_alloc_comp_2
 8/43 Test  #8: register_alloc_comp_2 ............   Passed    0.09 sec
      Start  9: register_alloc_comp_3
 9/43 Test  #9: register_alloc_comp_3 ............   Passed    0.09 sec
      Start 10: async_comp_alloc
10/43 Test #10: async_comp_alloc .................   Passed    0.14 sec

Probably I'm missing something, but maybe you could give more detail on what was meant by, "Sending data over MPI works as long as it does not affect allocatable components or allocatable objects at the target."

FYI for anyone else reading this, it sounds like the issue is that gcc7+opencoarrays can not to puts (e.g. array(:)[other_image]=array(:) ) but maybe it can do gets (array(:) = array(:)[other_image]) over MPI with allocatable data.

@gutmann we need to wait to hear from @vehre, but I don't think this is true in general, but it may be true in the specific case of allocatable components. I'm investigating.

Agreed, that is what I meant by with allocatable data, but I guess you mean it is even more specific to allocatable components of derived types? While that is sort of a special case, it may also be one of the most common cases outside of simple tests programs (I think pretty much every piece of vaguely modern Fortran code I have written or used has used allocatable arrays that are components of a derived type as the core data structure)

I'm starting to get confused as to what is allowed in F2008 vs the
allocatable components introduced in the TS. I agree with you that it's a
feature we want to add, and as quickly as we can, but right now it is
highly experimental and the complexities of accomplishing this are numerous
including changes needed on the GFortran compiler side.
Hopefully I can find some time (not today most likely, abstracts due and
other fires) to refresh my memory on the semantics for what is allowed in
pure F2008 vs the additions from the TS "Additional Parallel Features..."
and also run some tests to better illustrate what does and does not
currently work.
On Mon, Jul 31, 2017 at 1:26 PM Ethan Gutmann<EMAIL_ADDRESS>wrote:

Agreed, that is what I meant by with allocatable data, but I guess you
mean it is even more specific to allocatable components of derived types?
While that is sort of a special case, it may also be one of the most common
cases outside of simple tests programs (I think pretty much every piece of
vaguely modern Fortran code I have written or used has used allocatable
arrays that are components of a derived type as the core data structure)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sourceryinstitute/OpenCoarrays/issues/420#issuecomment-319136118,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAREPJCtOc4P2J3g8tLfUGahC6tZS_Qkks5sTg4ogaJpZM4OhlQg
.

For now, the top priority is that we document capabilities clearly and make any associated error messages more descriptive.  That's the point of issue #422, which should be quick to fix when one of us gets around to it (I hope to take a whack at it later this week).  I'm a big fan of incremental feature addition so it was a really big step when we started providing experimental support for allocatable components inside derived-type coarrays, but I think we all agree that introducing a regression in allocatable components of non-coarrays is very unfortunate regression that is likely to be a deal-breaker for a lot of users. @vehre, @zbeekman, and @sfilippone , we might need to put our heads together soon and start going through the relevant code to see if we can come up to speed on the underlying issues and collectively devise a solution at least on the library side.  If the solution requires modifying the compiler, that will probably have to wait for  @vehre or @afanfa who are best qualified to address this but have other work commitments that prevent them from working on it anytime soon.

Also, @zbeekman, I'll be glad to help with the question of what was supported in Fortran 2008 vs Fortran 2015.  I just changed the name of this issue to make clear that it's a regression, albeit one that I wasn't aware of and one that was probably intentional because it relates to procedures that were refactored/replaced to support allocatable components inside derived type coarrays.  I knew the refactoring only provided partial support for the new feature being introduced.  I didn't know before this issue that the refactoring also broke existing functionality other than issue #292.  For that reason, I also just added this to the 2.0.0 release milestone, which has a due date of tomorrow so we'll have to push that date out.  We can talk about it during the OpenCoarrays call tomorrow.

OK, I missed @gutmann's link to his demo repo. This is EXTREMELY helpful. As a first step it is worth trying to reproduce on other systems (non-SGI/MPT etc.) to get a sense of  how universal this issue is and whether the problem is definitely in the library, etc. I think I'm clear RE F2008 vs 2015 additional parallel features.... I'll try to give it a shot somewhere else soon.

but I think we all agree that introducing a regression in allocatable components of non-coarrays is very unfortunate regression that is likely to be a deal-breaker for a lot of users.

I don't think this is a regression. @gutmann said he's compiling with 6.3. This means we're building the old ABI for the GCC 6.3 interface. I would consider it a regression if older versions of OpenCoarrays before the allocatable component work started exhibit the same behavior.
Can someone try https://github.com/gutmann/caf_alloc_test on Cayenne with on Cayenne with an older version of OpenCoarrays? Looking back, to before allocatable components was added, would it be possible to try with https://github.com/sourceryinstitute/OpenCoarrays/releases/tag/1.7.4 on Cheyenne? It looks like the 1.7 series predate any work on allocatable components of coarray derived types.

@zbeekman You're right.  I confused this with issue #422 because the data structures involved in this issue have the same essential features. Issue #422, which is a regression because issue #442 relates to functionality that worked in GCC 6 but is not yet supported in GCC 7.  I just undid my edit for the title of this issue, but I'll leave this issue as part of the 2.0.0 milestone. I'll instead add the regression label to issue #422.

It's worth trying an older 1.7 release, I'm not 100% sure how much of the
older code got touched vs how much just got wrapped in #ifndef GCC7....
On Mon, Jul 31, 2017 at 7:08 PM Damian Rouson<EMAIL_ADDRESS>wrote:

@zbeekman https://github.com/zbeekman You're right. I confused this
with issue #422
https://github.com/sourceryinstitute/OpenCoarrays/issues/422 because
the data structures involved in this issue have the same essential
features. Issue #422
https://github.com/sourceryinstitute/OpenCoarrays/issues/422, which is
a regression because issue #442 relates to functionality that worked in GCC
6 but is not yet supported in GCC 7. I just undid my edit for the title of
this issue, but I'll leave this issue as part of the 2.0.0 milestone. I'll
instead add the regression label to issue #422
https://github.com/sourceryinstitute/OpenCoarrays/issues/422.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sourceryinstitute/OpenCoarrays/issues/420#issuecomment-319221760,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAREPNR2B_CVkJJtNU4owMvIAFjSCCLKks5sTl5hgaJpZM4OhlQg
.

Trying to catch up on this:

@vehre, I just built OpenCoarrays 1.9.0-32-g7dbcf92 using GCC 7.1 and the non-developer tests of allocatable components pass:
 Start  7: register_alloc_comp_1

7/43 Test  #7: register_alloc_comp_1 ............   Passed    0.09 sec
Start  8: register_alloc_comp_2
8/43 Test  #8: register_alloc_comp_2 ............   Passed    0.09 sec
Start  9: register_alloc_comp_3
9/43 Test  #9: register_alloc_comp_3 ............   Passed    0.09 sec
Start 10: async_comp_alloc
10/43 Test #10: async_comp_alloc .................   Passed    0.14 sec

All of the above tests just allocate and check that the component is allocated (also remote). They do not access any data in the allocated component (neither local nor remote).
From ** gutmann commented 5 days ago **

Agreed, that is what I meant by with allocatable data, but I guess you mean it is even more specific to allocatable components of derived types? While that is sort of a special case, it may also be one of the most common cases outside of simple tests programs (I think pretty much every piece of vaguely modern Fortran code I have written or used has used allocatable arrays that are components of a derived type as the core data structure)

It gets even more complicated. Let's look at all possible kinds of coarrays and what is supported in current gfortran and ocaf:

coarray kind
example
gfortran support
ocaf support

global saved coarrays of intrinsic types
float, saved :: caf[*]
yes
yes

global allocatable coarrays of intrinsic types
float, allocatable :: caf[*]
yes
yes

coarrays of derived type w/o allocatable components
type(T), saved :: tcaf[] ; type(T), allocatable :: t2caf[]
yes
yes

coarrays in derived types (saved and allocatable)
type T ; float :: caf[*] ; end type T
yes
yes

allocatable components in derived type coarrays
type T ; float, allocatable :: acc ; end type T ; type(T), saved :: caf[*]
yes
wip *)

coarray get where the target needs to be implicitly allocated
float, allocatable :: target ; float, saved :: caf[*] ; target = caf[1]
yes
wip *)

coarray put or getput where the target needs to be implicitly allocated
float, allocatable :: source ; float, saved :: caf[*] ; caf[1] = source
yes
no

Sorry for the bad formating of the code in the table. I could not figure quickly how to get this right. But I hope you get the intention.
*) The work-in-progress is to support get() and put() operations on the allocatable component. My first proposal on this proved incomplete and non-functional. Therefore, no at the moment allocatable components are not supported by opencoarrays. Not even a simple get() on them will work reliably.

From  'rouson commented 5 days ago':

but I think we all agree that introducing a regression in allocatable components of non-coarrays is very unfortunate regression that is likely to be a deal-breaker for a lot of users.

As I do understand this issue we are talking about allocation speed on large clusters with gfortran 6.x. Gfortran 6.x does not have any support for allocatable components in derived type coarrays. Therefore there is no impact of the code added to support those. When the allocation is slow for gfortran 6.x, then I assume that the mpi implementation has an issue either in the MPI_Win_allocate () or in the routines used to sync() after the allocate. There is just one MPI_Win_allocate in the whole routine to allocate the coarray's data. I don't know how to further optimize that.
Taking a look at your example and computing what is going on:
nx * ny *nz * sizeof(real) * coarray1-9 * 9-times calling alloc_test
100 1000 30 * 4 * 9 * 9 = 972000000 bytes ~ 1 GB
Given that each node has 36 cores and assuming that you recieve all 36 cores of one node, each node needs to have at least 36 GB. Given that each node has at least 64 GB this should be sufficient as long as sizeof(real) really is 4 bytes on your system. When it is 8 bytes there is the fault!
Nevertheless does this look to me more like a memory management issue of the underlying mpi-library then one of opencoarrays, but I am open to other suggestions.

Also, @zbeekman, I'll be glad to help with the question of what was supported in Fortran 2008 vs Fortran 2015. I just changed the name of this issue to make clear that it's a regression, albeit one that I wasn't aware of and one that was probably intentional because it relates to procedures that were refactored/replaced to support allocatable components inside derived type coarrays. I knew the refactoring only provided partial support for the new feature being introduced. I didn't know before this issue that the refactoring also broke existing functionality other than issue #292. For that reason, I also just added this to the 2.0.0 release milestone, which has a due date of tomorrow so we'll have to push that date out. We can talk about it during the OpenCoarrays call tomorrow.

For a gfortran 6.x the refactoring is not relevant. 6.x code was not changed for allocatable components of derived type coarrays, because 6.x does not support them (do not mix up with coarray in derived types. Their support is unchanged and present in 6.x).

Thanks @vehre If this is a bug or issue in the MPI library, that would be good to know.  Do you have a simple MPI test case in mind that might help shed light on that?  I'd be happy to run it on Cheyenne.
@rouson can you run with gfortran on swan?  If so, would you mind testing this example, or better the full coarray_icar with gfortran and a large number of image to see if the problem persists there?  I believe Brian has gfortran + opencoarrays running on one or both of their Crays, so maybe he can test coarray_icar with ~3000 images and see if it runs into similar problems with a different MPI library.
re: memory requirements:  While you are right that this test code pushes the memory requirements a little (though reals are 4 bytes not 8), the bigger problem came when running coarray_icar.  In that case, coarray_icar was using a domain of ~1000 x 1000 x 20, but that domain was split over 3000+ images, so each image was only using a tiny fraction of memory (even with 36 on one node it isn't a problem.)  I'm fairly certain I could reduce the total memory requirements of this test case with a local domain of ~20x20 and still see problems on a very large number of images.

Well, first of all coarray-allocated memory is a keeper: Explicitly allocated memory for a coarray, needs to be deallocated, i.e., gfortran does not automatically/implicitly deallocate a coarray. The memory is simply lost, but never reported when you run your program with valgrind and check for memory leaks, because opencoarray cleans up on termination and keeps a list of all coarrays allocated.
Well, the testcase could be a C-code, that mimicks the behaviour of allocating a larger chunks of mpi_wins.
What we forget here is, how remote memory access is implemented: e.g. on a single machine "parallel" machine it is shared memory and nearly every address is possible for remote access. But one clusters that might not be true depending on the mpi-library. The library may simply run out of address-space, because of paging limitation or the like. Just a guess.
That C-test could look like this (note I had to add .txt, because github does not know C-files :-)):
alloc_c_test.c.txt
This essentiall what your fortran program is doing. Maybe you can check with your mpi-library and shed some light, whether it is a mpi- or a coarray-issue.

I'm fairly confident there are no memory leaks in the problematic code (in coarray_icar) as we are never expecting to deallocate memory.  All arrays (and coarrays) are allocated once at the beginning of the simulation and are used throughout, however it is the initial allocation that is problematic. It looks like your c-code test is (deliberately?) leaking memory, can we simply make mem be [81] elements long and index it inside the for loop, then check the timing of each loop?  (it may be that leaking memory does not effect this problem so it doesn't matter, I just want to check.)
The concern with the fortran test case is not that the first allocate takes a long time, it is that the length of time required for an allocation increases with each successive allocation (though usually the very last allocation isn't the slowest).
If the test case you posted runs on 3600 images in <10minutes, would you consider that evidence that the MPI library is not the problem?
I'll see if I can try this C-code on cheyenne later today or tonight, note that I'll be on vacation for the following week, so if I don't get this done tonight, then it will be a little while before I can test it, but I will.

I think we may have a winner... so far that alloc_c_test.c has not completed after >30 minutes on 3600 processes!  I'm going to see if I can simplifiy it a little more with, e.g. a smaller chunksize, then pass this on to our computing group.  I'll report back if/when I figure anything else out.
I'll also see if I can get opencoarrays built with openmpi or mpich on cheyenne instead of the pre-built version that uses mpt (if they can't fix the "bug" in mpt).
Also curious what you mean by github does not know C-files
e.g. https://github.com/torvalds/linux/blob/master/kernel/reboot.c looks fine.

FYI, have tested this C-code on Cheyenne with alternate mpi libraries and achieved reasonable scaling characteristics, so I think we can pin this on mpt.  I've submitted a report to CISL, I'll update if I hear anything interesting, but we may be able to close this!

Ethan, I'm not shocked that there were issues in MPT. The FUN3D/NASA guys
had SGI engineers creating patches and installing hotfixes on Pleiades for
hosts of problems.
If you think this is definitive evidence that MPT is to blame (which
appears to be the case to me) then please close this issue. We can continue
discussing etc here even after it's closed.
Enjoy your vacation,
Best,
Zaak
On Sat, Aug 5, 2017 at 2:33 PM Ethan Gutmann<EMAIL_ADDRESS>wrote:

FYI, have tested this C-code on Cheyenne with alternate mpi libraries and
achieved reasonable scaling characteristics, so I think we can pin this on
mpt. I've submitted a report to CISL, I'll update if I hear anything
interesting, but we may be able to close this!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sourceryinstitute/OpenCoarrays/issues/420#issuecomment-320462505,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAREPBs56XS75xcaPOV6-o5qSLh8VBVgks5sVLV7gaJpZM4OhlQg
.

I'm pretty surprised, but I do consider this fairly definitive.  Closed.

Sorry for not responding earlier. I am German based (timezones and so on).
My intention with the memory leak was to simulate the small fortran test-code as closely as possible in C to expose issues of the opencoarrays handling. The intention was exactly what we learned: is it a mpt or a opencoarrays issue. With checking against mpich and openmpi you exactly figured what was the issue and the c-code was made exactly for this. Great job.
With " github does not know C-files" I meant, that when one attaches a file to one of these comments, the filename must not end on .c, but .txt, .pdf, .png or others.
Happy vacation.


Fix Workflow: Publish PyPI only on tag push

Context
Write now when we push to master a new commit, the workflow that publish the commit to PyPI fails because we publish the same tag since new tag is not created yet. So after this PR we will publish to PyPI only when a tag is created.

Overall LGTM, maybe we can also consider adding this in the pyproject.toml:

[tool.poetry-dynamic-versioning]
enable = true

As @hiohiohio suggested here.

@alessiocastrica @hiohiohio I investigated and there is also another file need to be dynamically updated init.py here
https://github.com/alpacahq/alpaca-py/blob/cccfb34b4c1f6bdc01807e7af7ee1fe606aac26f/alpaca/__init__.py#L1
So please let me merge this PR and come up with another one for dynamic versioning. What do you think?


 For native builds we can see some neat improvements. However, nothing that cannot be explained by sheer variance of inlining due to hot code path changes in UntypedValue. On the other side the Wasm builds are regression by roughly the same amount. Probably the same explanation holds true. Since Wasm targets are important for our use case we are not interested in regressions on that front even if it brings minor wins elsewhere.
The main problem with this experiment and its indifferent results is that this adds quite a lot of non-isolated unsafe code and therefore I declare this experiment as failed for now.
Closed.


Not working on Andriod Phone

I try to run the demo, from the demo page on my Android phone, the camera starts streaming but there is no image detection happening?

WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost
src.367e2190.js:269 Uncaught (in promise) Error: Failed to compile fragment shader.
at Object.s [as createFragmentShader] (src.367e2190.js:269)
at a.createProgram (src.367e2190.js:303)
at Object.n [as compileProgram] (src.367e2190.js:305)
at src.367e2190.js:339
at F.getAndSaveBinary (src.367e2190.js:339)
at F.compileAndRun (src.367e2190.js:339)
at F.gather (src.367e2190.js:339)
at r.ENV.engine.runKernel.x (src.367e2190.js:200)
at c.runKernel (src.367e2190.js:262)
at h.gather (src.367e2190.js:200)

Couldn't parse line number in error:
src.367e2190.js:269
precision highp float;
precision highp int;
varying vec2 resultUV;
const vec2 halfCR = vec2(0.5, 0.5);
bool isNaN(float val) {
float v1 = val * val;
float v2 = val * val;
return v1 == v2 ? false : true;
}
bool hasNaN(vec4 values) {
vec4 v1 = values * values;
vec4 v2 = values * values;
return any(notEqual(v1, v2));
}
float getNaN(vec4 values) {
return dot(vec4(1), values);
}
int round(float value) {
return int(floor(value + 0.5));
}
int imod(int x, int y) {
return x - y * (x / y);
}
const vec2 randomConst = vec2(
23.14069263277926, // e^pi (Gelfond's constant)
2.665144142690225 // 2^sqrt(2) (Gelfond–Schneider constant)
);
float random(float seed) {
return fract(cos(dot(resultUV * seed, randomConst)) * 12345.6789);
}
vec2 UVfrom1D(int texNumR, int texNumC, int index) {
int texR = index / texNumC;
int texC = index - texR * texNumC;
return (vec2(texC, texR) + halfCR) / vec2(texNumC, texNumR);
}
vec2 UVfrom2D(int texNumR, int texNumC, int numC, int row, int col) {
int index = row * numC + col;
int texR = index / texNumC;
int texC = index - texR * texNumC;
return (vec2(texC, texR) + halfCR) / vec2(texNumC, texNumR);
}
vec2 UVfrom3D(int texNumR, int texNumC, int stride0,
int stride1, int row, int col, int depth) {
// Explicitly use integer operations as dot() only works on floats.
int index = row * stride0 + col * stride1 + depth;
int texR = index / texNumC;
int texC = index - texR * texNumC;
return (vec2(texC, texR) + halfCR) / vec2(texNumC, texNumR);
}
vec2 UVfrom4D(int texNumR, int texNumC, int stride0,
int stride1, int stride2, int row, int col, int depth,
int depth2) {
// Explicitly use integer operations as dot() only works on floats.
int index = row * stride0 + col * stride1 + depth * stride2 + depth2;
int texR = index / texNumC;
int texC = index - texR * texNumC;
return (vec2(texC, texR) + halfCR) / vec2(texNumC, texNumR);
}
float sampleTexture(sampler2D textureSampler, vec2 uv) {
return texture2D(textureSampler, uv).r;
}
void setOutput(float val) {
gl_FragColor = vec4(val, 0, 0, 0);
}
uniform sampler2D A;
uniform sampler2D indices;
  ivec2 getOutputCoords() {
    return ivec2(resultUV.yx * vec2(55, 4));
  }

float getAFlat(int index) {
  vec2 uv = UVfrom1D(845, 4, index);
  return sampleTexture(A, uv);
}

float getA(int row, int col) {
  vec2 uv = (vec2(col, row) + halfCR) / vec2(4.0, 845.0);
  return sampleTexture(A, uv);
}

  float getIndicesFlat(int index) {
    vec2 uv = vec2(0.5, (float(index) + 0.5) / 55.0);
    return sampleTexture(indices, uv);
  }

float getIndices(int index) {
  return getIndicesFlat(index);
}

  void main() {
    ivec2 resRC = getOutputCoords();
    setOutput(getA(int(getIndices(resRC.x)),resRC.y));
  }

This sounds like an upstream tf.js issue, can you try running another demo perhaps Posenet? https://storage.googleapis.com/tfjs-models/demos/posenet/camera.html

I have the same  issue here, can anyone help? I've setup my own AWS instance for this demo, it works on computers but it doesn't work on cellphones. The code is the same, just different device, weird.

@owenljn I've tested on iPads and iPhones and it'll work for both. I don't have an android device to test on though. Can you try the Posenet demo above and let me know if it'll work on the device you're trying it out on? The error reported in this issue seems to stem from TF.js, though I can't be completely sure.

@MikeShi42 Sorry actually it's not something wrong with the code, in fact, something went wrong when I try to host your code in an AWS instance, the code runs perfectly on conventional computers, but it doesn't do detection on my cellphone. And I tried both of your live demos on my cellphone and I can verify they do work.

@owenljn could it be that you're using latest tfjs? There seems to be an issue of version 0.11 and above. If you're running 0.10 it should work
See: https://github.com/ModelDepot/tfjs-yolo-tiny/issues/10


Display how much funds receiver gets via online and offline transactions

For online transactions add info how much funds will get the recipient (after fees). https://zpl.io/aB3PdY0

For offline transactions add info how much funds will get the recipient (after fees). https://zpl.io/2pqg7rM

For Max anonymity transaction add info how much funds will get the recipient (after fees). https://zpl.io/blnekJX

For Public offline transaction add info how much funds will get the recipient (after fees). https://zpl.io/awldgBn

Please add separator lines after "transaction type" and before "recipient will receive" https://zpl.io/awldgBn

irrelevant as receiver will get all amount. fee is taken from sender's


[3.x] Fix touch events in WebXR with an "immersive-ar" session

Fixes https://github.com/godotengine/godot/issues/85482
This was a problem with calculating the position of touch events. It looks like the bug was introduced in PR https://github.com/godotengine/godot/pull/56819, and was fixed for Godot 4 as part of PR https://github.com/godotengine/godot/pull/68870, but the fix never got backported to Godot 3.
This PR is backporting the fix!

I have built the fixed template and test with my project. It's works! Touch inputs now are working properly.

Great! Thanks for testing :-)

Thanks!

Cherry-picked for 3.5.4.


Scroll shadow example

First off, thanks for an awesome plugin! It is by far the best implementation of custom scrollbars I've seen, well done!
My question: Is there a way to listen to the scrollable area dimensions changing, typically because of a browser window resize?
Use case: I'm implementing scroll shadow, like https://twitter.com/hakimel/status/685057803396988928, and currently I'm calculating viewport dimensions onScroll. When I resize the browser window, the shadow placement doesn't update, since onScroll doesn't fire.
My fallback plan is to listen to window.onresize, but figured I'd see if this is planned/already supported.
Thanks again!

I made a new example for that: Code & Demo.
Hope it helps!

Thank you, I figured that'd be the way :) Feel free to close this issue!


MSInheritanceAttr causes multiple issues with clangd

The implicit MSInheritanceAttr causes quite some issues, including incorrect hover results and failure on showing type hierarchy.
Code to reproduce with:
struct C
{
   void f();
};

using T = decltype(&C::f);

code on compiler-explorer with ast-dump. Put this code inside a namespace if you want to check with the AST functionality in VS Code.
This results in the following ast-dump:
TranslationUnitDecl 0x2755c318f90 <<invalid sloc>> <invalid sloc>
|-CXXRecordDecl 0x2755c3197f0 <<invalid sloc>> <invalid sloc> implicit struct _GUID
| `-TypeVisibilityAttr 0x2755c3198a0 <<invalid sloc>> Implicit Default
|-TypedefDecl 0x2755c319920 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
| `-BuiltinType 0x2755c319560 '__int128'
|-TypedefDecl 0x2755c319998 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
| `-BuiltinType 0x2755c319580 'unsigned __int128'
|-TypedefDecl 0x2755c319d80 <<invalid sloc>> <invalid sloc> implicit __NSConstantString '__NSConstantString_tag'
| `-RecordType 0x2755c319aa0 '__NSConstantString_tag'
|   `-CXXRecord 0x2755c3199f8 '__NSConstantString_tag'
|-CXXRecordDecl 0x2755c319de0 <<invalid sloc>> <invalid sloc> implicit class type_info
| `-TypeVisibilityAttr 0x2755c319eb0 <<invalid sloc>> Implicit Default
|-TypedefDecl 0x2755c319f30 <<invalid sloc>> <invalid sloc> implicit size_t 'unsigned long long'
| `-BuiltinType 0x2755c319180 'unsigned long long'
|-TypedefDecl 0x2755c4f1258 <<invalid sloc>> <invalid sloc> implicit __builtin_ms_va_list 'char *'
| `-PointerType 0x2755c4f1210 'char *'
|   `-BuiltinType 0x2755c319040 'char'
|-TypedefDecl 0x2755c4f12d0 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list 'char *'
| `-PointerType 0x2755c4f1210 'char *'
|   `-BuiltinType 0x2755c319040 'char'
|-CXXRecordDecl 0x2755c4f1330 <e:\\test.h:1:1, line:4:1> line:1:8 struct C definition
| |-DefinitionData pass_in_registers empty aggregate standard_layout trivially_copyable pod trivial literal has_constexpr_non_copy_move_ctor can_const_default_init
| | |-DefaultConstructor exists trivial constexpr needs_implicit defaulted_is_constexpr
| | |-CopyConstructor simple trivial has_const_param needs_implicit implicit_has_const_param
| | |-MoveConstructor exists simple trivial needs_implicit
| | |-CopyAssignment simple trivial has_const_param needs_implicit implicit_has_const_param
| | |-MoveAssignment exists simple trivial needs_implicit
| | `-Destructor simple irrelevant trivial constexpr needs_implicit
| |-MSInheritanceAttr 0x2755c4f1730 <col:1, line:4:1> Implicit __single_inheritance BestCase
| |-CXXRecordDecl 0x2755c4f1470 <line:1:1, col:8> col:8 implicit struct C
| `-CXXMethodDecl 0x2755c4f15a0 <line:3:4, col:11> col:9 referenced f 'void ()'
`-TypeAliasDecl 0x2755c4f17f0 <line:6:1, col:25> col:7 T 'decltype(&C::f)':'void (C::*)()'
  `-DecltypeType 0x2755c4f17b0 'decltype(&C::f)' sugar
    |-UnaryOperator 0x2755c4f1790 <col:20, col:24> 'void (C::*)()' prefix '&' cannot overflow
    | `-DeclRefExpr 0x2755c4f16c8 <col:21, col:24> 'void ()' CXXMethod 0x2755c4f15a0 'f' 'void ()' non_odr_use_unevaluated
    `-MemberPointerType 0x2755c4f1700 'void (C::*)()'
      |-RecordType 0x2755c4f13e0 'C'
      | `-CXXRecord 0x2755c4f1330 'C'
      `-FunctionProtoType 0x2755c4f1550 'void ()' cdecl
        `-BuiltinType 0x2755c319000 'void'

If I understand this dump correctly, the MSInheritanceAttr 0x2755c4f1730 <col:1, line:4:1> implies that this starts at column 1 of the line of thes tart of the parent (CXXRecordDecl 0x2755c4f1330 <e:\\test.h:1:1, line:4:1>) and goes until line 4 column 1. This is basically the same as the CXXRecordDecl.
If you hover over the class name, you get the documentation of __single_inheritance, a value of MSInheritanceAttr, which is not expected.
When requesting the type hierarchy, it fails to locate the node for which to start from.
Removing the decltype also removes the MSInheritanceAttr.
System information
Output of clangd --version: 16.0.6 (although AST dump on compiler explorer uses trunk and shows the same)
Editor/LSP plugin: VS Code with clangd v0.1.29
Operating system: Windows x64

I assume this attribute is Windows-specific.
Is it created by default on Windows builds, or only if the compile flags include -fms-compatibility -fms-extensions as in the Compiler Explorer link?

The flags are implicitly applied in Windows executables. I'm a 100% sure they are implicitly added when using the clang-cl driver, though, the docs seem to indicate that even clang++ would add this when used on windows.
From https://clang.llvm.org/docs/UsersManual.html#microsoft-extensions

clang has support for many extensions from Microsoft Visual C++. To enable these extensions, use the -fms-extensions command-line option. This is the default for Windows targets. Clang does not implement every pragma or declspec provided by MSVC, but the popular ones, such as __declspec(dllexport) and #pragma comment(lib) are well supported.
clang has a -fms-compatibility flag that makes clang accept enough invalid C++ to be able to parse most Microsoft headers. For example, it allows unqualified lookup of dependent base class members, which is a common compatibility issue with clang. This flag is enabled by default for Windows targets.

The flags where explicitly added on compiler explorer to be able to reproduce on Linux as well.

While I haven't debugged it to verify, given that the MSInheritanceAttr is a descendant node that takes up the entire source range of its ancestor CXXRecordDecl node, a likely diagnosis for various clangd features being broken is that SelectionTree (which is clangd's implementation of a "select the AST node at this source location" algorithm) selects the MSInheritanceAttr in many/all cases where we'd want the CXXRecordDecl to be selected instead.
Assuming there is a good reason for the AST modeling things this way, a fix at the clangd level could then be to teach SelectionTree to ignore the MSInheritanceAttr.

To be fair, I don't know if it was done for a good reason. It seems to be added for some edge case (member function pointers) and was added 9 years ago. Maybe @rnk knows some context as he added the attribute.


Fix the back arrow on intial Load

Details

Follow-up PR of https://github.com/Expensify/App/pull/7801. I reverted this part somehow from the previous PR before merging. This is the first part of the proposal https://github.com/Expensify/App/issues/7091#issuecomment-1042013769.
Fixed Issues

$ https://github.com/Expensify/App/issues/7091
Tests

Open the app on a web browser.
open any chat.
resize the browser to mobile view.
Click the back arrow to open the LHN.

[x] Verify that no errors appear in the JS console

Contributor (PR Author) Checklist

[x] I verified the PR has a small number of commits behind main
[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I clearly indicated the environment tests should be run in (Staging vs Production)

[x] I wrote testing steps that cover success & fail scenarios (if applicable)
[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests & verify they passed on:

[x] iOS / native
[x] Android / native
[x] iOS / Safari
[x] Android / Chrome
[x] MacOS / Chrome
[x] MacOS / Desktop

[x] I verified there are no console errors related to changes in this PR
[x] I followed proper code patterns (see Reviewing the code)

[x] I added comments when the code was not self explanatory
[ ] I put all copy / text shown in the product in all src/languages/* files (if applicable)
[ ] I followed proper naming convention for platform-specific files (if applicable)
[x] I followed style guidelines (in Styling.md) for all style edits I made
[x] I followed the JSDocs style guidelines (in STYLE.md)

[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I corroborated the UI performance was not affected (the performance is the same than main branch)
[x] If I created a new component I verified that a similar component doesn't exist in the codebase

PR Reviewer Checklist

[ ] I verified the PR has a small number of commits behind main
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the testing environment is mentioned in the test steps

[ ] I verified testing steps cover success & fail scenarios (if applicable)
[x] I checked that screenshots or videos are included for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] iOS / native
[ ] Android / native
[ ] iOS / Safari
[ ] Android / Chrome
[ ] MacOS / Chrome
[ ] MacOS / Desktop

[ ] I verified there are no console errors related to changes in this PR
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified comments were added when the code was not self explanatory
[x] I verified any copy / text shown in the product was added in all src/languages/* files (if applicable)
[x] I verified proper naming convention for platform-specific files was followed (if applicable)
[x] I verified style guidelines were followed
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components are not impacted by changes in this PR (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified the UI performance was not affected (the performance is the same than main branch)
[x] If a new component is created I verified that a similar component doesn't exist in the codebase

QA Steps

Open the app on a web browser. (do not navigate to any chat).
resize the browser to mobile view.
Click the back arrow to open the LHN.

Note: Test this PR with https://github.com/Expensify/App/pull/7801

[ ] Verify that no errors appear in the JS console

Screenshots

Web

https://user-images.githubusercontent.com/24370807/156860526-4ed2df04-8ee5-4f6e-9697-13d94f4fba30.mp4
Mobile Web

https://user-images.githubusercontent.com/24370807/156914691-4934f665-d8ba-4b0f-a7fb-20ad15b16c6c.mp4
Desktop

https://user-images.githubusercontent.com/24370807/156914533-708da872-221e-4455-83e6-e134c1dec98b.mp4
iOS

https://user-images.githubusercontent.com/24370807/156914555-8ef95187-64ac-4810-a7a7-2a19f07b4d24.mp4
https://user-images.githubusercontent.com/24370807/155207955-cea8345f-c39b-4c3b-9c49-0e2f17c836fe.mp4
Android

https://user-images.githubusercontent.com/24370807/156912073-d3ca0791-e714-47c5-a3fa-e96bed283c33.mp4

@thienlnam This PR needs to go before awaiting PROD deploy so that it can be tested on next QA cycle with the main PR.

@parasharrajat Good point, I'll prioritize this review then.  Taking a look now

Also ?? 😕

I don't know about this. Maybe you have write access so you see this. the commit is actually from the forked repo and we are making a PR from fork to this repo. If you are seeing this first time, it is possible that Github added this feature recently.

Yeah, I used the old branch from the previous PR so maybe it is picking up that branch again as that was not deleted from newDot repo. it is up-to-date with main.

Verified testing, just the small comments above left!

Changes done.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by @thienlnam in version: 1.1.42-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Fix self-find query

Issue Addressed
Resolves #108
Proposed Changes
Skip incorporate the discovered peer into closest_peers if the peer is the target in order to avoid self-find query.

Hey @ackintosh
Thank you very much for this :pray:
I had a look through. The solution you propose here, means that we exclude the actual target from our queries. i.e if we search for a specific target and it is known, the results will not include that we found the actual target.
I think the behaviour that we want is that if a node claims to know the actual target, we should then attempt to contact that target. If it is reachable, then we add it to the results.
#117 is an attempt at solving this, where we still contact the target, and if its contactable return it with the results of the query.
Could you test this PR and see if it resolves your issues also?
If you agree with me, perhaps we close this PR in favour of #117.
Thanks again for finding this bug.

I think the behaviour that we want is that if a node claims to know the actual target, we should then attempt to contact that target. If it is reachable, then we add it to the results.

I understand the correct behaviour, thank you @AgeManning !
I'm testing #117 on this test plan, will let  you know once the test has been finished.

Closing this PR in favour of #117.

I've finished the test, and have left the comment.


Fix OCSP_basic_verify() cert chain construction in case bs->certs is NULL

Now the certs arg is not any more neglected when building the signer cert chain.
Added case to test/recipes/80-test_ocsp.t proving fix for 3-level CA hierarchy.
See also http://rt.openssl.org/Ticket/Display.html?id=4620
It seems that OCSP_basic_verify(bs, certs, st, flags) unfortunately is
not documented, but from its code it becomes clear that the "certs"
parameter is meant to be a set of untrusted certificates, which is first
used (together with bs->certs) to determine the signer cert of the OCSP
response "bs" and then is partly(!) used to construct the chain of certs
towards a trusted (root) cert in the store passed in the "st" parameter.
As long as the OCSP response pointed to by "bs" includes a non-NULL
bs->certs field, OCSP_basic_verify() takes the union of any certs in the
"certs" parameter and in bs->certs as untrusted certs for chain
construction, but if bs->certs is NULL, i.e. when the OCSP responder did
not include any certs its response, for some reason OCSP_basic_verify()
does not take "certs" but bs->certs, which corresponds to the empty set.
This fix takes "certs" as the set of untrusted certs in case bs->certs is NULL.
Checklist

[x] tests are added or updated

The automatic Travis CI build given above failed due to an unrelated memory leak in test/sslapitest.c
See https://github.com/openssl/openssl/issues/4127 for more detail.

CLA on file, removing tag.

This is unreview-able. It looks like partial re-base. Unfortunately I don't have an advice how to do it right, as it always worked for me as I expected...

Oops, I had made a mistake when rebasing, such that my commit went somewhat below the head.
Thanks for the comment. Now it should be fine.

For some reason, the cla-check failure tag appears still present, although Rich has cleared the need-cla label.
Anything (else) needed for reviewing this pull request?
The actual code change of the crypto library for the fix is tiny: 2 new lines.
Maybe some explanations on the extension of the OCSP tests are due:

The new test case "NON-DELEGATED; 3-level CA hierarchy" has been designed to fail without the fix and to pass after it has been applied, demonstrating the effectiveness of the fix.
I tried to stick with the available test data and scripts as much as possible:
I was glad to find an existing OCSP test response (ND1.ors) with empty certs field and a corresponding signer cert (ND1_Issuer_ICA.pem) for which I could extend the cert chain to contain three elements (in this case by adding a cross-certification cert, resulting in the new file ND1_Issuer_ICA-Cross.pem, and a new root cert file ND1_Cross_Root.pem).
Unfortunately, the existing Perl subroutine test_ocsp() did not distinguish between trusted and untrusted certs and used the CAfile parameter for both. I could have added a new parameter for untrusted certs to all test cases, but I did not want to touch them (and clean up the cert arguments for them accordingly). Instead I added a case distinction allowing for an optional untrusted parameter, so far used only by the new test case.

@mattcaswell thanks for the first review.
I've simplified test_ocsp as you suggested.

This applies cleanly to master and 1.1.0. I suspect the issue is present in 1.0.2 as well, but this PR won't apply cleanly to that branch due to the test. It may be appropriate to have a second PR which just makes the main change without the test for 1.0.2.

I haven't tried with 1.0.2.
Could it make sense to do a version distinction inside the test?
Otherwise, how to do a PR with just the fix specifically for 1.0.2?

The test uses the new test framework which isn't available in 1.0.2, so couldn't be back ported without additional work. There may be an equivalent test in the old 1.0.2 way of doing things that you could amend - I've not checked - but it's probably not worth the effort. To do a PR to fix specifically 1.0.2, just create a new branch based on OpenSSL_1_0_2-stable and make the relevant change. Then create the PR in the normal way, but instead of master (the default) select OpenSSL_1_0_2-stable as the target branch.

Re-formulated the certs check as requested.
Did not do the change regarding 'shift' since AFAICS this would work for the last parameter only.


Copy metadata to avoid corruption

Makes a copy of the metadata to process log event in order to avoid corruption.
Fixes: #308

:broken_heart: Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-09-15T11:01:02.000+0000

Duration: 7 min 8 sec

Steps errors
Expand to view the steps failures

make build

Took 0 min 3 sec . View more details here
Description: make build

make build

Took 0 min 2 sec . View more details here
Description: make build

:robot: GitHub comments
To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)


Direct DB Connection problem+HTTP API Connection problem

request failed, status: 403 Forbidden after press Save&Test
in logs i saw this text:
t=2020-12-22T07:53:32+0000 lvl=eror msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=POST path=/api/datasources/11/resources/zabbix-api status=500 remote_addr=my_ip time_ms=261 size=83 referer=https://mygrafanadesk.com/datasources/edit/11/
t=2020-12-22T07:53:32+0000 lvl=eror msg="Zabbix API request error" logger=plugins.backend pluginId=alexanderzobnin-zabbix-datasource error="request failed, status: 403 Forbidden"
Zabbix version is 4.2.8, Grafana version is v7.3.6, plugin version 4.0.2., i tried to reinstall the plugin, same error

zgraf is Zabbix Super Administrator, firewall is set correctly on both sides and nothing
P.S. i use my grafana in the K8S and in this log i see this
t=2020-12-22T14:36:04+0000 lvl=eror msg="Zabbix API request error" logger=plugins.backend pluginId=alexanderzobnin-zabbix-datasource error="request failed, status: 403 Forbidden"
t=2020-12-22T14:36:04+0000 lvl=eror msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=POST path=/api/datasources/11/resources/zabbix-api status=500 remote_addr=my_ip time_ms=333 size=83 referer=https://grafana.mydomain.com/datasources/edit/11/

what is mean "Forbidden", which side told this message ?

Forbidden means you don't have access to the URL. Probably firewall misconfiguration or additional auth required (basic auth or whatever).


Implement StopOnCtrlC flag

If set, the start trace waits for Ctrl+C instead of termination. When Ctrl+C is pressed, run the stop trace method.

This is blocking, as far as I know, so no chain file trimming is a down-side.


Consolidated verilator copy-tree actions

This also avoids reliance on the behavior of mkdir and cp which may differ on different systems.

Nice! Thanks for the contribution


Training speed

It takes Quadro GV100  5 hours for 1 epoch.    Anybody trained the model on RTX3090? I failed to compile SpConv lib on 3090.

Found it. #32

Did you solve your issue on compiling with 3090? @chenpengxin

Did you solve your issue on compiling with 3090? @chenpengxin

I think the compile errors are caused by high-version pytorch. And the compile errors may be as follow 2 parts (what I met):

torch::jit::register... in src/spconv/all.cc: In high-version pytorch, the register func is not in the jit namespace, or the jit namespace is deprecated. You just need to modify torch::jit::register as torch::register. (remove the jit namespace).
Use the ninja to build, you need to modify BuildExtension to BuildExtension.with_options(use_ninja=False) in setup.py.


Scaffolding tool should not be shipped to nuget...

Per a discussion today with @DamianEdwards and @rustd, this tool would only live in WTE, and not be uploaded to nuget.org.
We need to change this: https://github.com/aspnet/Coherence/blob/dev/packages/packages.csv#L10, but also make it possible to sign it.
cc @Eilon @pranavkm

@sayedihashimi FYI.

Is there any change to make in this repo? Or is the only change in packages.csv and setup? Does WTE itself need to change to adapt to this change?

cc @barrytang as well.

@Eilon I believe there's no change in this repo, though we'll need to figure out how to sign the tool...
Also, I assume M.E.CodeGeneration.* (these lines) will also need to be changed.

I talked to @danroth27 and I think we need to ship the scaffolding package on nuget otherwise people opening a project created in VS won't be able to restore it on a machine without VS.

Tagging aspnet/Release/issues/4

TFS Bug#232589

From previous discussions we do not have the ability to remove this package. If we wanted to do that we would need some proposed solution from the runtime team.
I don't personally see any value in doing this and I'd rather the team be focused on important deliverables. If the runtime team is still interested in this please come back with a specific technical proposal and then we can evaluate it. Closing for now.


[BD-46] feat: added CSS custom media query transformer

Description

Ensure we can generate @custom-media from design tokens with style-dictionary.
Ensure we add support for postcss-custom-media to @edx/frontend-build's default Webpack configs for all MFEs.

Issue: https://github.com/openedx/paragon/issues/2015
Relative link: Frontend build
Merge Checklist

[ ] If your update includes visual changes, have they been reviewed by a designer? Send them a link to the Netlify deploy preview, if applicable.
[ ] Does your change adhere to the documented style conventions?
[ ] Do any prop types have missing descriptions in the Props API tables in the documentation site (check deploy preview)?
[ ] Were your changes tested using all available themes (see theme switcher in the header of the deploy preview, under the "Settings" icon)?
[x] Were your changes tested in the example app?
[ ] Is there adequate test coverage for your changes?
[ ] Consider whether this change needs to reviewed/QA'ed for accessibility (a11y). If so, please add wittjeff and adamstankiewicz as reviewers on this PR.

Post-merge Checklist

[ ] Verify your changes were released to NPM at the expected version.
[ ] If you'd like, share your contribution in #show-and-tell.
[ ] 🎉 🙌 Celebrate! Thanks for your contribution.

Thanks for the pull request, @PKulkoRaccoonGang!
When this pull request is ready, tag your edX technical lead.

@PKulkoRaccoonGang 🎉 Your pull request was merged!  Please take a moment  to answer a two question survey so we can improve your experience in the future.

Also merged/released the associated @edx/frontend-build PR here at v12.6.0: https://github.com/openedx/frontend-build/releases/tag/v12.6.0

:tada: This PR is included in version 21.0.0-alpha.14 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 22.0.0-alpha.1 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Enable DescribeLaunchConfigurations

core/controleplane/config/templates
When scaling from 0 -> n nodes we require DescribeAutoscalingGroups in order to scale up based on docs: https://github.com/kubernetes/autoscaler/blob/5302c37d6d6e89a2eeac0c74b4afe6caadfa3340/cluster-autoscaler/cloudprovider/aws/README.md#scaling-a-node-group-to-0
fixes: https://github.com/kubernetes-incubator/kube-aws/issues/1169

Codecov Report

Merging #1181 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1181   +/-   ##
=======================================
  Coverage   36.22%   36.22%
=======================================
  Files          63       63
  Lines        3821     3821
=======================================
  Hits         1384     1384
  Misses       2223     2223
  Partials      214      214

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8bbf89d...437c121. Read the comment docs.

LGTM. Thank you very much for your contribution!


Orand Pull Request

Project Abstract
Orand is an Elliptic Curve Verifiable Random Function (ECVRF: https://datatracker.ietf.org/doc/html/draft-irtf-cfrg-vrf-05) as a service that offers high performance, zero latency and 30 times lower cost than current providers. Orand offers on-chain verification and pseudorandom for projects that seek integrity and transparency in their features with a wide variety of case studies from card games, MMORPG to NFT collectibles
Verifiable randomness: We’re using Verifiable Random Function (VRF) to generate randomness the process described here https://www.rfc-editor.org/rfc/rfc6979 . Curve secp256k1 was used to minimizing verification cost for smart contract on Ethereum.
Dispersity: A distributed system with many participants/nodes will join to generate randomness by using Multi Party Computation (MPC).
Unpredictability: A VRF will perform with the input is previous randomness and it’s also require half of participants to participate in MPC.
High throughput: Game/application server could request randomness from the Orochi đRNG system. The result will be provided as soon as half of participants participate in the MPC.
Fault Proof: If the game/application server tries to delay the feeding process to manipulate the result, a fault proof will be committed so sue the game server.
Multi-chain: All EVM compatible blockchains can be supported.
Grant level

[ ] Level 1:  Up to $10,000, 2 approvals
[ ] Level 2:  Up to $30,000, 3 approvals
[x] Level 3:  Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

[x] The application template has been copied and aptly renamed (project_name.md).
[x] I have read the application guidelines.
[x] Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (USDT) address in the application).
[x] The software delivered for this grant will be released under an open-source license specified in the application.
[x] The initial PR contains only one commit (squash and force-push if needed).
[x] The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
[x] I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @hubertnguyen:matrix.org (change the homeserver if you use a different one)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 2 committers have signed the CLA.:x: semuelle:x: hubertnguyen88You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for the application. One quick initial feedback: Please only add one file to our repo as part of the PR. So only the one that you add to the application folder.

Got it, could you please help proceed further, are you still reviewing it?

Thanks for the update. But you are still adding two files for the repo (RFP and application). Could you change this?
Hi, i have deleted in RFP folder. I intend to fork application to Orochi's report and make a pull request for application from there, instead from my personal git account. Is it ok?

Thanks for the update. But you are still adding two files for the repo (RFP and application). Could you change this?

Hi, i have deleted in RFP folder. I intend to fork application to Orochi's report and make a pull request for application from there, instead from my personal git account. Is it ok?

Thanks. The application now contains only one file. So this is correct. Regarding the application itself, could you add more technical details, like the programming language to the milestones as well as the default deliverables of the template: https://github.com/w3f/Grants-Program/blob/master/applications/application-template.md#milestone-1-example--basic-functionality See:

“❗ The default deliverables 0a-0d below are mandatory for all milestones, and deliverable 0e at least for the last one.”

Thank you, we will update the milestone information. And will make a pull request from Orochi's repo instead my personal account.


[android] added tabLayout, removed TitlePageIndicator

well, TabLayout is 100% new for me, looking at it

it's all in @marcelinkaaa googling powers.
Before this we had replaced it with PagerTitleStrip but it sucked.

nice, I prefer the animation (with color fading) from apollo, but I guess you are all ok with how it works now. If you think UX is ok, I'm fine with it. I also see that you just fixed the compilation issue.
LGTM

@marcelinkaaa :clap:

I'm fine with the current animation, and like to have an option of horizontal scroll to get to the last tab faster. It's also so much easier to change and maintain, at least for me. :)

I don't want to merge at this moment because I don't know if @gubatron is working on it (last minute change)

No merging yet, we are trying to see if we can more easily apply tablayout in transfers, browse, etc.

yeah don't merge, there's more work coming with this TabLayout stuff, already got the selection styles working and some of the interaction on Search

.
I believe quite a lot of code will be gone once this is functionally on par with the uber customized radiogroup, my guess is that the UI will be a lot faster too, a lot less code happening

@marcelinkaaa functionality has been plugged, once you've gotten down how to make icons look correctly (hopefully with just styles, if not we'll have to play with the "intrinsic" bounds and what not programatically), then I'll do the same for My Files, and once that's done, it's going to be major cleanup time.
I've just put the code side by side for now.

just asking, what's the status of this?

@aldenml  Angel is out for now, and I will need help with connecting the tabs to the corresponding views and creating a custom tab view for horizontal search tabs.
I am finishing redoing the icons - need to adjust sizes, etc. The old ones can be deleted after that.

I'll start connecting this over the weekend, I'm almost done with Image Fragment viewer, FrostWire is going to look and feel kick ass.

just rebased this branch with master without pulling the latest changes.
@marcelinkaaa please do not pull this branch, instead push force whatever you have on your computer.

@gubatron done.

yay, thanks!

closing this, renaming branch to tablayout-refactor


wrong result from TvgSaver + PictureTvg

the result from the examples: TvgSaver + PictureTvg sould look like this:

and it looks now a bit different:

It looks composition breaks the memory chunks....


Template files, minimum and complete examples.

Description
This PR adds minimum and complete examples to the module. It also adds standard template files.
Features/Fixes/Patches/Changes list:
Please check the CHANGELOG.md
Checklists:
PR Owners Checklist:

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?
[x] Have you checked there aren't open Pull Requests of upstream or parallel code that will cause conflicts?
[x] Have you updated the CHANGELOG.md?
[x] Have you updated the README.md(s)?
[x] OWNER: Does your submission pass?

[x] Pre-commit (attach logs/print screen to this PR)
[x] Checkov/terrascan (attach logs/print screen to this PR)
[x] Terraform resources examples build/destroy successfully with Terraform 0.14.11 minimum?

[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you written new tests for your core changes, as applicable?
[x] Have you successfully run tests with your changes locally?

PR Reviewers Checklists?

[ ] Are all your comments/changes resolved?
[ ] Are you happy with the OWNER checklists and additional information provided?
[ ] Does your review tests pass?

[ ] Terraform resources examples build/destroy successfully with Terraform 0.14.11 minimum?

checkov_logs.txt
Checkov logs


unixPB: macOS install thunderbird, blackhole and PDFwrite for JCK

Checklist

[x] commit message has one of the standard prefixes
[ ] FAQ.md updated if appropriate
[ ] other documentation is changed or added (if applicable)
[ ] playbook changes run through VPC or QPC (if you have access)
[ ] for inventory.yml changes, bastillion/nagios/jenkins updated accordingly

Also removed xquartz which is no longer needed

@sxa updated PTAL


Update NPSP Settings to include Matched Donor in top header

Critical Changes
Changes
Issues Closed
New Metadata
Deleted Metadata

@sam-knox - this is the item we spoke about in the standup today that will require screen shot updates.

@jjbennett - Jenny, what branch can I use in a scratch org to see the new settings page?

@sam-knox You can use this branch to see that - feature/jen-settings-roles

@force2b - Thanks Mike. Still learning how to navigate these PR/branch thingys.