API request: RenderTarget functions

@asny: related to discussion #93
I've implemented a custom multipass render mode (averaging pixel colors for overlapping photos) using three-d's RenderTarget and ImageEffect structs. However, I had to temporarily extend three-d's API locally.
RenderTarget uses copy and copy_to_screen (and related _color and _depth functions) to render its content to another RenderTarget or to the screen display buffer, respectively. They do this with

an ImageEffect to run a copy fragment shader on every pixel
a RenderStates struct

To implement my custom render mode, I made more general variants of the color-only versions of these functions, which I tentatively called apply_effect_color and apply_effect_to_screen_color. These differ from the copy functions only in that the ImageEffect and RenderStates are provided as parameters, instead of being set inside the function.
pub fn apply_effect_color(&self, other: &Self, effect: &ImageEffect, viewport: Viewport) -> Result<(), Error>
pub fn apply_effect_to_screen_color(&self, effect: &ImageEffect, render_states: &RenderStates, viewport: Viewport) -> Result<(), Error>

My request is to add RenderTarget functions that expose this functionality: however, we can probably do better than the interface I used here. I'm not sure if this is consistent with the API's style, so feel free to ignore or modify this if it doesn't match:
All of the RenderTarget::copy* functions could be expressed as one function, like:
pub fn apply_effect(&self, buffers: &Buffers, destination: &Destination, image_effect: &ImageEffect, render_states: &RenderStates, viewport: Viewport) -> Result<(), Error>
with something like
pub enum Buffers {
	Color,
	Depth,
	ColorAndDepth,
}
pub enum Destination {
	Screen,
	Other(&RenderTarget),
}

and possibly enums to switch between default and custom values for the ImageEffect and RenderState, if non-awkward names can be found.
This could be used to reimplement the existing functions, as well.

Thanks for some input on this. I had a lot of iterations on this and ended up not needing it so I really don't know if it is the best API. Also I realized that rendering with a copy shader is faster than using glBlit which is just weird but the API was made with blit in mind so might be possible to simplify now. I will look more into it as soon as I have time.

 So you suggest a function like this:
pub fn apply_effect(&self, other: &Self, image_effect: &ImageEffect, render_states: &RenderStates, viewport: Viewport) -> Result<(), Error> {
    other.write(&ClearState::none(), || {
        effect.apply(render_states, viewport)?;
        Ok(())
    })?;
    Ok(())
}

Why not just insert that code into your project? I made the ImageEffect struct for easy rendering of a full screen effect, so you can use that and combine with the write function of whatever render target you want to render to.
I think the Buffers and Destination input parameters are a great idea to avoid duplicate code, I will do that as soon as possible 👍

I can almost make something like that work, but only almost. The reason I modified three-d's RenderTarget code is because its color_texture and context members are private and have no public accessors.
One of my temporary RenderTarget extension methods looks like this (obviously copied from RenderTarget::copy_color_to_screen):
    pub fn apply_effect_to_screen_color(&self, effect: &ImageEffect, render_states: &RenderStates, viewport: Viewport) -> Result<(), Error> {
        if self.color_texture.is_none() {
            Err(Error::FailedToCopyFromRenderTarget {
                message: "Cannot copy color when the render target does not have a color texture."
                    .to_owned(),
            })?;
        }
        Screen::write(&self.context, &ClearState::none(), || {
            effect.use_texture(self.color_texture.unwrap(), "colorMap")?;
            effect.apply(*render_states, viewport)?;
            Ok(())
        })?;
        Ok(())
    }

In order to use the current RenderTarget's color_texture as an input to the custom ImageEffect's shader, I need access to it to pass it to use_texture. This can't currently be done from outside RenderTarget itself.
(I could write a function like the one you suggested if these RenderTarget members were made public, but I'm not sure that the API is improved by that.)
I really like the potential elegance of applying ImageEffects to RenderTargets with a specified destination (a RenderTarget or the screen), which is why the modification of RenderTarget to support more general ImageEffects seemed intuitive to me. The user provides their custom ImageEffects object and everything just works, in a way that's based on a more general form of RenderTarget's existing copy functions. The user needs to use the right names for input textures (like "colorMap" and "depthMap" in the existing copy fragment shader), but this can be prominently documented somewhere.

You have access to both the context (it is the graphics context you pass into almost all of the functionality, including when constructing the RenderTarget) and the textures (you pass a reference to them when constructing the RenderTarget) so you can have exactly the same code outside of RenderTarget. It will look something like this:
let context = window.gl().unwrap();
let effect = ImageEffect::new(..);
// ....
let color_texture = ColorTargetTexture2D::new(..);
let render_target = RenderTarget::new_color(
            &context,
            &color_texture,
        ).unwrap();
render_target.write(&ClearState::default(), || {
        // render something here
        }).unwrap();
Screen::write(&context, &ClearState::none(), || {
            effect.use_texture(&color_texture, "colorMap")?;
            effect.apply(RenderStates {..}, frame_input.viewport)?;
            Ok(())
        }).unwrap();

instead of something like this:
let context = window.gl().unwrap();
let effect = ImageEffect::new(..);
// ....
let color_texture = ColorTargetTexture2D::new(..);
let render_target = RenderTarget::new_color(
            &context,
            &color_texture,
        ).unwrap();
render_target.write(&ClearState::default(), || {
        // render something here
        }).unwrap();
render_target.apply_effect_to_screen_color(effect,
       RenderStates {..},
       frame_input.viewport
       ).unwrap()

I'm not saying I wont do it, I just have to think it through before doing it. And at least you can do the same without any changes to the API and even with roughly the same amount of code.

You're right, I'm not sure how I didn't think of this. It worked quickly and I'm now back on an unmodified master version of three-d.
There's one (very very minor) inelegance now: one of my utility functions has to receive a RenderTarget and its ColorTargetTexture2D instance as separate parameters, because the RenderTarget itself can't be used as its source within the function. I wouldn't have opened this issue if that was the only problem I had, so feel free to close the issue unless you want to use it to track related API changes.

Perfect 👍
Actually, you can just create a new RenderTarget each time you need it, then you only have to pass in the texture (and the context, but you probably need that for something else as well). You can see the RenderTarget as functionality to write to a texture, so no need to keep it around, all of the data is saved in the texture. I know it is a bit weird, but it cannot be functionality directly on the texture because a render writes to both a color and a depth texture at the same time.
Anyway, I will keep this issue open a bit longer until I have tried out the Buffer and Destination input parameters to avoid duplicate code.

I did some refactoring and it's now very clean. I hadn't thought of the RenderTarget type as a lightweight wrapper to be created immediately before use, but everything seems to work well now that I'm using it that way.
I don't remember exactly, but I think the reason I initially passed around pre-made RenderTargets was that I wasn't sure how costly they were to create. I still haven't figured out where, if anywhere in this code, large color buffer allocations actually take place.
I'm not sure if there's an easy way to do this, but somehow documenting the performance cost of different API actions might be useful.

I actually wrote in the documentation that you can create a RenderTarget each time, but I totally agree, it is difficult to know for the user what should be created each time and what needs to be created once. To make it even more confusing, it might be expensive to create a render target on some platforms, it depends on the implementation, but I think it does not cost much on most modern platforms. Even creating textures to render into can be done each frame on most platforms since it is reusing the allocated data. Anyway, I have tried to improve the documentation and I have used your suggestion about a Destination enum (I called it CopyDestination). Finally, I have added a write and copy function on all of the target textures, so you only need to create a render target when you want to write or copy a color and depth texture at the same time. All of these changes (and more) have been merged in #96. What do you think?

Well, at least there was some good news as well 👍
I'm sorry for breaking everything, I have a bad conscience every time I change the API, but most things require some iterations to be good. Also, I really appreciate that you are testing everything, but if you get tired of it, please go ahead and stick with a specific released version or a specific commit.
So the occasion for changing this was the addition of picking, so you can now just pass in meshes or anything else that implements the Geometry trait to the pick function. The same now applies to the deferred rendering pass and shadow map creation. If the transformation (and cull type) was not stored on the mesh, this information had to be passed in along with each mesh. This would not be a big problem if it was just one function, however, often you want to apply several functions to a mesh, for example create a shadow map, pick and render it. All of this should use the same transformation (and cull type) and it therefore makes everything much cleaner that it is a part of the mesh data. In addition, I think it makes sense that you mutate the mesh if you transform it so now you can guarantee that the mesh is in the correct place if you pass an immutable mesh to a function. On the other hand, the render function does not mutate anything, so that is fine to have on an immutable mesh.
Ok, so enough of me trying to explain. Why was it a big issue to change your code to use a mutable mesh? (I have tried several times that it complicates things to change from immutable to mutable, so I perfectly understand, just want a bit more context). Do you have any other suggestions how to solve this? Or is it possible to make your code nice again using the mutable mesh somehow?

Having reviewed some of the Pickable-related code, I think I agree with the decision to integrate the needed render state into the object instead of adding more method parameters. (Although I must admit, I don't fully understand what this code is actually doing.) The occasional API breakage is a small price to pay for the benefits of being directly involved with the API as it's developed, and it's not like my code is stable either, so break things as you see fit, as far as I'm concerned.
I also reviewed my own code, and I think I can do a decent refactor to clean up most of the undesirable changes, but the restoration of immutability would still be better. One specific annoyance is that the Mesh mutability requirement changed my storage type (of my container struct "LoadedImageMesh") from Rc<LoadedImageMesh> to Rc<RefCell<LoadedImageMesh>>, which means that I now have a RefCell borrow failure codepath to handle in a function that has been used in the code with the expectation of guaranteed success. I'm not exactly sure how much code would have to be changed to begin handling errors from where it was called.
It occurs to me that if Mesh implemented Copy (or a low cost Clone), then I could make a lightweight copy at the Mesh::render call site and mutate that. Is this technically possible, or is something in Mesh too costly to quickly duplicate?

I'm glad you feel like that and I hope other users have the same attitude 👍
Implementing Clone was a really good idea (implemented in 917a082), it is really low-cost. I think the most expensive is to copy the transformation matrix so it is probably ok to do it each frame. Introducing RefCells at a high level is not nice, so I understand why you want to clone instead, but I will leave that up to you 💪
I will close the issue, but feel free to give more feedback here or elsewhere.

@asny The new clone feature really cleans up the code, I like it. One problem though:

thread 'main' panicked at 'attempt to subtract with overflow', /media/sf_RustDevVM_Shared/temp_repos/three-d/src/object/mesh.rs:383:13

Relevant code:
impl Drop for Mesh {
    fn drop(&mut self) {
        unsafe {
            MESH_COUNT -= 1;
            if MESH_COUNT == 0 {
                PROGRAM_DEPTH = None;
                PROGRAM_PICK = None;
                PROGRAM_COLOR = None;
                PROGRAM_TEXTURE = None;
                PROGRAM_PER_VERTEX_COLOR = None;
            }
        }
    }
}

It looks like this expects an increment operation for every Mesh::new() call, but Mesh::clone() is derived and doesn't do the same thing. Therefore, when a clone is dropped along with the rest of the new()-created Meshes, it tries to decrement into negative numbers.

You are absolutely right and I actually thought of this but forget to implement it. I must have too many things on my mind at the moment. Anyway, it should be fixed in a44f09e 💪

We're getting closer: I can build and run and no panic occurs, but:
Each cloned Mesh is drawn for 1 frame and then disappears. I managed to restore normal rendering by commenting out the impl Drop for VertexBuffer block in vertex_buffer.rs: it looks like my temporary Mesh clone is deleting the buffer when it gets dropped, leaving the original without one. I assume an analogous issue exists with ElementBuffer, since its drop code also deletes a buffer. Those two are the only Drop implementors in Mesh, so fixing them is probably all that is needed.
An easy fix would be to store each Mesh's buffers in Rcs, so they propagate through clones and only get dropped when the last Mesh is dropped. Not sure if there are downsides to this, though.

That makes a lot of sense also but I did not think of that. Maybe there was a reason I didn’t implement clone 🤦
It is a good suggestion to wrap Mesh buffers in an Rc, but that means you are not able to update them. This works fine for Mesh, at least at the moment, but not other types like InstancedMesh where you want to be able to change the instance buffers. It would also make sense that you can change the data in the buffers of Mesh, so I don't really know what to do. I have to think about it, so will reopen this issue.
Related for both you and me, there is this method that allows you to mutate the value in a Rc but it requires that you only have one reference at that moment. Could you use that to solve your problem?

Yeah, the mutation requirement complicates this. The idiomatic reflex is to use Rc<RefCell>, but that's what we're trying to avoid here. Rc::get_mut fails if another reference exists, which means that it won't work with my current mutability workaround: when I make a local clone to mutate Mesh::transformation, it will not have the only reference and will always fail. There is also Rc::make_mut, which would create a clone instead of failing (clone-on-write, effectively), but I'm not sure how to use that to our advantage here.
This may seem a bit ad-hoc, but this solution would actually be easier than anything we've tried: a new version of Mesh::render (called Mesh::render_ext, or something better) that accepts transformation and cull as arguments and uses them instead of the Mesh struct's members. Given that I mutate transformation and cull, but not the buffer contents, this would work in my current code with only trivial changes, and probably wouldn't complicate the API too much,

I didn’t have much time this weekend, but I gave it some thought and got almost nowhere. I have semi-reverted the changes, so now only Mesh is cloneable, but I am unsure whether to keep that or not. It might be ok for now, but not if we want to make the buffers editable, then everything fails. As I see it, one either wants to have a static and cloneable mesh or an editable mesh, not both. But making a new static and cloneable mesh type is also not a good option since that leads to a static phong mesh type and a static instanced mesh type and suddenly there is a lot of work involved, so I need to get this at least somewhat right. Another idea is to mimic Rc so the mesh is cloneable and the buffers can also be changed, but only if there exists one clone.
I also thought about your solution but it will probably lead to lots of confusion that you can set the transformation on the mesh and in a render function, so are they multiplied or only one of them are used or what.
My point is that I will need to think more about it, hopefully I will have some more thinking-time soon 🤓 And it might end with that it is not possible to clone a mesh. So maybe it is a good time to ask why you needed multiple references to the same mesh in the first place? I think it is valid thing (as I said, you want either a static and cloneable mesh or a dynamic mesh), I just want to know the use-case. As a side node, if the Mesh is cloneable, you do not need the Rc<Mesh>, you can just clone it directly when needed, this way you avoid cloning each frame 💪

I decided on making Mesh and PhongMesh implement Clone 💪 I think this is quite common that you want to duplicate meshes, whereas for the other objects it doesn’t really make sense. Also, mutating the buffers in a mesh is not that common and you probably want a custom mesh type in that case anyway. I'm trying to close this again 🎉

@asny bf31e66d looks good, everything is working as expected.
A quick example of how I'm using Mesh::clone: I have a Mesh that is used for line drawing: its vertex data is a square made of  2 triangles, and that square is transformed into a line for rendering by a Mat4 from a dedicated function. The only things that change between uses are

the transformation matrix
the color uniform provided to the fragment shader

Drawing a line with a Mesh called 'line_mesh' looks like this:
let line_mat4 =   line_transform(/* details about the line location and thickness */);
color_program.use_uniform_vec4("color", &Vec4::new(0.8, 0.8, 0.2, 1.0)).unwrap();

//apply the transformation for this line to a local mutable clone of the line Mesh
let mut mesh = line_mesh.clone();
mesh.transformation = line_mat4;
mesh.render(&color_program, render_states, frame_input.viewport, &camera)?;

This way, the codebase benefits from handling Meshes as immutable objects, except for a few lines around the render call site.

Nice. Looks good and makes a lot of sense 👍 I am glad we came to that solution 🎉


Share extension not inheriting values from FLUTTER_BUILD_NAME and FLUTTER_BUILD_VERSION variables

When I am trying to create a build for the App Store, I get the following errors:

App Store Connect Operation Error
The bundle 'Payload/Runner.app/PlugIns/.appex' is missing plist key. The Info.plist file is missing the required key: CFBundleShortVersionString. Please find more information about CFBundleShortVersionString at https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundleshortversionstring

App Store Connect Operation Error
This bundle Payload/Runner.app/PlugIns/.appex is invalid. The Info.plist file is missing the required key: CFBundleVersion. Please find more information about CFBundleVersion at https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundleversion

My Info.plist is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
	<dict>
		<key>CFBundleDevelopmentRegion</key>
		<string>$(DEVELOPMENT_LANGUAGE)</string>
		<key>CFBundleDisplayName</key>
		<string>SHAREEXT-NAME</string>
		<key>CFBundleExecutable</key>
		<string>$(EXECUTABLE_NAME)</string>
		<key>CFBundleIdentifier</key>
		<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
		<key>CFBundleInfoDictionaryVersion</key>
		<string>6.0</string>
		<key>CFBundleName</key>
		<string>$(PRODUCT_NAME)</string>
		<key>CFBundlePackageType</key>
		<string>XPC!</string>
		<key>CFBundleShortVersionString</key>
		<string>$(FLUTTER_BUILD_NAME)</string>
		<key>CFBundleVersion</key>
		<string>$(FLUTTER_BUILD_NUMBER)</string>
		<key>NSExtension</key>
		<dict>
			<key>NSExtensionAttributes</key>
			<dict>
				<key>PHSupportedMediaTypes</key>
				<array>
					<string>Image</string>
				</array>
				<key>NSExtensionActivationRule</key>
				<dict>
					<key>NSExtensionActivationSupportsWebURLWithMaxCount</key>
					<integer>1</integer>
					<key>NSExtensionActivationSupportsImageWithMaxCount</key>
					<integer>5</integer>
				</dict>
			</dict>
			<key>NSExtensionMainStoryboard</key>
			<string>MainInterface</string>
			<key>NSExtensionPointIdentifier</key>
			<string>com.apple.share-services</string>
		</dict>
	</dict>
</plist>

If I populate the version values manually into the extension's Info.plist, distribution works fine, but I suppose it should be taking them from pubspec.yaml. Shouldn't it? Am I understanding this wrong? Thank you.

You may need to set the configuration for your Share Extension


GpuMat clone is not working correctly

// Given an input image call resizedImage
Mat resizedImage;
// Image is copied to the GPU - Below are two ways of copying to the GPU. Note that I have not commented out the non-working version.
cv::cuda::GpuMat srcDev(resizedImage);
cudaMemcpy(srcDev.data ,resizedImage.data, resizedImage.rows*resizedImage.cols*sizeof(unsigned char) ,cudaMemcpyHostToDevice);
// A destination matrix on the GPU is also allocated. For my specific use, I want the destination matrix to be equal to the src. I can copy the data in the following two ways. One of them works and one does not.
cv::cuda::GpuMat destDev(imSize,imSize,CV_8UC1);
destDev = srcDev.clone(); // This one does not.
cudaMemcpy(destDev.data ,srcDev.data, resizedImage.rows*resizedImage.cols*sizeof(unsigned char) ,cudaMemcpyDeviceToDevice); // This approach works

// I simply copy the data back to the CPU and print out the difference between the matrix. There should be no difference. However, with the .clone() function there is a difference.
Mat back(imSize,imSize,CV_8UC1);
cudaMemcpy(back.data ,destDev.data, checkingInputs3.rows*checkingInputs3.cols ,cudaMemcpyDeviceToHost);
cout << back-resizedImage << endl;

Not sure why this is happening. Based on the documentation I expected identical behaviors.

It is not a bug.
GpuMat by default is not contiguous. You should use cudaMemcpy2D:
cudaMemcpy2D(srcDev.data, srcDev.step,
    resizedImage.data, resizedImage.step,
    resizedImage.cols*sizeof(unsigned char), resizedImage.rows,
    cudaMemcpyHostToDevice);

You misunderstand.
The GpuMat clone function is not cloning the matrices correctly. That is the problem.

The bug in this line:
// I simply copy the data back to the CPU and print out the difference between the matrix. There should be no difference. However, with the .clone() function there is a difference.
Mat back(imSize,imSize,CV_8UC1);
cudaMemcpy(back.data ,destDev.data, checkingInputs3.rows*checkingInputs3.cols ,cudaMemcpyDeviceToHost);
cout << back-resizedImage << endl;

It should use cudaMemcpy2D:
destDev = srcDev.clone();

// I simply copy the data back to the CPU and print out the difference between the matrix. There should be no difference. However, with the .clone() function there is a difference.
Mat back(imSize,imSize,CV_8UC1);
cudaMemcpy2D(back.data, back.step,
    destDev.data, destDev.step,
    destDev.cols * destDev.elemSize(),
    destDev.rows, cudaMemcpyDeviceToHost);
cout << back-resizedImage << endl;


http step extension shows an error on first load

Describe the Bug
The http step extension shows an error on first load. It's usually rerendered in seconds but sometime it persists and need to manually reopen the tab.
Steps to Reproduce the Bug or Issue

Change DSL type to KameletBinding
add http-source
click added http-source
click entended configuration tab

Screenshots or Videos
https://user-images.githubusercontent.com/265462/223808202-0a84a2a7-0554-4766-960d-f0e4ff44e8d6.mp4
Platform

OS: [e.g. macOS, Windows, Linux]
Browser: [e.g. Chrome, Safari, Firefox]
Version: [e.g. 0.30.0]

note: I reproduce with Chrome but not with Firefox


Incorrect type for labels parameter: "array of undefineds"

What article on docs.github.com is affected?
https://docs.github.com/en/free-pro-team@latest/rest/reference/issues#create-an-issue
What part(s) of the article would you like to see updated?

https://docs.github.com/en/free-pro-team@latest/rest/reference/issues#update-an-issue--parameters
https://docs.github.com/en/free-pro-team@latest/rest/reference/issues#create-an-issue--parameters

The labels parameter's type is listed as array of undefineds
Additional information

Thanks for opening an issue @gr2m 💝

This seems to be a bit more complicated than content only:

The content page where the issue is perceived (docs/edit/main/content/rest/reference/issues.md) uses an specific include for describing REST operations (docs/includes/rest_operation.html);
-The include file above programatically parses the operation descriptions contained in the npm package @github/rest-api-operations;
While the referenced repository for this package (https://github.com/github/rest-api-operations) is not publicly available, you can find the same descriptions at https://github.com/github/rest-api-description (was the first deprecated in favor of the latter?)
According to the related operation description contained in api.github.com.json (line 29364), the labels parameter is an array that requires either a string or an object with multiple parameters.

My guess is that the property parsing mechanism doesn't do well with the oneOf array property (which establishes the either condition), so it returns array of undefined as the property type for labels (should be: something like array of string OR object).
@janiceilene perhaps the best course of action ehre would be to re-triage this to engineering - what do you think?

Amazing analysis 👍🏼

@casals and @gr2m
It looks like you're correct:
              oneOf:
                - type: string
                - type: object

I will discuss it with the Docs Ecosystem team and triage it on our project board to get it fixed.

@gr2m, this looks like a bug with our internal schema. I don't think we allow creating an issue with an array of label objects only an array of strings. I'm going to verify this with the owner of the API.

This has been moved to our internal repo, so I'm going to close out this PR. Thank so much!


Upgrade to gradle 2

Upgrading to gradle 2 will solve the problem with Java 1.8 compatability.  I'm not sure how disruptive this upgrade will be, but is should be a non-functional-changing updgrade, and hence we might be able to put it into EA-3.

left this one open to provide better documentation to the community on tested versions. sent a draft to daphne/gaurav and will post to wiki later today.

Documented tested versions for software users must download on their own here: https://github.com/marklogic/marklogic-samplestack/wiki/Version-Compatibility
-> @gghai, @laurelnaiad, @grechaw good to get your confirmation.
Opened #510 to upgrade testing infrastructure to match node/npm recommendations.
(note on original issue: Since gradle is included and packaged up as a dependency not mentioned on that page. on-going updates for all libraries should be considered in releases going forward as part of normal release process).


net: if: Assert that link layer addr was set by driver

There's a lot of code which assumes net_if_get_link_addr(iface)->addr
Forgetting to set it leads to deferred, spectacular crashes. It's
impractical to assert it on every usage. So, instead let's assert
it after call to driver->init(), as that is supposed to set it.
Signed-off-by: Paul Sokolovsky<EMAIL_ADDRESS>

This comes from my attempt to write a network driver from scratch. Quite an ordeal you know. Comes after few hours of debugging of this crash:
***** Booting Zephyr OS zephyr-v1.13.0-1494-gfabf4596e1 *****
eth_tx
qemu: fatal: Lockup: can't escalate 3 to HardFault (current priority -1)

R00=0000002a R01=2000af28 R02=00018bb5 R03=00000000
R04=2000af48 R05=00000000 R06=00000f59 R07=2000af28
R08=00000000 R09=ffffffff R10=00000000 R11=00000000
R12=00000000 R13=2000aedc R14=0000106f R15=00000000
XPSR=00000003 ---- A handler
FPSCR: 00000000

And yeah, I yet to see this change to log back to me in consistent manner, due to https://github.com/zephyrproject-rtos/zephyr/issues/11008 (or maybe not that, but some other issue). But this is definitely the right change.

Codecov Report

Merging #11010 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #11010      +/-   ##
==========================================
+ Coverage   53.21%   53.22%   +<.01%
==========================================
  Files         217      217
  Lines       26667    26668       +1
  Branches     5913     5914       +1
==========================================
+ Hits        14192    14193       +1
  Misses      10059    10059
  Partials     2416     2416

Impacted Files
Coverage Δ

subsys/net/ip/net_if.c
65.59% <100%> (+0.02%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aa2bdbe...48802f3. Read the comment docs.


cant use the plugin when im using papermc

yes i know its for papermc but when i tried to do it it said "NO SIMPLE VOICE CHAT CLIENT" when i cant even download the mod in my server bc the soft ware is for plugins :/

Please provide more context. I have no idea what you are talking about.
The plugin must be installed on the server along with Simple Voice Chat. The client that uses it must not have Simple Voice Chat installed.

sorry english is my second lanauga any ways when i decided to make simple voice chat plugin and the simple voice chat discord bridge plugin work i used the command /dvc start and it said it started but the bot left then it said this any way to fix it because i tried a soft ware that adds plugins and mods together but it wasnt starting server

You can only use this plugin if you have not installed the Simple Voicechat mod.
This is even said in the error message.

Thank you! Yes, please just read the error message. It says, "Please check that you DO NOT have Simple Voice Chat installed on your client." This means that you must not have Simple Voice Chat installed on your client to use the addon.


hyprland/workspaces - no highlight on active spaces

Issue
When "persistent_workspaces": { } in config.jsonc is set, all non-empty non-selected workspace don't get highlighted.
Versions (affected at the time)
0.9.23, 0.9.24
Other context
Archlinux repos, hyprland 0.30.0
style.css:
...
/* Workspace not selected, and not empty */
#workspaces button {
    padding: 0 3px;
    background-color: transparent;
    color: @red;
}

/* Workspace not selected, and empty */
#workspaces button.persistent {
    background-color: transparent;
    color: #333333;
}

/* Selected workspace */
#workspaces button.active {
    background-color: @base; /*#64727D;*/
    color: #ffffff;
    box-shadow: inset 0 -3px @pink;
}

#workspaces button:hover {
    background: rgba(0, 0, 0, 0.2);
    box-shadow: @crust;
    color: @mauve;
}

#workspaces button.urgent {
    background-color: #eb4d4b;
    color: white;
}
...

config.jsonc:
...
     "hyprland/workspaces": {
        "active-only": false,
        "disable-scroll": true,
        "warp-on-scroll": false,
        "sort-by-number": true,
        "persistent_workspaces": {
          "1": ["eDP-1"],
          "2": ["eDP-1"],
          "3": ["eDP-1"],
          "4": ["eDP-1"],
          "5": ["eDP-1"],
          "6": ["eDP-1"],
          "7": ["eDP-1"],
          "8": ["eDP-1"],
          "9": ["eDP-1"],
          "10": ["eDP-1"],
          "101": ["DP-1"],
          "102": ["DP-1"],
          "103": ["DP-1"],
          "104": ["DP-1"],
          "105": ["DP-1"],
          "106": ["DP-1"],
          "107": ["DP-1"],
          "108": ["DP-1"],
          "109": ["DP-1"],
          "110": ["DP-1"]
        },
     "format": "{icon}", //use "{name}: {icon}" to display numbers
     "format-icons": {
        "1": "",
        "2": "",
        "3": "",
        "4": "",
        "5": "",
        "6": "",
        "7": "",
        "8": "",
        "9": "",
        "10": "",
        "101": "",
        "102": "",
        "103": "",
        "104": "",
        "105": "",
        "106": "",
        "107": "",
        "108": "",
        "109": "",
        "110": "",
        "urgent": "",
        "focused": "",
        "default": ""
         }
     },
...

hyprland.conf:
(persistent flag doesnt make a difference)
# Set workspaces for 1st monitor
workspace = 1, monitor:eDP-1, persistent:true, default:true
workspace = 2, monitor:eDP-1, persistent:true
...
workspace = 9, monitor:eDP-1, persistent:true
workspace = 10, monitor:eDP-1, persistent:true

# Set workspaces for 2nd monitor
workspace = 101, monitor:DP-1, persistent:true, default:true
workspace = 102, monitor:DP-1, persistent:true
...
workspace = 109, monitor:DP-1, persistent:true
workspace = 110, monitor:DP-1, persistent:true

Workaround at 03.11.2022
downgraded to waybar 0.9.22

I have the same issue. Occupied workspaces are no longer highlighted in version 0.9.23 and 0.9.24. last version that still worked is 0.9.22
@BoopyTheFox  persistent_workspaces is deprecated, use persistent-workspaces

Same here.

I'm not sure what you mean with "highlighted", can you explain what you expect to see and what you actually see?
I don't get any problems with the latest version, here is my style.css:
/* non-empty workspaces */
#workspaces button {
    background-color: #4c566a;
    /* Use box-shadow instead of border so the text isn't offset */
    box-shadow: inset 0 -3px transparent;

    color: #d8dee9;
    transition: all 0.2s ease;

    border: none;
    border-radius: 0;
}

#workspaces button:hover {
    /* transition-duration: .1s; */
    color: #eceff4;
    box-shadow: inset 0 -2px #ffffff;
    /* https://github.com/Alexays/Waybar/wiki/FAQ#the-workspace-buttons-have-a-strange-hover-effect */
    text-shadow: inherit;
}

/* non-empty persistent workspaces */
#workspaces button.persistent {
    background-color: #4c566a;
}

/* empty persistent workspaces */
#workspaces button.empty {
    background-color: #2e3440;
}

/* active or visible workspaces */
#workspaces button.active,
#workspaces button.visible {
    background-color: #4c566a;
    box-shadow: inset 0 -4px #eceff4;
}

#workspaces button.urgent {
    color: #bf616a;
}

I think it's because I use empty for empy persistent workspaces, and persistent for occupied persistent workspaces. I don't immediately see why this would cause problems if you don't have the empty class in style.css, though.

I think it's because I use empty for empy persistent workspaces, and persistent for occupied persistent workspaces. I don't immediately see why this would cause problems if you don't have the empty class in style.css, though.

Indeed, that was the issue. This CSS was working before the update:
#workspaces {
  background: black;
  border-radius: 10px;
  padding: 5px 8px;
  margin: 10px;
  box-shadow: 0px 0px 3px rgba(0,0,0,1);
}

#workspaces button {
	border-radius: 0px;
	color: @white;
	/* transition: all 0.25s ease-in-out; */
}

#workspaces button.persistent {
	color: @brblack;
}

#workspaces button.active {
  color: @brgreen;
  text-shadow:
    0.25px 0.25px 0 @brgreen,
    -0.25px 0.25px 0 @brgreen,
    -0.25px -0.25px 0 @brgreen,
    0.25px -0.25px 0 @brgreen;
}

#workspaces button:hover {
	background-color: transparent;
	color: @brgreen;
}

Adding #workspaces button.empty solved the issue:
#workspaces {
  background: black;
  border-radius: 10px;
  padding: 5px 8px;
  margin: 10px;
  box-shadow: 0px 0px 3px rgba(0,0,0,1);
}

#workspaces button {
	border-radius: 0px;
	color: @white;
	/* transition: all 0.25s ease-in-out; */
}

#workspaces button.persistent {
	color: @white;
}

#workspaces button.empty {
    color: @brblack;
}

#workspaces button.active {
  color: @brgreen;
  text-shadow:
    0.25px 0.25px 0 @brgreen,
    -0.25px 0.25px 0 @brgreen,
    -0.25px -0.25px 0 @brgreen,
    0.25px -0.25px 0 @brgreen;
}

#workspaces button:hover {
	background-color: transparent;
	color: @brgreen;
}

I think it's because I use empty for empy persistent workspaces, and persistent for occupied persistent workspaces. I don't immediately see why this would cause problems if you don't have the empty class in style.css, though.

Indeed, that was the issue. This CSS was working before the update:
#workspaces {
  background: black;
  border-radius: 10px;
  padding: 5px 8px;
  margin: 10px;
  box-shadow: 0px 0px 3px rgba(0,0,0,1);
}

#workspaces button {
	border-radius: 0px;
	color: @white;
	/* transition: all 0.25s ease-in-out; */
}

#workspaces button.persistent {
	color: @brblack;
}

#workspaces button.active {
  color: @brgreen;
  text-shadow:
    0.25px 0.25px 0 @brgreen,
    -0.25px 0.25px 0 @brgreen,
    -0.25px -0.25px 0 @brgreen,
    0.25px -0.25px 0 @brgreen;
}

#workspaces button:hover {
	background-color: transparent;
	color: @brgreen;
}

Adding #workspaces button.empty solved the issue:
#workspaces {
  background: black;
  border-radius: 10px;
  padding: 5px 8px;
  margin: 10px;
  box-shadow: 0px 0px 3px rgba(0,0,0,1);
}

#workspaces button {
	border-radius: 0px;
	color: @white;
	/* transition: all 0.25s ease-in-out; */
}

#workspaces button.persistent {
	color: @white;
}

#workspaces button.empty {
    color: @brblack;
}

#workspaces button.active {
  color: @brgreen;
  text-shadow:
    0.25px 0.25px 0 @brgreen,
    -0.25px 0.25px 0 @brgreen,
    -0.25px -0.25px 0 @brgreen,
    0.25px -0.25px 0 @brgreen;
}

#workspaces button:hover {
	background-color: transparent;
	color: @brgreen;
}

I also faced the same issue. They haven't updated the wiki yet.

I think anyone can update the wiki, be the change you want to see in the world ;)

Solution
Use #workspaces button.empty for empty workspaces.
persistent - for non-empty, non-active workspaces
active and visible for active and visible workspaces

sorry for bumping this old post but I'm trying to implement the same thing but with circles and I can't wrap my head around how to work with the special characters (icons from this point onward)?
The above example works fine if you don't have icons for workspace indicator but how would you accomplish the same thing but with icons
This is what I could build up
"hyprland/workspaces": {
	"active-only": false,
	"all-outputs": true,
	"format": "{icon}",
	"show-special": false,
	"on-click": "activate",
	"on-scroll-up": "hyprctl dispatch workspace e+1",
	"on-scroll-down": "hyprctl dispatch workspace e-1",
	"persistent-workspaces": {
		"1": [],
		"2": [],
		"3": [],
		"4": [],
		"5": [],
		"6": [],
		"7": [],
		"8": [],
		"9": [],
		"10": [],
	},
	"format-icons": {
		"active": "",
		"default": "",
		"urgent": "",
	},
},

#workspaces button {
  color: @white;
  background-color: @module-bg;
  padding: 4px;
}

#workspaces button.active {
  color: @module-bg;
  background-color: @white;
}

#workspaces button.persistent {
  background-color: @module-bg;
  color: @white;
}

#workspaces button.empty {
  color: @white;
}
#workspaces button.urgent {
  color: @red;
}

Now I want a filled circle as a occupied worspace but I can't get my head around how I can accompish that?
From what I can gather, I can only show colors but no icon as indicator for occupied workspace

Hi @greeid, the config is different now. Let me give you an example for what you want to achieve. This is actually a modified version of my Waybar.
"hyprland/workspaces": {
  "disable-scroll": false,
  "on-scroll-up": "hyprctl dispatch workspace -1",
  "on-scroll-down": "hyprctl dispatch workspace +1",
	"format": "{icon}",
	"format-icons": {
    "active": "",
    "default": "",
    "urgent": "",
	},
  "persistent-workspaces": {
       "*": 10
  }
},

Thanks @Axenide for the reply but what I wanted to accomplish was that it should have   as the active,   as default/empty and   as occupied workspace but I could only change the color for the occupied workspace and not the icons?
Or is it not possible in waybar??

@greeid Of course you can change the icon color. Here is a CSS that should work:
#workspaces button {
	color: white;
}

#workspaces button.persistent {
	color: white;
}

#workspaces button.empty {
    color: gray;
}

#workspaces button.active {
  color: green;
}

#workspaces button:hover {
	color: green;
}

I could only change the color for the occupied workspace and not the icons?

@greeid Of course you can change the icon color.

Hmm I don't think u get my question :sweat:

Hmm I don't think u get my question 😓

Sorry, I misread. Yeah, can't do that on Waybar yet.

@greed-d You may have moved on from wanting to color the circles based upon workspace state, but I just implemented what you were trying to do. I'm dropping the solution here in case anyone else wants to do something similar.

You can set the color of the icons using pango in the waybar config, not the style.css. Here's an excerpt from my config:
  "hyprland/workspaces": {
    "format": "{icon}",
    "format-icons": {
      "active": "",
      "persistent": "",
      "empty": "",
    },
    "persistent-workspaces": {
      "*": 10
    }

All you need to do is set  for each icon as desired. Nothing needs to be added to the css.

Nice, thanks for putting it here


Update index.md

Fixed second paragraph for better understanding.

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

@MedranoCE Thank you for  creating this PR.  However, since it was created, the underlying document has changed.  Can you please review and resolve the current merge conflicts?  It could be the changes to the underlying document are so significant that your changes are no longer relevant.  If the latter is the case, it might be easier to just close this PR and create a new PR with any changes you still feel are important to make.
If you need any assistance or have questions about resolving the merge conflicts, please reply back or contact someone in the Contributors Room on Gitter.
Thank you.

We are restructuring a lot of the code and learning resources referenced in your pull request, so for now I am going to go ahead and close this pull request. Thanks again for opening this pull request and for contributing to the community.


[Don't merge to master:] Create docs changelog on staging

Apologies, this change got committed directly to staging. Please don't merge this PR directly to master. I'll close it once we adjust the changelog wording.

Instantiates changelog, covering first two releases.
Added to left nav's existing Reference section, to avoid heading clutter.

Closing – this PR simply records the Changelog page I added directly onto the staging branch.


Update README.md

Is it really neccessary to know this info when you are running a Dev Container

No, this is for those of us who run only the python API (only me I guess) 😔

Is it really neccessary to know this info when you are running a Dev Container

No, this is for those of us who run only the python API (only me I guess) 😔

Sucks to suck man


General UI updates plus improvements on mobile

General Updates:

Move Mini Credits from common to here because image links were not working from common
Update splash screen & styling
Remove ecliptic toggle
Standardize font size & line height with defaults and calculate other values relative to that

Small Screen Updates:

Don't show controls on small screen
Use flex box layout, so info box (with eclipse image) goes vertical on narrower screens.
Don't show credit icons on small screens
Hide time slider label on small screens (and switch date chip back to date/time so time is still displayed on screen when slider isn't)

@johnarban and @Carifio24, if you are able to test this out on your phones and tablets, that would be very helpful. I didn't realize until I checked on my phone how truly terrible our layout was on phones. I'm sure there will be other weird things that happen with different aspect ratios.


Fix resolvers returning undefined

Fixes #17
⚠I added a failing test case in 993a417: however, all tests passed even with the failing test case. But it failed (correctly) if I'd run only that case in particular.

@DanielMSchmidt do you have the opportunity to look at this please?

Hey @alexstrat Daniel is off for two weeks I will do my best to support the PR :)

@nLight ah ok, good for him! No worry, it can wait 2 weeks if not your priority!

@nLight I have been able to pinpoint the issue with the tests and proposed a fix in #25. Do you think it would be possible to review/merge/release this fix (and #23) soon please?

@alexstrat I will take a look at this tomorrow, sorry for the delay 🙇

I had some time and took a look. I removed ever .only and removed two fields from queries in tests and everything was green. But also with the .only everything passed just fine, so I think I don't understand the problem we are facing here 🙈

@DanielMSchmidt, I'm sorry, I don't understand your last comment 😬

I think I don't understand the problem we are facing here
What problem do you mean? #24 (skipped tests) or #17 (missing field for resolvers returning undefined)

I rebased with master and made necessary changes in aafb8e0

:tada: This PR is included in version 3.0.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Adblock Plus can hide sharing link

When Adblock Plus has the "Block social media icons tracking" option enabled, it injects a stylesheet that, among other things, explicitly sets display: none !important on any element with the class .sharing-link. This causes the sharing link to be hidden in Conclave (and is really not Conclave's fault at all).
As far as I can tell, the class .sharing-link is only referenced once in the code base, to set display: inline-block—it might be worth changing the class name or targeting the element in another manner.

Thanks for bringing this to our attention. We should be able to change that class name easily enough. Will do so when I get some time.


Quorum App API: Implement PUT /api/v1/product/{trackingID}

Description
As a developer I want to use API to get update parameters of a  product on Quorum network so that I can integrate with frontend.
Accepts an id url parameter and a JSON body with the below format:
{
  "productName":"Dextrose",
  "health":"String",
  "misc": {"name":"Expensive Dextrose"},
  "trackingID": "0d15d7b8-caaa-468d-8b83-aae049b40f46",
  "counterparties": ["PartyB","PartyC","PartyD"]
}

and calls the UpdateProduct transaction with the supplied JSON.
It should return an HTTP 400 error if productName, health, or misc are not supplied.
It should return any errors received from the Fabric SDK.
Otherwise it should return the id of the updated product in the below format:
"0d15d7b8-caaa-468d-8b83-aae049b40f46"

Acceptance Criteria

Update should only happen when caller is the custodian

This issue is wontfix as Update Product is not implemented. This is so that the supplychain APIs are consistent across different platforms.


checker falsely flags halo inside shape

see https://github.com/bioimage-io/spec-bioimage-io/pull/76
best guess is that the problem might be here: https://github.com/bioimage-io/python-bioimage-io/blob/2f7184f0f35e8dd195c4045247fdefa9fabc5502/pybio/spec/schema.py#L91-L95

Yes, it looks like halo was mistakenly put here
instead of the OutputShape.
cc @FynnBe

I think the error lies in https://github.com/bioimage-io/spec-bioimage-io/pull/76
In our "reference example" halo is also part of the output tensor description, not of the shape:
https://github.com/bioimage-io/pytorch-bioimage-io/blob/113e6d01989e39cd955cb2ead311086548d95026/specs/models/unet2d_nuclei_broad/UNet2DNucleiBroad.model.yaml#L45
This is because the shape might be defined with reference_input, etc. but it may also be a fixed shape : [1, 2, 3], and halo may be specified independently...
I see some arguments for including halo under shape, but as of now that is indeed 'wrong'

I think the error lies in bioimage-io/spec-bioimage-io#76
In our "reference example" halo is also part of the output tensor description, not of the shape:
https://github.com/bioimage-io/pytorch-bioimage-io/blob/113e6d01989e39cd955cb2ead311086548d95026/specs/models/unet2d_nuclei_broad/UNet2DNucleiBroad.model.yaml#L45

I am pretty sure that we have put halo in the shape (going back to the discussions about how to compute the output shape).
This is rather an error here and in the reference example.

This also becomes clear if you actually read what we have in the spec description:
- reference_input name of the reference input tensor
- scale list of factors 'output_pix/input_pix' for each dimension
- offset position of origin wrt to input
- halo the halo to crop from the output tensor (for example to crop away boundary efects or for tiling). The halo should be croped from both sides, i.e. shape_after_crop = shape - 2 * halo. The halo is not cropped by the bioimage.io model, but is left to be cropped by the consumer software. Use offset if the model output itself is cropped.

This refers to offset without any further modifiers, which strongly implies that its on the same level of hierarchy.

I see how this ambiguity arose over time, as we had further discussions about the meaning of halo and offset. While offset only makes sense with the presence of reference_input, halo is independent of how the output shape is defined. If at any time we decided to move halo into shape I am not opposed to changing the validator accordingly, but I am puzzled about the use case of a fixed output shape that has a halo. Because of this case I see no other way (without further changes) to keep halo out of shape.

If at any time we decided to move halo into shape

It was always part of shape, see these commits were we introduced the shape descriptions:
https://github.com/bioimage-io/spec-bioimage-io/commit/9d1dfb642a62f92790d71b11b5ff9bf4891b17ea
https://github.com/bioimage-io/spec-bioimage-io/commit/dc545503e9052865ff931abaead92f3e6becddc0

I am puzzled about the use case of a fixed output shape that has a halo

This is again going back to the discussions we had about this, but halo is a hint for the consumer software on what parts of the output to crop away in order to get the "valid" part of the prediction, e.g. to avoid border effects.
So in that sense it should not matter if the output shape is fixed or computed from offset and scale, the treatment of halo is always identical.

So in that sense it should not matter if the output shape is fixed or computed from offset and scale, the treatment of halo is always identical.

exactly, but in case shape is specified explicitly, halo cannot be below shape like offset  and scale
I am thinking of this scenario:
output:
  shape:  [10, 20, 30]  # explicit shape
  halo: [1, 2, 3]  # halo is outside of shape

Ok, I agree. This is an error in the spec. I will add a PR to fix it in there properly.

See https://github.com/bioimage-io/spec-bioimage-io/pull/77.

This is fixed.


feat: remove settings option for connection invitation

Summary of Changes
Removed Connection invitation from menu list
Related Issues
N/A
Pull Request Checklist
Tick all boxes below to demonstrate that you have completed the respective task. If the item does not apply to your this PR check it anyway to make it apparent that there's nothing to do.

[x] All commits contain a DCO Signed-off-by line (we use the DCO GitHub app to enforce this);
[x] Updated LICENSE-3RD-PARTY.md for any added dependencies or vendored components;
[x] Updated documentation as needed for changed code and new or modified features;
[x] Added sufficient tests so that overall code coverage is not reduced.

If you have any questions to any of the points above, just submit and ask! This checklist is here to help you, not to deter you from contributing!
Pro Tip 🤓

Read our contribution guide at least once; it will save you a few review cycles!
Your PR will likely not be reviewed until all the above boxes are checked and all automated tests have passed.

PR template adapted from the Python attrs project.

Is the ConnectionInvitation screen still used after this is removed?

Is the ConnectionInvitation screen still used after this is removed?

Ah, right, I guess not anymore


Prototype Pollution [High Severity][https://snyk.io/vuln/SNYK-JS-Y18N-1021887] in<EMAIL_ADDRESS>

I'm submitting a...

[x] Bug report

Current behavior
High vulnerability
Expected behavior
No vulnerability
Solution
Should upgrade yargs dependency to 16

In db-migrates case this is very low urgency, since the attack vector is not really present. Please feel free to validate that these upgrades do not cause any issues and open a PR to get this upgraded more quickly.

https://github.com/db-migrate/node-db-migrate/pull/718


Follower adds entries to DB via non-libevent thread.

I sat down at my computer meaning to do this, only to find your PR. :smiley:

LGTM, other than the leftover header. Thanks!


Add HIP workshop to documentation/workshops

Description
Hi all, this adds a link to a 1-day Napari/Python/Jupyter workshop we are running in Germany
Type of change

[x] This change is a documentation update

References
How has this been tested?
I clicked on the links.
Final checklist:

[x] My PR is the minimum possible work for the desired functionality
[ ] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] If I included new strings, I have used trans. to make them localizable.
For more information see our translations guide.

I don't have authority but thanks for tagging @andy-sweet :D
great material @haesleinhuepf!


Mac: Investigate issues with PrjFSKext colliding with System Center Endpoint Protection (and other kexts and anti-virus software)

If our kext is loaded at the same time as the kext that ships with System Center Endpoint Protection, the machine will panic on next file access. I suspect this will occur with other AV kexts as well, we need to investigate what mitigations we can take in our kext to not collide with others on a machine.

Do we have a reasonably symbolicated panic trace for this? Normally, multiple vnode listeners should be able to coexist with each other and kexts implementing MAC framework policies. (Their callbacks are just called one after the other.) I'm wondering if it's therefore something to do with the I/O calls we make from the kext for querying file attrs and xattrs among other things.

Whenever we end up investigating here, we should also investigate Sophos Antivirus for Mac OS X.

I can confirm that there is evidently some bad interaction with Avast, as I've now set up a clean Mojave install, installed Avast and VFS4G and then did a gvfs clone followed by a build and it went from about a minute to long enough that the Mac eventually went into standby before completing the build, so I didn't get a reliable time measurement, but it's definitely a dreadful slowdown. I'll try to dig deeper into what's causing it. CPU load is near enough idle according to Activity Monitor.

I have a bit of news regarding the AV interaction performance degradation issue:
There is no significant slowdown for the following scenarios:

Regular git clone and build (non-VFS4G) with Avast AV "File Shield" active, PrjFS kext loaded, VFS4G repo mounted elsewhere.
Perform build in freshly cloned and mounted VFS4G repo with Avast AV "File Shield" inactive.
Build after "clean" of previous successful build in mounted VFS4G repo with Avast AV "File Shield" active (i.e. build with AV off, turn AV back on, clean build artefacts, build again)

So it seems to be some interplay between non-hydrated files and the AV.
Running the build with dtruss reveals that certain file I/O operations (I've mainly seen stat64) seem to take just a fraction over 60 seconds (!) to complete. Obviously this holds up the proceedings rather badly. It's not absolutely every I/O call made by the build processes, but certainly enough of them. The calls don't fail, they just take over a minute to succeed.
60 seconds screams "timeout" - my guess right now is that there's some kind of deadlock happening between PrjFS/VFS4G and the AV, and the AV breaks it by timing out after 60 seconds. I will investigate further.

I got stuck on a missing KDK of all things. A fully patched system is on macOS 10.14.3 18D109, but the only KDK for 10.14.3 is 18D21c, which I think is a beta. There are a bunch of 10.14.4 KDKs, but for some reason Software Update isn't picking up any beta releases when enrolled in the beta programme. 🙄
I've filed a radar about the missing KDKs and will do another fresh install with 10.13.x or 10.14.2 next and try with AV there.

I've now managed to get kernel stacks of the relevant processes in the semi-seized-up system and confirmed that it's effectively a kauth deadlock between the AV and PrjFS/VFS4G. The scenario is again that I'm performing a build in a freshly cloned VFS4G repo (with hot local VFS4G cache to eliminate any networking effects.) What's happening is:

Build tool accesses empty file
PrjFS kext requests hydration from provider
Provider writes hydrated data to the file.
Provider closes the file, triggering KAUTH_FILEOP_CLOSE event.
Avast AV's fileop handler requests a scan(?) of the file from its user space daemon.
Its user daemon attempts to access the file
PrjFS kext notices the file is marked empty & requests hydration from provider
Provider is still stuck in close() for that same file.

There are a few obvious mitigations against various variants of this sort of scenario, but it's going to be really difficult to comprehensively defend against the entire type of problem, so I'm thinking we should make this a multi-layered fix.
Immediate suggestions, I will probably think of more:

I've been told another similar product prevents the equivalent of our hydration operation by simply denying access to non-hydrated files to processes running as root. If we're going to implement something like this I'd suggest we prevent hydration by any other user apart from the owner of the provider. Issues:

Note this won't prevent interaction with similar kexts that happen to block on processes running as the same user.
This would break the deadlock but note that in our situation this would actually prevent the virus scanner from scanning this file for no particularly good reason.

The provider should mark the file as hydrated and resuming the kext before closing the file. I know this sounds vaguely crazy but this alone would already break the deadlock we're running into here, and I can't think of any negative effects.
Generalising the previous point: Audit all our kernel-to-user messages and check if each of them really needs to be blocking, and if a specific message does, then check if it could be unblocked earlier than our code currently does.
Time out on hydration and other blocking messages. The AV clearly does this on its own kauth handler (it actually seems to be 30 seconds, but there are 2 sequential blockers which add up to 60 seconds in hydration), and this is why progress is eventually made. We should have this as a last line of defense. Given that hydration might legitimately take a long time for large files on a slow network link, we could have the provider keep "checking in" with the kext every now and then as long as it's genuinely making progress to reset the timeout.

Any thoughts on this?

For reference, these are the relevant kernel stacks I saw
Build tool:
task                 vm_map               ipc_space            #acts flags    pid       process             io_policy  wq_state  command
0xffffff806695ba70   0xffffff8066970d00   0xffffff8051b998c0       7 D        882   0xffffff8066952db0                -1 -1 -1    xcodebuild
  thread                   thread_id  processor            base   pri    sched_mode      io_policy       state    ast          waitq                            wait_event           wmesg                thread_name
  0xffffff804c89f8b0       0x1b69     0xffffff8203066000   31     31     timeshare                       WU                    0xffffff81e26acb00               0xffffff9215ecb520   io.gvfs.PrjFSKext.Sleep
    kernel_stack = 0xffffff9215ec8000
    stacktop = 0xffffff9215ecb210
    0xffffff9215ecb210 0xffffff8025f82ddd machine_switch_context((thread_t) old = 0xffffff804c89f8b0, (thread_continue_t) continuation = 0x0000000000000000, (thread_t) new = 0xffffff804767d000)
    0xffffff9215ecb290 0xffffff8025e8767a thread_invoke((thread_t) self = 0xffffff804c89f8b0, (thread_t) thread = 0xffffff804767d000, (ast_t) reason = 0)
    0xffffff9215ecb2e0 0xffffff8025e8663f thread_block_reason((thread_continue_t) continuation = <>, , (void *) parameter = <>, , (ast_t) reason = <>, )
    0xffffff9215ecb320 0xffffff8025e7b73b thread_block [inlined]((thread_continue_t) continuation = <>, )
    0xffffff9215ecb320 0xffffff8025e7b730 lck_mtx_sleep_deadline((lck_mtx_t *) lck = 0xffffff804572bbf0, (lck_sleep_action_t) lck_sleep_action = 0, (event_t) event = 0xffffff9215ecb520, (wait_interrupt_t) interruptible = 0, (uint64_t) deadline = <>, )
    0xffffff9215ecb3a0 0xffffff802632f7f8 _sleep((caddr_t) chan = 0xffffff9215ecb520 "\x06\x03", (int) pri = 50, (const char *) wmsg = <>, , (u_int64_t) abstime =<PHONE_NUMBER>92, (int (*)(int)) continuation = 0x0000000000000000, (lck_mtx_t *) mtx = 0xffffff804572bbf0)
    0xffffff9215ecb3e0 0xffffff802632fdb2 msleep((void *) chan = 0xffffff9215ecb520, (lck_mtx_t *) mtx = 0xffffff804572bbf0, (int) pri = 50, (const char *) wmsg = 0xffffff7fa672da72 "io.gvfs.PrjFSKext.Sleep", (timespec *) ts = <>, )
    0xffffff9215ecb430 0xffffff7fa6729cf6 Sleep(int, void*, Mutex*)((int) seconds = 5, (void *) channel = 0xffffff9215ecb520, (Mutex *) mutex = 0xffffff7fa6734cd8)
    0xffffff9215ecb5a0 0xffffff7fa6729181 TrySendRequestAndWaitForResponse(short, MessageType, vnode*, FsidInode const&, char const*, int, char const*, int*, int*)((VirtualizationRootHandle) root = 1, (MessageType) messageType = MessageType_KtoU_HydrateFile, (vnode_t) vnode = 0xffffff805b5892e8, (const FsidInode &) vnodeFsidInode = 0xffffff9215ecb668: {,   fsid = {,     val = ([0] = 16777243, [1] = 24),   },   inode = 1496794, }, (const char *) vnodePath = 0xffffff9215ecb720 "/Users/test/devel/Adium/src/Plugins/Purple Service/Resources/es.lproj/DCPurpleOscarJoinChatView.xib", (int) pid = 882, (const char *) procname = 0xffffff9215ecb700 "xcodebuild", (int *) kauthResult = 0xffffff9215ecb694, (int *) kauthError = 0xffffff9215ecbbac)
    0xffffff9215ecbb30 0xffffff7fa67278b6 HandleVnodeOperation(ucred*, void*, int, unsigned long, unsigned long, unsigned long, unsigned long)((kauth_cred_t) credential = 0xffffff8059e30880, (void *) idata = 0x0000000000000000, (kauth_action_t) action = 2176, (uintptr_t) arg0 = 0xffffff804d548840, (uintptr_t) arg1 = 0xffffff805b5892e8, (uintptr_t) arg2 = 0x0000000000000000, (uintptr_t) arg3 = 0xffffff9215ecbbac)
    0xffffff9215ecbb90 0xffffff80262e8664 kauth_authorize_action((kauth_scope_t) scope = 0xffffff8045c54a00, (kauth_cred_t) credential = <>, , (kauth_action_t) action = <>, , (uintptr_t) arg0 = 0xffffff804d548840, (uintptr_t) arg1 = 0xffffff805b5892e8, (uintptr_t) arg2 = <>, , (uintptr_t) arg3 = <>, )
    0xffffff9215ecbbb0 0xffffff80260a1939 vnode_authorize((vnode_t) vp = <>, , (vnode_t) dvp = <>, , (kauth_action_t) action = <>, , (vfs_context_t) ctx = <>, )
    0xffffff9215ecbbf0 0xffffff80260c0ff0 vn_stat((vnode *) vp = 0xffffff805b5892e8, (void *) sb = 0xffffff9215ecbca8, (kauth_filesec_t *) xsec = 0x0000000000000000, (int) isstat64 = 1, (vfs_context_t) ctx = 0xffffff804d548840)
    0xffffff9215ecbf10 0xffffff80260afcd7 fstatat_internal((vfs_context_t) ctx = <no location, value may have been optimized out>, , (user_addr_t) path = <no location, value may have been optimized out>, , (user_addr_t) ub = <no location, value may have been optimized out>, , (user_addr_t) xsecurity = <no location, value may have been optimized out>, , (user_addr_t) xsecurity_size = <no location, value may have been optimized out>, , (int) isstat64 = <no location, value may have been optimized out>, , (uio_seg) segflg = UIO_USERSPACE, (int) fd = <no location, value may have been optimized out>, , (int) flag = <no location, value may have been optimized out>, )
    0xffffff9215ecbf40 0xffffff80260b046f stat64((proc_t) p = <>, , (stat64_args *) uap = 0xffffff804d5486e8, (int32_t *) retval = <>, )
    0xffffff9215ecbfa0 0xffffff8026403968 unix_syscall64((x86_saved_state_t *) state = <>, )
    0x0000000000000000 0xffffff8025e1e9c6 kernel`hndl_unix_scall64 + 0x16
    stackbottom = 0xffffff9215ecbfa0

GVFS.Mount:
task                 vm_map               ipc_space            #acts flags    pid       process             io_policy  wq_state  command
0xffffff8057596048   0xffffff806ac60f00   0xffffff8057502ba0      25 D        977   0xffffff806acc3000                -1 -1 -1    GVFS.Mount
[…]
  thread                   thread_id  processor            base   pri    sched_mode      io_policy       state    ast          waitq                            wait_event           wmesg                thread_name
  0xffffff80502da3c0  [WQ] 0x3640     0xffffff82073f6000   31     31     timeshare                       WU       Q            0xffffff81e6a338d0               0xffffff921a253b90   com.avast.FileShield.susp_block
    kernel_stack = 0xffffff921a250000
    stacktop = 0xffffff921a2538f0
    0xffffff921a2538f0 0xffffff802a182ddd machine_switch_context((thread_t) old = 0xffffff80502da3c0, (thread_continue_t) continuation = 0x0000000000000000, (thread_t) new = 0xffffff804d7f5520)
    0xffffff921a253970 0xffffff802a08767a thread_invoke((thread_t) self = 0xffffff80502da3c0, (thread_t) thread = 0xffffff804d7f5520, (ast_t) reason = 0)
    0xffffff921a2539c0 0xffffff802a08663f thread_block_reason((thread_continue_t) continuation = <>, , (void *) parameter = <>, , (ast_t) reason = <>, )
    0xffffff921a253a00 0xffffff802a07b73b thread_block [inlined]((thread_continue_t) continuation = <>, )
    0xffffff921a253a00 0xffffff802a07b730 lck_mtx_sleep_deadline((lck_mtx_t *) lck = 0xffffff804d4f2870, (lck_sleep_action_t) lck_sleep_action = 0, (event_t) event = 0xffffff921a253b90, (wait_interrupt_t) interruptible = 0, (uint64_t) deadline = <>, )
    0xffffff921a253a80 0xffffff802a52f7f8 _sleep((caddr_t) chan = 0xffffff921a253b90 "\x01", (int) pri = 0, (const char *) wmsg = <>, , (u_int64_t) abstime =<PHONE_NUMBER>47, (int (*)(int)) continuation = 0x0000000000000000, (lck_mtx_t *) mtx = 0xffffff804d4f2870)
    0xffffff921a253ac0 0xffffff802a52fdb2 msleep((void *) chan = 0xffffff921a253b90, (lck_mtx_t *) mtx = 0xffffff804d4f2870, (int) pri = 0, (const char *) wmsg = 0xffffff7fad388ac8 "com.avast.FileShield.susp_block", (timespec *) ts = <>, )
    0xffffff921a253b00 0xffffff7fad384e4d com.avast.FileShield + 0x2e4d
    0xffffff921a253b20 0xffffff7fad384f54 com.avast.FileShield + 0x2f54
    0xffffff921a253bf0 0xffffff7fad3857fd com.avast.FileShield + 0x37fd
    0xffffff921a253e40 0xffffff7fad384280 com.avast.FileShield + 0x2280
    0xffffff921a253ea0 0xffffff802a4e8923 kauth_authorize_action [inlined]((kauth_scope_t) scope = <>, , (kauth_cred_t) credential = 0xffffff805eb80630, (kauth_action_t) action = <>, , (uintptr_t) arg0 = <>, , (uintptr_t) arg3 = 0x0000000000000000)
    0xffffff921a253ea0 0xffffff802a4e88a6 kauth_authorize_fileop((kauth_cred_t) credential = 0xffffff805eb80630, (kauth_action_t) action = 438647696, (uintptr_t) arg0 = 0xffffff921a253b90, (uintptr_t) arg1 = 0xffffff804bcee800)
    0xffffff921a253f00 0xffffff802a4f39c3 close_internal_locked((proc_t) p = 0xffffff806acc3000, (int) fd = 124, (fileproc *) fp = 0xffffff804df53f78, (int) flags = 0)
    0xffffff921a253f40 0xffffff802a4f7d84 close_nocancel((proc_t) p = 0xffffff806acc3000, (close_nocancel_args *) uap = <>, , (int32_t *) retval = <>, )
    0xffffff921a253fa0 0xffffff802a603968 unix_syscall64((x86_saved_state_t *) state = <>, )
    0x0000000000000000 0xffffff802a01e9c6 kernel`hndl_unix_scall64 + 0x16
    stackbottom = 0xffffff921a253fa0
[…]

com.avast.daemon:
task                 vm_map               ipc_space            #acts flags    pid       process             io_policy  wq_state  command
0xffffff80498df7a8   0xffffff804afe7300   0xffffff804b1815a0       5 D        231   0xffffff804c4c0490             TQ -1 -1 -1    com.avast.daemon
[…]
  thread                   thread_id  processor            base   pri    sched_mode      io_policy       state    ast          waitq                            wait_event           wmesg                thread_name
  0xffffff804b53a4f0       0x35f6     0xffffff82073e6000   20     20     timeshare       T               WU                    0xffffff81e6a39000               0xffffff9219fa3520   io.gvfs.PrjFSKext.Sleep
    kernel_stack = 0xffffff9219fa0000
    stacktop = 0xffffff9219fa3210
    0xffffff9219fa3210 0xffffff802a182ddd machine_switch_context((thread_t) old = 0xffffff804b53a4f0, (thread_continue_t) continuation = 0x0000000000000000, (thread_t) new = 0xffffff8049f5bda0)
    0xffffff9219fa3290 0xffffff802a08767a thread_invoke((thread_t) self = 0xffffff804b53a4f0, (thread_t) thread = 0xffffff8049f5bda0, (ast_t) reason = 0)
    0xffffff9219fa32e0 0xffffff802a08663f thread_block_reason((thread_continue_t) continuation = <>, , (void *) parameter = <>, , (ast_t) reason = <>, )
    0xffffff9219fa3320 0xffffff802a07b73b thread_block [inlined]((thread_continue_t) continuation = <>, )
    0xffffff9219fa3320 0xffffff802a07b730 lck_mtx_sleep_deadline((lck_mtx_t *) lck = 0xffffff8049bc4990, (lck_sleep_action_t) lck_sleep_action = 0, (event_t) event = 0xffffff9219fa3520, (wait_interrupt_t) interruptible = 0, (uint64_t) deadline = <>, )
    0xffffff9219fa33a0 0xffffff802a52f7f8 _sleep((caddr_t) chan = 0xffffff9219fa3520 "g4", (int) pri = 50, (const char *) wmsg = <>, , (u_int64_t) abstime =<PHONE_NUMBER>40, (int (*)(int)) continuation = 0x0000000000000000, (lck_mtx_t *) mtx = 0xffffff8049bc4990)
    0xffffff9219fa33e0 0xffffff802a52fdb2 msleep((void *) chan = 0xffffff9219fa3520, (lck_mtx_t *) mtx = 0xffffff8049bc4990, (int) pri = 50, (const char *) wmsg = 0xffffff7faa92da72 "io.gvfs.PrjFSKext.Sleep", (timespec *) ts = <>, )
    0xffffff9219fa3430 0xffffff7faa929cf6 Sleep(int, void*, Mutex*)((int) seconds = 5, (void *) channel = 0xffffff9219fa3520, (Mutex *) mutex = 0xffffff7faa934cd8)
    0xffffff9219fa35a0 0xffffff7faa929181 TrySendRequestAndWaitForResponse(short, MessageType, vnode*, FsidInode const&, char const*, int, char const*, int*, int*)((VirtualizationRootHandle) root = 4, (MessageType) messageType = MessageType_KtoU_HydrateFile, (vnode_t) vnode = 0xffffff8061baa5d0, (const FsidInode &) vnodeFsidInode = 0xffffff9219fa3668: {,   fsid = {,     val = ([0] = 16777245, [1] = 24),   },   inode = 1527221, }, (const char *) vnodePath = 0xffffff9219fa3720 "/Users/test/devel/Adium2/src/Plugins/Purple Service/zhm", (int) pid = 231, (const char *) procname = 0xffffff9219fa3700 "com.avast.daemon", (int *) kauthResult = 0xffffff9219fa3694, (int *) kauthError = 0xffffff9219fa3bac)
    0xffffff9219fa3b30 0xffffff7faa9278b6 HandleVnodeOperation(ucred*, void*, int, unsigned long, unsigned long, unsigned long, unsigned long)((kauth_cred_t) credential = 0xffffff804c2f8630, (void *) idata = 0x0000000000000000, (kauth_action_t) action = 2176, (uintptr_t) arg0 = 0xffffff80506554a8, (uintptr_t) arg1 = 0xffffff8061baa5d0, (uintptr_t) arg2 = 0x0000000000000000, (uintptr_t) arg3 = 0xffffff9219fa3bac)
    0xffffff9219fa3b90 0xffffff802a4e8664 kauth_authorize_action((kauth_scope_t) scope = 0xffffff804a107200, (kauth_cred_t) credential = <>, , (kauth_action_t) action = <>, , (uintptr_t) arg0 = 0xffffff80506554a8, (uintptr_t) arg1 = 0xffffff8061baa5d0, (uintptr_t) arg2 = <>, , (uintptr_t) arg3 = <>, )
    0xffffff9219fa3bb0 0xffffff802a2a1939 vnode_authorize((vnode_t) vp = <>, , (vnode_t) dvp = <>, , (kauth_action_t) action = <>, , (vfs_context_t) ctx = <>, )
    0xffffff9219fa3bf0 0xffffff802a2c0ff0 vn_stat((vnode *) vp = 0xffffff8061baa5d0, (void *) sb = 0xffffff9219fa3ca8, (kauth_filesec_t *) xsec = 0x0000000000000000, (int) isstat64 = 1, (vfs_context_t) ctx = 0xffffff80506554a8)
    0xffffff9219fa3f10 0xffffff802a2afcd7 fstatat_internal((vfs_context_t) ctx = <no location, value may have been optimized out>, , (user_addr_t) path = <no location, value may have been optimized out>, , (user_addr_t) ub = <no location, value may have been optimized out>, , (user_addr_t) xsecurity = <no location, value may have been optimized out>, , (user_addr_t) xsecurity_size = <no location, value may have been optimized out>, , (int) isstat64 = <no location, value may have been optimized out>, , (uio_seg) segflg = UIO_USERSPACE, (int) fd = <no location, value may have been optimized out>, , (int) flag = <no location, value may have been optimized out>, )
    0xffffff9219fa3f40 0xffffff802a2b056f lstat64((proc_t) p = <>, , (lstat64_args *) uap = 0xffffff8050655350, (int32_t *) retval = <>, )
    0xffffff9219fa3fa0 0xffffff802a603968 unix_syscall64((x86_saved_state_t *) state = <>, )
    0x0000000000000000 0xffffff802a01e9c6 kernel`hndl_unix_scall64 + 0x16
    stackbottom = 0xffffff9219fa3fa0
[…]

I made the adjustment to not wait for close() after hydration to complete in the provider. (suggestion 2 above) This didn't entirely solve the problem, it only removed one of the 30-second timeouts, so hydrating still took 30 seconds (down from the original 60).
The GVFS.mount stack during this second deadlock looked like this:
  thread                   thread_id  processor            base   pri    sched_mode      io_policy       state    ast          waitq                            wait_event           wmesg                thread_name
  0xffffff802e31c000  [WQ] 0x2346     0xffffff81e3ae5000   31     31     timeshare                       WU                    0xffffff81c3135010               0xffffff81e3b0af20   com.avast.FileShield.susp_block
    kernel_stack = 0xffffff81e3b08000
    stacktop = 0xffffff81e3b0ac80
    0xffffff81e3b0ac80 0xffffff8006982ddd machine_switch_context((thread_t) old = 0xffffff802e31c000, (thread_continue_t) continuation = 0x0000000000000000, (thread_t) new = 0xffffff8029346da0)
    0xffffff81e3b0ad00 0xffffff800688767a thread_invoke((thread_t) self = 0xffffff802e31c000, (thread_t) thread = 0xffffff8029346da0, (ast_t) reason = 0)
    0xffffff81e3b0ad50 0xffffff800688663f thread_block_reason((thread_continue_t) continuation = <>, , (void *) parameter = <>, , (ast_t) reason = <>, )
    0xffffff81e3b0ad90 0xffffff800687b73b thread_block [inlined]((thread_continue_t) continuation = <>, )
    0xffffff81e3b0ad90 0xffffff800687b730 lck_mtx_sleep_deadline((lck_mtx_t *) lck = 0xffffff802937d3a0, (lck_sleep_action_t) lck_sleep_action = 0, (event_t) event = 0xffffff81e3b0af20, (wait_interrupt_t) interruptible = 0, (uint64_t) deadline = <>, )
    0xffffff81e3b0ae10 0xffffff8006d2f7f8 _sleep((caddr_t) chan = 0xffffff81e3b0af20 "\x01", (int) pri = 0, (const char *) wmsg = <>, , (u_int64_t) abstime =<PHONE_NUMBER>81, (int (*)(int)) continuation = 0x0000000000000000, (lck_mtx_t *) mtx = 0xffffff802937d3a0)
    0xffffff81e3b0ae50 0xffffff8006d2fdb2 msleep((void *) chan = 0xffffff81e3b0af20, (lck_mtx_t *) mtx = 0xffffff802937d3a0, (int) pri = 0, (const char *) wmsg = 0xffffff7f89b88ac8 "com.avast.FileShield.susp_block", (timespec *) ts = <>, )
    0xffffff81e3b0ae90 0xffffff7f89b84e4d com.avast.FileShield + 0x2e4d
    0xffffff81e3b0aeb0 0xffffff7f89b84f54 com.avast.FileShield + 0x2f54
    0xffffff81e3b0af80 0xffffff7f89b857fd com.avast.FileShield + 0x37fd
    0xffffff81e3b0b840 0xffffff7f89b849eb com.avast.FileShield + 0x29eb
    0xffffff81e3b0b8a0 0xffffff8006ce8664 kauth_authorize_action((kauth_scope_t) scope = 0x00000000000186a0, (kauth_cred_t) credential = <>, , (kauth_action_t) action = <>, , (uintptr_t) arg0 = 0xffffff81e3b0af20, (uintptr_t) arg1 = 0xffffff8039eea458, (uintptr_t) arg2 = <>, , (uintptr_t) arg3 = <>, )
    0xffffff81e3b0b8f0 0xffffff8006aa1ae4 vnode_authorize [inlined]((vnode_t) vp = 0xffffff8039eea458, (kauth_action_t) action = <>, )
    0xffffff81e3b0b8f0 0xffffff8006aa1aae vn_authorize_open_existing((vnode_t) vp = 0xffffff8039eea458, (componentname *) cnp = 0xffffff81e3b0bd28, (int) fmode = <>, , (vfs_context_t) ctx = 0xffffff802d833e50, (void *) reserved = <>, )
    0xffffff81e3b0b980 0xffffff8006ac04e2 vn_open_auth((nameidata *) ndp = 0xffffff81e3b0bbd8, (int *) fmodep = 0xffffff81e3b0b9d4, (vnode_attr *) vap = 0xffffff81e3b0bd70)
    0xffffff81e3b0bbc0 0xffffff8006aab58f open1((vfs_context_t) ctx = 0xffffff802d833e50, (nameidata *) ndp = 0xffffff81e3b0bbd8, (int) uflags = <>, , (vnode_attr *) vap = 0xffffff81e3b0bd70, (fp_allocfn_t) fp_zalloc = <>, , (void *) cra = 0xffffff802d833e50, (int32_t *) retval = 0xffffff802d833d38)
    0xffffff81e3b0bf40 0xffffff8006aac117 open1at [inlined]((fp_allocfn_t) fp_zalloc = <>, , (void *) cra = <>, )
    0xffffff81e3b0bf40 0xffffff8006aac0f7 openat_internal [inlined]((vfs_context_t) ctx = <>, , (user_addr_t) path = <>, , (int) flags = <>, , (int) mode = <>, , (int) fd = -2, (uio_seg) segflg = UIO_USERSPACE)
    0xffffff81e3b0bf40 0xffffff8006aac04a open_nocancel((proc_t) p = <>, , (open_nocancel_args *) uap = <>, , (int32_t *) retval = 0xffffff802d833d38)
    0xffffff81e3b0bfa0 0xffffff8006e03968 unix_syscall64((x86_saved_state_t *) state = <>, )
    0x0000000000000000 0xffffff800681e9c6 kernel`hndl_unix_scall64 + 0x16
    stackbottom = 0xffffff81e3b0bfa0

I've now implemented proposed solution 1 in addition to 2, which has fixed the problem with Avast in my tests. I've pushed what I've got so far to PR #798.

Incidentally, I'm seeing a lot of hydrations due to stat()/lstat() syscalls. Someone remind me why we need to fully hydrate on KAUTH_VNODE_READ_ATTRIBUTES and why we can't prime empty files with the correct metadata?

Here's some info on why we hydrate on KAUTH_VNODE_READ_ATTRIBUTES from a discussion in the old repo/over email:
We had issues with pre compiled header files that are built during a large repo build being invalidated by our hydration, which would fail the build.
For any project that built a .pch, whenever the next item was built in that target, we would error out with an error like:
file /some/header/in/our/virtualization/root.h has been modified since the precompiled header /BuildOutput/some/project/foo.pch was built
What would happen is that clang would stat every header that would be part of a pch and would put these stat values into a data structure within clang. Once the data structure is built, it would then open the headers (triggering hydration) and build the pch. Whenever the pch is referenced as part of the build further down the line, clang compares the contents of the cached stat structure to the headers on disk. Since we hydrated the files, we moved the timestamps forward from what they were in that data structure, triggering the above error. We made a choice to not try and play with timestamps to let them always roll forward rather than trying to set them backwards. On Windows, we made the same choice and I believe similar products have as well.
If you want to dig into the clang source to understand this behavior, most of the interesting logic is in this class: http://clang.llvm.org/doxygen/classclang_1_1FileSystemStatCache.html

We had issues with pre compiled header files that are built during a large repo build being invalidated by our hydration, which would fail the build.

Thank you for reminding me of that - I knew you'd explained it before but for the life of me could neither remember it nor locate your original answer.

Copying the comment over from #798

So how about we look at the process's ancestry in addition to the user? If the process owner doesn't match the repo's mounter, we could walk up the process hierarchy and see if we find a process that is owned by the mounter. Then the chances are extremely high it's a user-initiated process and should be allowed to hydrate.

I think walking up the process hierarchy sounds like a good approach, but I don't think should limit hydration based on what user performed the mount (e.g. a user might log into a machine to inspect the state of a repo that was mounted by a build agent).
In offline discussions with other kext authors I've heard that the approach they've taken when deciding to hydrate is based on root vs. non-root rather than looking for specific users:

Non-root processes. Obviously these are ok.
Root processes with a non-root parent. These are processes that have been sudo'ed. These are ok.
Root processes with all parents also running as root. These are system processes and are blocked.

And from what I've heard this approach has worked well for them.  Could we try going with this same approach?

 In other words, after hydrating, can we reset the mtime to what it was before hydrating and thereby avoid hydrating on *stat*()? There should be no race of any kind here as we halt all I/O on the file while hydration is in progress anyway.

We might be able to get away with setting the time back today, but if we were to stop blocking READ_ATTRIBUTES then I think we'd have a race where READ_ATTRIBUTES crosses with a hydration happening in another thread and the READ_ATTRIBUTES reads the newer mtime (before we've set it back).
The other consideration was that typically files that have their attributes read usually also have their contents read, and so hydrating for READ_ATTRIBUTES prevents any mtime related issues (and the files likely would have been hydrated anyway).
Note also that the decision to hydrate during READ_ATTRIBUTES was made with the assumption that AV and scanners would not be expanding empty placeholder directories, and so we'll probably want to block those processes from hydrating files and from expanding un-expanded directories.
/cc: @jeschu1 @nickgra

I think walking up the process hierarchy sounds like a good approach, but I don't think should limit hydration based on what user performed the mount (e.g. a user might log into a machine to inspect the state of a repo that was mounted by a build agent).

The reason I restricted it further like this was that I was concerned that AV(-like) software might be running their daemons as non-root, but thinking about this some more that seems unlikely due to permissions issues with opening files for scanning. The only other potentially relevant fact I'll throw out there is that user IDs below 500 are reserved for system services/daemons. So potentially we could apply it to users <500 instead of just user 0.

Non-root processes. Obviously these are ok.

I'm not sure this is obvious at all, there could be kauth kexts paired with non-root processes. Such as ours. To be fair, I'm having a hard time coming up with one that we might conflict with. Has anyone tried the (not very wise but totally plausible that a user might do this) case of cloning a VFS4G repo into a OneDrive, Dropbox, etc. folder?

And from what I've heard this approach has worked well for them. Could we try going with this same approach?

It sounds pretty sane and should certainly work for the specific case we've run into here.

We might be able to get away with setting the time back today, but if we were to stop blocking READ_ATTRIBUTES then I think we'd have a race where READ_ATTRIBUTES crosses with a hydration happening in another thread and the READ_ATTRIBUTES reads the newer mtime (before we've set it back).

True; solvable with the vnode cache though, incidentally.

The other consideration was that typically files that have their attributes read usually also have their contents read, and so hydrating for READ_ATTRIBUTES prevents any mtime related issues (and the files likely would have been hydrated anyway).

Did checking the ACCESS action flag solve the hydration-by-Finder issue? If not, this probably would, as the Finder probably calls stat() if that's still an issue.

Note also that the decision to hydrate during READ_ATTRIBUTES was made with the assumption that AV and scanners would not be expanding empty placeholder directories, and so we'll probably want to block those processes from hydrating files and from expanding un-expanded directories.

OK, good point, in the next iteration I'll make sure to prevent expanding directories where hydration would be disallowed.

The only other potentially relevant fact I'll throw out there is that user IDs below 500 are reserved for system services/daemons. So potentially we could apply it to users <500 instead of just user 0.

Interesting!  Do you have a recommendation as to which approach we start with (0 or <500)?

Has anyone tried the (not very wise but totally plausible that a user might do this) case of cloning a VFS4G repo into a OneDrive, Dropbox, etc. folder?

Good point, and I don't think anyone has tried this (I'm not sure what it's behavior on Windows would be either).  I also don't think this is a scenario we need/want to support, and so unless the behavior is really bad (e.g. kernel panics, the entire system permanently hanging), I don't think we need to add code to support the scenario.
/cc: @jrbriggs

True; solvable with the vnode cache though, incidentally.

No need to dig deep into this discussion here, but out of curiosity how do you imagine the vnode cache being able to help?  Were you thinking it could be used as a synchronization mechanism for READ_ATTRIBUTES and hydrating a file?

Did checking the ACCESS action flag solve the hydration-by-Finder issue? If not, this probably would, as the Finder probably calls stat() if that's still an issue.

The main issue we had with Finder so far was that its KAUTH_VNODE_DELETE check was getting interpreted as a "PreDelete" event, and every file in the repo would be added to ModifedPaths.dat.
The ACCESS flag check solved the VNODE_DELETE issue, but it did not impact Finder's hydration behavior.
This is a good point to keep in mind though if we find that Finder's hydration ends up leading to a poor user experience.

OK, good point, in the next iteration I'll make sure to prevent expanding directories where hydration would be disallowed.

Sounds good 👍

Has anyone tried the (not very wise but totally plausible that a user might do this) case of cloning a VFS4G repo into a OneDrive, Dropbox, etc. folder?

Good point, and I don't think anyone has tried this (I'm not sure what it's behavior on Windows would be either). I also don't think this is a scenario we need/want to support, and so unless the behavior is really bad (e.g. kernel panics, the entire system permanently hanging), I don't think we need to add code to support the scenario.

I've not tested this scenario but it's on my backlog of things I want to get to eventually.
I agree that we should do the bare minimum to work around potential user error like making a VFS4G clone in a OneDrive folder, but I do want to call out that we need to support scenarios where OneDrive's kernel extension is loaded and running with their equivalent of a virtualization root mounted while a VFS4G virtualization root is mounted in a separated directory.

The only other potentially relevant fact I'll throw out there is that user IDs below 500 are reserved for system services/daemons. So potentially we could apply it to users <500 instead of just user 0.

Interesting! Do you have a recommendation as to which approach we start with (0 or <500)?

It probably won't make much of a difference either way, the chances of a non-root daemon trying to access files in a VFS4G repo are pretty slim. Still, there are a few spotlight-related processes running as the _spotlight user for example, so I think I might try <500 and see what happens.

Good point, and I don't think anyone has tried this (I'm not sure what it's behavior on Windows would be either). I also don't think this is a scenario we need/want to support, and so unless the behavior is really bad (e.g. kernel panics, the entire system permanently hanging), I don't think we need to add code to support the scenario.

It could very well cause a permanent hang; the only reason the AV deadlock got itself unstuck was because of a timeout in the AV kext. So I think this is worth checking out.

True; solvable with the vnode cache though, incidentally.

No need to dig deep into this discussion here, but out of curiosity how do you imagine the vnode cache being able to help? Were you thinking it could be used as a synchronization mechanism for READ_ATTRIBUTES and hydrating a file?

More or less; I'd probably put a boolean flag on each cache entry indicating that all non-provider I/O on the vnode should be blocked until the provider replies (or we hit a timeout).

The main issue we had with Finder so far was that its KAUTH_VNODE_DELETE check was getting interpreted as a "PreDelete" event, and every file in the repo would be added to ModifedPaths.dat.
The ACCESS flag check solved the VNODE_DELETE issue, but it did not impact Finder's hydration behavior.

OK, makes sense, thanks.


APEXMALHAR-1950 #resolve #comment added stable annotation to semantic…

… version inclusion list

@tweise please review and merge


Question about "well known" database guid

Looking at this unit test expected output I have a question about the following database guid.
https://github.com/stephenegriffin/mfcmapi/blob/48c2a6becf0248076848025c3711cb1b2eb7ee45/UnitTest/SmartViewTestData/Out/4-eid-out10.dat#L5
I can't find it inside exchange protocols, windows protocols or office protocols but it seems like a constant we can find in our outlook registries (Perf/RoamingStreamsCache). When outlook cancels a search done with MS-OXWSOLPS it re-submits the search on a folder with this database guid.
Can you point me to what this database guid refers to or why you added a test for this specific message object entry id ?

why you added a test for this specific message object entry id ?
I grabbed a random eid from my mailbox to build a test around. :) The data for that test has been in my private tests for years so I don't really know what it pointed to.


Bump deps to kube 1.14

bump deps(*)
updates due to make generate after dep bump

/assign @adambkaplan @dmage

Created https://github.com/openshift/release/pull/4033

/lgtm


Weird sidebar icon with no content appearing after installing the extension

👓 What did you see?
A messed up sidebar icon with a messed up name and no content
✅ What did you expect to see?
Some kind of sidebar or no sidebar at all... I don't know if the sidebar is intended at all ('cause it doesn't show up anywhere in the README)
📦 Which tool/library version are you using?
Cucumber 1.2.8
🔬 How could we reproduce it?
Steps to reproduce the behavior:

Install Cucumber 1.2.8 on VSCode 1.69.2 for MacOs Monterey
Reload VScode
Open a project either locally or via the popular Remote - Containers extension
Look for a generic sidebar icon
Click on the icon
Acknowledge that the sidebar's metadata and content are not displayed

The Cucumber extension doesn't have a sidebar, so I'm not sure this has anything to do with this plugin.

If I disable the Cucumber plugin, those sidebar button and pane go away altogether.

I tried to drag that view around and, as soon as I did so, the view's metadata and content automagically got populated (its name is "Source Control Repositories"). I have no idea why it kept showing up with those incorrect metadata at launch, but it's now fixed.


[amazonlinux] fix changelogTemplate and fix release date

fixes latest release date
fixes changelogtemplate [ so it wont redirects again and again ]

ref: #1206


bingeWatch chrome extension

Is your extension request related to a problem? Please describe.
It synchronises video playback and adds group voice call and chat, gives the feel of theatre.synchronizes
Describe the solution you'd like.
Sync Movies & Shows in HD.
Always stay precisely in sync when you're watching shows together. Enjoy HD video quality and fast buffering in all of your Binge parties.
Instant Connect.
Create parties in seconds right on the Netflix.com website to watch thousands of great Netflix shows together with friends.
Group Voice call.
While the video is playing, users can socialize with other participants through a built-in chat feature that supports both call and text with built-in emojis.
Describe alternatives you've considered.
NO alternatives at this point
Add any other context or screenshots about the extension here.
No response

Can I get assigned to this issue under JWOC ?


Change update_deps.sh to use docker (default) or crane

Please provide a description of this PR:
This change is to better support developers on MacOS on silicon devices.
On Apple devices, commands like sed and grep behave differently. Some of the Istio scripts, like update_deps.sh, use these commands and fail when run on the MacOS. One can install software to make the commands work as they do on Linux. Another option is to simply to do make shell  to get a Linux command line. From this Linux command line, the scripts can be run.
One issue right now is that the crane command segfaults when run in the make shell container. An issue with qemu?
This PR reverts a change that solely uses crane in update_deps.sh, instead allowing the developer to export ISTIO_DOCKER_BUILDER=crane to use crane. I made docker the default to mimic the behavior in docker-builder.

Adding a DNM to allow @GregHanson to test this on his Apple Silicon computer.

This feels a bit wrong to me... do we expect arm machines to be able to run the build container successfully? I cannot imagine it works (or works well, at least) in many other places.
At the very least we should continue to use the proper way (crane) instead of hacking around it just for ARM.
Maybe its time to get multiarch build-tools

With qemu, there is a possibility of running intel/amd containers on arm machines. This works for Docker Desktop and podman. While I agree this isn't ideal and we should have arm versions of the build-tools and build arm images, that will be a longer process.

I'd like to leave this one set a little longer.

Closing this PR for now. Will reopen if the users are still seeing issues with crane on MacOS Silicon devices.


Mock observations model

Types of changes

[ ] Enhancement (project structure, spelling, grammar, formatting)
[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

A description of the changes proposed in the Pull Request
MockObservations model generates mock observations for testing and demo purposes

closed PR since mock-observations model is going into the official models repo


REST > Added public endpoints

@Ninjeneer Let me know if I can deploy the backend with the current version of the mobile in prod 2.0.16 or if I have to deploy the mobile first.
Thanks!

@Ninjeneer Let me know if I can deploy the backend with the current version of the mobile in prod 2.0.16 or if I have to deploy the mobile first. Thanks!

Backend can be deployed with the current mobile version. Changes in this PR does not affect yet the HTTP calls


Docker Registry DDOS prevention

When dealing with images beyond a very small size it is quite easy to cause network congestion or perform distributed denial of service against Docker Registries housing algorithms. With smaller clusters this isn't as problematic, but once the size moves into double / triple digits, it becomes a serious issue.
In newer versions of DCOS (1.9+) image garbage collection is on by default, which conflicts with Scale's management of Docker images. Scale intends to pull all images locally and then be able to rely on that node local cache, but with DCOS garbage collecting we are left with a constant re-pulling of images to as job executions are launched.
The proposed solution is for Scale to manage individual mirrors for each registry backend where images are cached internally to the cluster. This would eliminate the need to repeatedly pull images to each node in the cluster and provide a central point within the cluster for storage.

Discussion on the DCOS mailing list should be referenced for possible solutions we can either recommend or integrate as a subsystem of Scale: https://groups.google.com/a/dcos.io/forum/#!topic/users/d5lFlUjwwC0

This is a high priority to eliminate the need for managing images per node. It would allow us to eliminate scale pull tasks in favor of a cluster local cache. This will likely be a requirement for certain production environments that operate with Docker GC in place.


(minor fix) document that aws-exports.js is meant to be public

(minor fix) document that aws-exports.js is meant to be public
Description of changes:
To help users be more at ease with security concerns of aws-exports.js, placing link to docs and "how to use" directly inside the opening comments, instead of forcing them to break flow and look it up on our site.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

horrendously out of date