🐛 Camera stretched on Android

What's happening?
Camera is stretched on Android, behavior is not consistent as sometimes when closing the navbar and reopening the stretch is gone.
.
issue reproduced on these two devices:

Samsung Galaxy S9 SM G960F, Android version 8
OnePlus 5T, Android version 10

Reproduceable Code
const device = useCameraDevice('back');
const format =  getCameraFormat(device, [
        { resizeTo: { width: camContainerPosition?.width ?? '', height:
        camContainerPosition?.height ?? '' } },
        { photoResolution: 'max' },
        { videoResolution: 'max' },
        { autoFocusSystem: 'contrast-detection' },
        { videoStabilizationModes: ['on', 'on', 'cinematic-extended'] }
    ]);
    const fps = format.maxFps >= 240 ? 240 : format.maxFps;
    const touchToFocus = async (event) => {
        if (!device.supportsFocus || !isCameraFocusReady.current) return;
        let point = {
            x: Math.round(event.pageX - camLocation.x),
            y: Math.round(event.pageY - camLocation.y)
        };
        try {
            isCameraFocusReady.current = false;
            await cameraRef?.current?.focus(point);
            setTimeout(() => {
                isCameraFocusReady.current = true;
            }, 200);
        } catch (error) {

        }
    };
 <Camera
   format={format}
   isActive={isAndroid ? !paused : true}
   photo={true}
   orientation={'portrait'}
   ref={cameraRef}
   enableHighQualityPhotos={true}
   fps={fps}
   preset='photo'
   enableZoomGesture={true}
   onLayout={handleCameraLayout}
   device={device}
   zoom={device.neutralZoom}
   style={{ flex: 1 }}
   onTouchEnd={(x) => touchToFocus(x.nativeEvent)}
   />

Relevant log output
no logs

Camera Device
{"sensorOrientation":
"landscape-right",
"hardwareLevel": "full",
"maxZoom": 8,
"minZoom":1,
"supportsLowLightBoost":false,
"neutralZoom":1,
"physicalDevices":["wide-angle-camera"],
"supportsFocus":true,
"supportsRawCapture" true,
"isMultiCam":false,
"name":"BACK(0)",
"hasFlash" true,
"has Torch" true,
"position":"back",
"id":"0"}

Device
OnePlus 5T, Android version 10
VisionCamera Version
3.6.4
Can you reproduce this issue in the VisionCamera Example app?
I didn't try (⚠ your issue might get ignored & closed if you don't try this)
Additional information

[X] I am using Expo
[ ] I have enabled Frame Processors (react-native-worklets-core)
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar issues in this repository and found none.

@mrousavy Sponsoring this one as well :)

Could you check if format is undefined?

@rsnr thank you!! ❤

behavior is not consistent as sometimes when closing the navbar and reopening the stretch is gone.

@NuraQ so if you minimize the app and open it again, it looks correct? Then it's definitely a race condition in PreviewView's set size funcs.
I'll investigate this soon!

@mrousavy  After testing on two devices, this was the result:

on One Plus, it was consistent.
"Minimizing" by the right button, or sending to background by the middle button, did not have impact (still camera is stretched).

On Samsung, I think the back middle button doesn't always behave the same in terms of impact no stretching ( it some times fixes stretch issue and sometimes it doesn't).

okok lemme take a look over the weekend or next week, its a race condition

Hey,
I've just started integrating react-native-vision-camera v3 into my project (i'm currently using react-native-camera) and encountered some issues related to aspect ratio on Android (similar to the problem described in this opened issue).
Usage
I am using the resizeMode='contain' mode to display the entire preview:
<Camera
    ref={this.cameraRef}
    style={{ position: 'absolute', top: 0, left: 0, right: 0, bottom: 0 }}
    device={device}
    format={format}
    isActive={true}
    photo={isPhoto}
    video={isVideo}
    audio={isVideo}
    resizeMode='contain'
/>

Tested on:

HUAWEI P30 (ELE-L29) with Android 10
react-native: 0.72.4
react-native-vision-camera: 3.6.10

Issues
Observed Issues when I switch from one format to another:

The view is never resized (the call to requestLayout() doesn't seem to work).
The aspect ratio is not always correct after a format change.

For the first issue, it appears to be a problem related to react-native.
I found a solution on this GitHub issue: https://github.com/facebook/react-native/issues/17968
Here is the patch to fix the issue:
diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt
index 0aa8932..7b1c91e 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraView.kt
@@ -242,4 +242,17 @@ class CameraView(context: Context) :
   override fun onCodeScanned(codes: List<Barcode>, scannerFrame: CodeScannerFrame) {
     invokeOnCodeScanned(codes, scannerFrame)
   }
+
+  override fun requestLayout() {
+    super.requestLayout()
+    post(measureAndLayout)
+  }
+
+  private val measureAndLayout = Runnable {
+    measure(
+        MeasureSpec.makeMeasureSpec(width, MeasureSpec.EXACTLY),
+        MeasureSpec.makeMeasureSpec(height, MeasureSpec.EXACTLY)
+    )
+    layout(left, top, right, bottom)
+  }
 }

For the second issue, it occurs when the surface is updated ("PreviewView Surface updated!").
The ratio is not necessarily updated, and I don't know why.
The only solution I found is to restart the entire device configuration.
Here is the patch to fix the issue:
diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
index 8ad60cc..15b85ff 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
@@ -187,6 +187,19 @@ class CameraSession(private val context: Context, private val cameraManager: Cam

         override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {
           Log.i(TAG, "PreviewView Surface updated! ${holder.surface} $width x $height")
+
+          launch {
+            mutex.withLock {
+              try {
+                configuration?.let { configuration ->
+                  configureOutputs(configuration)
+                  configureCaptureRequest(configuration)
+                }
+              } catch (error: Throwable) {
+                Log.e(TAG, "Failed to configure CameraSession! Error: ${error.message}", error)
+              }
+            }
+          }
         }

         override fun surfaceDestroyed(holder: SurfaceHolder) {

However, this is not perfect, and the camera does not load instantly (refer to the 3rd video where you can see the incorrect ratio, and then 1 or 2 seconds later, the correct ratio after reloading the camera configuration).
@mrousavy > Do you have any idea where this issue might be coming from? I've tried various approaches but haven't found an alternative fix. I'm available to experiment with other solutions if needed.
Videos
To illustrate, here are videos of my application at different stages of the fixes:
Without any fix:
https://github.com/mrousavy/react-native-vision-camera/assets/78229184/98129bb6-8a15-4ff4-a79a-e0d8b30ceb7d
With the fix for issue 1:
https://github.com/mrousavy/react-native-vision-camera/assets/78229184/0d2e47d8-04ed-4c4e-b1c8-31895babd88e
With the fix for issue 2:
https://github.com/mrousavy/react-native-vision-camera/assets/78229184/6458f9e9-b6b6-4273-a4b5-58ea4ec72dc8

Hey - thanks for the insights man! I need to take a look at this, probably just a small fix to avoid any race conditions here. Will have some time soon, maybe in 1-2 weeks

Has there been any progress on this? Still experiencing the issue

Not sure if this helps, but when displaying an image taken with the camera, I found that I had to swap the height and width properties returned from the photo object.
const photo = await cameraRef.current.takePhoto()

Here is a (dirty) quickfix that helps me:
Add the line:
private val aspectRatio: Float get() = size.width.toFloat() / size.height.toFloat()
to the class PreviewView in
react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt.
Then change the methode onMeasure in the same file to:
@SuppressLint("DrawAllocation")
override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
  super.onMeasure(widthMeasureSpec, heightMeasureSpec)
  val viewWidth = MeasureSpec.getSize(widthMeasureSpec)
  val viewHeight = MeasureSpec.getSize(heightMeasureSpec)

  Log.i(TAG, "PreviewView onMeasure($viewWidth, $viewHeight)")

  val newWidth: Int
  val newHeight: Int

  val actualRatio = if (viewWidth > viewHeight) aspectRatio else 1f / aspectRatio
  if (viewWidth < viewHeight * actualRatio) {
    newHeight = viewHeight
    newWidth = (viewHeight * actualRatio).roundToInt()
  } else {
    newWidth = viewWidth
    newHeight = (viewWidth / actualRatio).roundToInt()
  }

  Log.d(TAG, "Measured dimensions set: $newWidth x $newHeight")
  setMeasuredDimension(newWidth, newHeight)
}

Here is a (dirty) quickfix that helps me:
Add the line:
private val aspectRatio: Float get() = size.width.toFloat() / size.height.toFloat()
to the class PreviewView in react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt.
Then change the methode onMeasure in the same file to:
@SuppressLint("DrawAllocation")
override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
  super.onMeasure(widthMeasureSpec, heightMeasureSpec)
  val viewWidth = MeasureSpec.getSize(widthMeasureSpec)
  val viewHeight = MeasureSpec.getSize(heightMeasureSpec)

  Log.i(TAG, "PreviewView onMeasure($viewWidth, $viewHeight)")

  val newWidth: Int
  val newHeight: Int

  val actualRatio = if (viewWidth > viewHeight) aspectRatio else 1f / aspectRatio
  if (viewWidth < viewHeight * actualRatio) {
    newHeight = viewHeight
    newWidth = (viewHeight * actualRatio).roundToInt()
  } else {
    newWidth = viewWidth
    newHeight = (viewWidth / actualRatio).roundToInt()
  }

  Log.d(TAG, "Measured dimensions set: $newWidth x $newHeight")
  setMeasuredDimension(newWidth, newHeight)
}

Thanks, it works for me

It's a race condition, I fixed it on JS side by init the camera dimension at zero, settimeout waiting for the camera launch, and then resizing the camera view.

i am also have this issue but when i make orientation lock to landscape

@JuMaruhn's fix partially fixed my issue - it made the preview properly fill its container (I was getting blank bars on the sides), but for me the preview was still distorted.
I too experienced some inconsistency, sometimes it would fix itself by flipping between front/back, minimising app, etc.
I was able to workaround this issue by not feeding in format straight away, and instead doing this:
function CameraPreview() {
  const [isInitialised, setIsInitialised] = useState(false);

  return (
    <View onLayout={() => setIsInitialised(true)}>
      <CameraView
        format={
          !isInitialised && Platform.OS === "android" ? undefined : format
        }
        resizeMode={"contain"}
        style={{
          aspectRatio: 3 / 4,
        }}
        {...yourOtherProps}
      />
    </View>
  );
}

This I think is very similar to @j-jonathan's approach except I think it's a little lighter/quicker than reinitialising the entire camera config.

@Jigar2311 I had the same problem in landscape mode and the fix of @JuMaruhn fixed my problem.

Sure @Alexis200007, let me see if it works for me

I have the same problem in the version 3.7.0
In version 3.5.1 there was no this problem
I have tried to solve the problem by updating the size in the camera style 250 milliseconds after the component has been mounted
but I don't like this
...
const [adjustCameraSize, setAdjustCameraSize] = useState(false);

useEffect(() => {
    if (device && isFocussed && cameraPermission === Permissions.RESULTS.GRANTED) {
      setTimeout(() => {
        setAdjustCameraSize(true);
      }, 250);
    }
  }, [isFocussed, device, cameraPermission]);

...

<Camera
       style={[{ ...StyleSheet.absoluteFill }, { width: adjustCameraSize ? '100%' : '120%' }]}
       device={device}
        isActive
        codeScanner={codeScanner}
      />

Hey! Does this fix your issue maybe? https://github.com/mrousavy/react-native-vision-camera/pull/2377 (still testing)

Hey!
After 8 hours of debugging, I finally found the culprit! I fixed the preview stretching issue in this PR: https://github.com/mrousavy/react-native-vision-camera/pull/2377
If you appreciate my dedication and free support, please consider 💖 sponsoring me on GitHub 💖 so I can keep providing fixes and building out new features for my open-source libraries in my free time.

I am still experiencing the same with 3.7.1, the preview is stretched.

The same with 3.8.2, the preview is stretched.

@JuMaruhn's fix partially fixed my issue - it made the preview properly fill its container (I was getting blank bars on the sides), but for me the preview was still distorted. This was on a Samsung Galaxy S22.
I too experienced some inconsistency, sometimes it would fix itself by flipping between front/back, minimising app, etc.
I was able to workaround this issue by not feeding in format straight away, and instead doing this:
function CameraPreview() {
  const [isInitialised, setIsInitialised] = useState(false);

  return (
    <View onLayout={() => setIsInitialised(true)}>
      <CameraView
        format={
          !isInitialised && Platform.OS === "android" ? undefined : format
        }
        resizeMode={"contain"}
        style={{
          aspectRatio: 3 / 4,
        }}
        {...yourOtherProps}
      />
    </View>
  );
}

This is very similar to @j-jonathan's approach except I think it's a little lighter/quicker than reinitialising the entire camera config.

it works for me

it works for me. Thanks

huntfpoly12 Works for me as well as a temp. fix

in my case I'm not specifying the format prop and I'm still getting the issue
"react-native-vision-camera": "^3.8.2"

I am also not specifying the format prop and none of the fixes work for me on a Nokia G22. Whatever settings I use the height is slightly stretched, switching between camera and app doesn't fix either. My app is hardcoded to be portrait only.

Still happening on 3.8.2. None of the temp fixes work reliably for me. I'm sponsoring this issue @mrousavy 🙌

I'm having this issue also.

Thanks @grantsingleton! Will look into this soon!

Its definitely a race condition because whilst developing other areas of the app when I have 'completed' my work flow and passed back to the camera screen its fine. Weirdly, it actually flips between 3, stretched height, really stretched height (square becomes long rectangle) and normal.

Same issue after upgrade to 3.8.2.

Same issue :/

We are also experiencing this issue. We enormously respect and love what you @mrousavy are doing so our small company became a sponsor. If you have any time this issue would have our preference.

Thanks @Dingenis, which company?
I'll take a look at this soon 🙏

Hey @mrousavy I'm a relatively new sponsor, so I'm not certain how asking for issue prioritisation works. If it's just asking here in the comments, then I'd really appreciate if you could focus on this issue too. Thank you!

Thanks for the sponsorship @Menardi, I'll take a look soon! 🙏

I really don't want to sound ungrateful or rude here, but this issue is widespread and totally broken for all android users. Its a massive issue. I am not sure how you don't have it in your demo app. I also really feel like this is the only repo I have ever seen where in the issues are target sponsorship amounts.
Why not make this require a license where people have to pay for it instead if the issue is your time and needing to be funded? I completely agree that time is not free so I don't expect it for nothing.
The thing which stops me sponsoring is seeing your reply's say you will do it soon, what is soon, what am I sponsoring? Will it be this month, this year?

Thanks @Dingenis, which company?
I'll take a look at this soon 🙏

Hey @mrousavy, that's great! Thank you! Our company is Spaces Experiences.

Unfortunately this is still occurring even on the new beta release, albeit differently. Before the preview was stretched horizontally, but it now stretches vertically, then on refreshing the camera it stretches horizontally.
Interestingly, if I remove the videoAspectRatio prop from the useCameraFormat hook, it still stretches vertically on initialisation, but on refresh it is then normal.

Created a PR to fix this issue once and for all: https://github.com/mrousavy/react-native-vision-camera/pull/2519 💪🚀
Thanks to everyone who sponsored me on GitHub to fix this bug!! 💖
Try the PR and see if it works (by checking out that branch and running the example app)

Thanks for your hard work @mrousavy! I've tested this fix in the example app on a Pixel 7a (Android 14), and a Pixel 3a (Android 12) and the view is no longer stretched when using the fix/preview-stretching branch. 🙌
I did notice one unexpected side effect on the 7a, which is that the default zoom level seems to be much more zoomed in on the fixed branch. I didn't see this behaviour on the 3a, so I presume it must be something to do with the 7a having two rear cameras. Setting the prop zoom={device.neutralZoom} fixed this so the zoom level was consistent between main and fix/preview-stretching.
I don't think it's a bug, but thought it was worth mentioning as it might come up once the fix is released. I've included some photos taken on each branch below if you're interested.
Either way, thanks very much for the fix!

Screenshots showing default zoom differences (Pixel 7a)

Default zoom: main
Default zoom: fix/preview-stretching

Thanks for your hard work @mrousavy! I've tested this fix in the example app on a Pixel 7a (Android 14), and a Pixel 3a (Android 12) and the view is no longer stretched when using the fix/preview-stretching branch. 🙌

Great to hear!!! 💪

I did notice one unexpected side effect on the 7a, which is that the default zoom level seems to be much more zoomed in on the fixed branch. I didn't see this behaviour on the 3a, so I presume it must be something to do with the 7a having two rear cameras. Setting the prop zoom={device.neutralZoom} fixed this so the zoom level was consistent between main and fix/preview-stretching.

Huh, interesting. Maybe this has something to do with me not resetting the zoom on device change properly, but it is weird nevertheless... 🤔

Hello! I added version 3.9.0-beta.2 and now I am experiencing a different phenomenon. Before, the problem was that the camera preview stretched vertically, now the problem is that it stretches horizontally.
Photo from React native vision camera

Photo in normal conditions

Any suggestions or recommendations you can give me so this will work on Android? Here is my component.
    <Camera
      ref={cameraRef}
      style={StyleSheet.absoluteFill}
      device={device}
      isActive={isActive}
      photo={true}
    />

Hello! I added version 3.9.0-beta.2 and now I am experiencing a different phenomenon. Before, the problem was that the camera preview stretched vertically, now the problem is that it stretches horizontally.
Photo from React native vision camera

Photo in normal conditions

Any suggestions or recommendations you can give me so this will work on Android? Here is my component.
    <Camera
      ref={cameraRef}
      style={StyleSheet.absoluteFill}
      device={device}
      isActive={isActive}
      photo={true}
    />

same issue here on my old Galaxy J5

Hello! I added version 3.9.0-beta.2 and now I am experiencing a different phenomenon. Before, the problem was that the camera preview stretched vertically, now the problem is that it stretches horizontally.

@mrousavy I'm currently with this exactly same issue but looking into 3470b4f I can see that you inverted some width/height props, reverting these props to correct width/height order all stretching issues are gone.
Here's a patch I made:
diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
index fef6998..c339ea7 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/CameraSession.kt
@@ -313,7 +313,7 @@ class CameraSession(private val context: Context, private val cameraManager: Cam
       )
       outputs.add(output)
       // Size is usually landscape, so we flip it here
-      previewView?.setSurfaceSize(size.width, size.height)
+      previewView?.setSurfaceSize(size.height, size.width)
     }

     // CodeScanner Output
diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt
index c862be7..2d26fe9 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/core/PreviewView.kt
@@ -43,13 +43,13 @@ class PreviewView(context: Context, callback: SurfaceHolder.Callback) : SurfaceV
     holder.addCallback(callback)
   }

-  suspend fun setSurfaceSize(width: Int, height: Int) {
+  suspend fun setSurfaceSize(height: Int, width: Int) {
     withContext(Dispatchers.Main) {
-      size = Size(width, height)
+      size = Size(height, width)
       Log.i(TAG, "Setting PreviewView Surface Size to $size...")
       requestLayout()
       invalidate()
-      holder.resize(width, height)
+      holder.resize(height, width)
     }
   }

@@ -73,7 +73,7 @@ class PreviewView(context: Context, callback: SurfaceHolder.Callback) : SurfaceV

   private fun getSize(contentSize: Size, containerSize: Size, resizeMode: ResizeMode): Size {
     // TODO: Take sensor orientation into account here
-    val contentAspectRatio = contentSize.height.toDouble() / contentSize.width
+    val contentAspectRatio = contentSize.width.toDouble() / contentSize.height
     val containerAspectRatio = containerSize.width.toDouble() / containerSize.height

     val widthOverHeight = when (resizeMode) {
diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/CameraCharacteristics+getPreviewSize.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/CameraCharacteristics+getPreviewSize.kt
index ecd525c..e758c45 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/CameraCharacteristics+getPreviewSize.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/CameraCharacteristics+getPreviewSize.kt
@@ -9,7 +9,7 @@ fun getMaximumPreviewSize(): Size {
   // See https://developer.android.com/reference/android/hardware/camera2/params/StreamConfigurationMap
   // According to the Android Developer documentation, PREVIEW streams can have a resolution
   // of up to the phone's display's resolution, with a maximum of 1920x1080.
-  val display1080p = Size(1920, 1080)
+  val display1080p = Size(1080, 1920)
   val displaySize = Size(
     Resources.getSystem().displayMetrics.widthPixels,
     Resources.getSystem().displayMetrics.heightPixels
diff --git a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/SurfaceHolder+resize.kt b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/SurfaceHolder+resize.kt
index cc0fe9c..6ef3880 100644
--- a/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/SurfaceHolder+resize.kt
+++ b/node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/SurfaceHolder+resize.kt
@@ -7,7 +7,7 @@ import kotlin.coroutines.resume
 import kotlinx.coroutines.suspendCancellableCoroutine

 @UiThread
-suspend fun SurfaceHolder.resize(width: Int, height: Int) {
+suspend fun SurfaceHolder.resize(height: Int, width: Int) {
   return suspendCancellableCoroutine { continuation ->
     val currentSize = this.surfaceFrame
     if (currentSize.width() == width && currentSize.height() == height) {

Okay yea then I'm guessing this is related to the Orientation issue (https://github.com/mrousavy/react-native-vision-camera/issues/1891), because the devices you are testing on have a different Camera Hardware-Sensor orientation than my devices. We need to take that into account.

Okay yea then I'm guessing this is related to the Orientation issue (#1891), because the devices you are testing on have a different Camera Hardware-Sensor orientation than my devices. We need to take that into account.

@mrousavy How come all of the stretching issues (horizontal and vertical) are not present in 3.6.4? Were there any features or changes that altered the resizing methods after 3.6.4?

I had these issues on 3.6.4 so it wasn't introduced there, I think its a mixture of things for multiple devices.
Interestingly though, this is now for me broken in a way that I can work around 100% of the time. The initial camera loads with the width and height inverted. The 'fix' is to set them to 0 and activate the camera, and then in a useEffect set them to the devices width and height again. This loads the camera correctly, although is lower quality to the native camera app.

Its a hell of a lot more manageable now for those that do have the issue @mrousavy, so much appreciated. The camera is much quicker as well.

Thanks for your feedback @JshGrn, building in open-source I normally only hear negative feedback/bug reports - good to hear some positive feedback once in a while ❤
But yea, overall there's A LOT of effort and thinking and careful design construction that goes into the codebase of VisionCamera, especially on Android, to make it predictable, thread-safe, fast and stable. This is something that's really hard to achieve with Camera2, so I'm really happy about how it works right now, even though the Preview stretching issue is still bugging me.

Its fixable by simply activating the camera with a 0 width and height and then setting it in a useEffect right away, then onwards its perfect.
I was getting crashes a lot when I kept the camera active whilst swapping between react-navigation stacks, but to fix the crash I simply deactivated the camera on blur and reactivated it on focus, that stopped all the crashing.
I definitely don't use it for the full features this package offers, but for the ones I do its fantastic, and now I can work around it then its fine for my use-case.
Much appreciated

This is very helpful as I'm experiencing the same issues. Just for reference, what version of vision camera are you using with this workaround @JshGrn ? As @mrousavy mentioned, some of the older versions are unpredictable. Although it seems like I'm getting crashes on any version with slow androids when navigating through the stack so this workaround might be a step in the right direction. Thank you.

Its fixable by simply activating the camera with a 0 width and height and then setting it in a useEffect right away, then onwards its perfect.
I was getting crashes a lot when I kept the camera active whilst swapping between react-navigation stacks, but to fix the crash I simply deactivated the camera on blur and reactivated it on focus, that stopped all the crashing.
I definitely don't use it for the full features this package offers, but for the ones I do its fantastic, and now I can work around it then its fine for my use-case.
Much appreciated

This is very helpful as I'm experiencing the same issues. Just for reference, what version of vision camera are you using with this workaround @JshGrn ? As @mrousavy mentioned, some of the older versions are unpredictable. Although it seems like I'm getting crashes on any version with slow androids when navigating through the stack so this workaround might be a step in the right direction. Thank you guys for your work. 🚀

I am using the latest at the time of writing (3.9.1). The app crashes if the camera stays active whilst navigating to another stack and then back to the camera, not the first time but if I was to scan -> go to stack -> scan -> go to stack, it will eventually crash. What I am now doing is:
On the camera component in the camera stack view:
isActive={isActive && scanEnabled}

and
useEffect(() => {
    return navigation.addListener('focus', () => setScanEnabled(true));
}, [navigation]);

useEffect(() => {
    return navigation.addListener('blur', () => setScanEnabled(false));
}, [navigation]);

isActive is defined on the View which wraps all of the components that is set to true with onLayout:
<View onLayout={() => processLayout()}>
stuff here
</View>

that function essentially does some logic to deal with the aspect ratio by waiting 250ms and then setting the width and height properly. The fix to stop the crashing is by listening to the listeners of blur and focus, without those I can crash the app every time by quick navigation back and forth from the camera stack.

I am seeing this issue on 3.9.2

Adding the suggestion
  useEffect(() => {
    if (Platform.OS === 'ios') return;

    const timeout = setTimeout(
      () => setStyle({ height: '100%', width: '100%' }),
      250
    );

    return () => clearTimeout(timeout);
  }, []);

does not resolve for Android

The hack for android is not working for me, still stretched on first mount... what version does not have this issue?

What I'm doing instead is the following and it works flawlessly on 3.9.2 by executing right when the camera is ready and won't rely on a fixed time to update the camera preview width:
<Camera
      device={device}
      style={{...StyleSheet.absoluteFill, width: Dimensions.get('screen').width - 1 + stretchFix}}
      onInitialized={() => setStretchFix(1)} />

what is stretchFix @inzqne ?

What I'm doing instead is the following and it works flawlessly on 3.9.2 by executing right when the camera is ready and won't rely on a fixed time to update the camera preview width:
<Camera
      device={device}
      style={{...StyleSheet.absoluteFill, width: Dimensions.get('screen').width - 1 + stretchFix}}
      onInitialized={() => setStretchFix(1)} />

what is stretchFix @inzqne ?

this will just be a useState(0) judging by the use.

`const [cameraActive, setCameraActive] = React.useState(false);
      <View style={styles.frame}>
          <Camera
            ref={cameraRef}
            style={cameraActive && styles.cameraPreview}
            device={device}
            isActive={isActive}
            photo={true}
            onInitialized={() => setCameraActive(true)}
          />

      </View>`

      My temporary solution on Android 12 and below

Still have this issue in 3.9.1, stretched on first camera open

Same problem here for 3.9.0

Still have this issue in 3.9.1, stretched on first camera open

Same problem here for 3.9.0

I posted a work-around earlier in the thread that works flawlessly on 3.9.2, might be worth trying it out. Note that it is on 3.9.2

Still have this issue in 3.9.1, stretched on first camera open

Same problem here for 3.9.0

I posted a work-around earlier in the thread that works flawlessly on 3.9.2, might be worth trying it out. Note that I am using it on 3.9.2 specifically.

Hey @inzqne, thanks for your reply! The package does not have version 3.9.2 on npm. How do I install it?

Hey! I just released a new V4 beta (v4.0.0-beta.7) where I fixed a bunch of issues! 💪🚀 Can you test that and see if that fixes the issue for you? 😅
yarn add react-native-vision-camera@beta

You might need to increase the compileSdk to the newest Android SDK (34) if you are behind.

Still have this issue in 3.9.1, stretched on first camera open

Same problem here for 3.9.0

I posted a work-around earlier in the thread that works flawlessly on 3.9.2, might be worth trying it out. Note that I am using it on 3.9.2 specifically.

Hey @inzqne, thanks for your reply! The package does not have version 3.9.2 on npm. How do I install it?

My mistake, I meant 3.9.1

Thank you both! @inzqne @mrousavy

Hey @mrousavy ! I just tested the beta version
"react-native-vision-camera": "^4.0.0-beta.8",

And it fixed the stretched issue 🎉 I had on my Android when I was testing on 3.9.1 !!
For anyone wondering, my Android specs are:

Brand: UMIDIGI
Device Name: G3 Plus
Version Information: Baseband, kernel version
Android Version: 13
CPU: ums9230
Running memory: 4 GB
Model: MP25
Configuration Information: MP25HA
(Here is the official phone's spec details)

Thanks!! Hopefully the beta can be released sometime soon?

Awesome, great to hear!

Hey @mrousavy
Any estimation as to when the latest Beta enhancements will be promoted to the stable release version?

Thanks guys, great to hear that V4 now works perfectly! If you appreciate my work, please consider 💖 sponsoring me on GitHub 💖 so I can keep maintaining this library, fixing bugs and building new features! :)

when stable

@orenagiv not sure yet. I got a few other things to do, especially some things for RN MMKV as well to migrate to new arch and other things.

Thanks @mrousavy 🙏
I can confirm v4.0.0-beta.10 resolved the stretch issue.
So we will use this version.

What does two blackBars mean?

What does two blackBars mean?

I was having the same issue when I put the camera component inside another component, was showing two black bars, one in the top and another in a bottom. And I switch to version 3.6.4 that was stable and works fine.
Now, I tryied to reproduce the same problem and isn't appering more. But when put the props "photo={true}" on Camera component, the fps drops.

This bug still appears in the ShadowLens app which should use v4 react-native-vision-csmera. Device: Huawei P10 Lite


Some more resources

http://nodeup.com/eighty / https://github.com/nodeup/contribute/issues/49 - added in #21 (more resources in the website page)
http://baugues.com/depression - added in #20
https://github.com/cascadiajs/2015.cascadiajs.com/issues/228
https://github.com/berlinphp/berlinphp.github.com/pull/78
https://github.com/dopodcast/dopodcast.github.io/issues/12 (plenty more resources in the comments)

Developers & Depression: http://blog.baugues.com/speaking
Open Sourcing Mental Illness: http://funkatron.com/osmi.html
Frustration Driven Development: http://www.youtube.com/watch?v=1BYZCC5khjE
Stanford's Sapolsky On Depression in U.S. (Full Lecture) http://www.youtube.com/watch?v=NOAgplgTxfc

https://github.com/jsfest/oakland-cfp/issues/31
https://github.com/dmanners/avoiding-burnout
https://medium.com/@balupton/surviving-free-culture-f99b39ceb059 - added in #19

Many thanks! I will definitely add these as I find time.

[ ] http://nodeup.com/eighty / nodeup/contribute#49 - added in #21 (more resources in the website page)
[ ] http://baugues.com/depression - added in #20
[ ] cascadiajs/2015.cascadiajs.com#228
[ ] berlinphp/berlinphp.github.com#78
[ ] dopodcast/dopodcast.github.io#12 (plenty more resources in the comments)
[ ] Developers & Depression: http://blog.baugues.com/speaking
[ ] Open Sourcing Mental Illness: http://funkatron.com/osmi.html
[ ] Frustration Driven Development: http://www.youtube.com/watch?v=1BYZCC5khjE
[ ] Stanford's Sapolsky On Depression in U.S. (Full Lecture) http://www.youtube.com/watch?v=NOAgplgTxfc
[ ] jsfest/2014-oakland-cfp#31
[ ] https://github.com/dmanners/avoiding-burnout
[ ] https://medium.com/@balupton/surviving-free-culture-f99b39ceb059 - added in #19

Adding the rest of this in a forthcoming commit.


Tweens lag and fail to catch up (regression from 2.1)

Phaser 2.3.0: http://jsfiddle.net/nkovacs/8x23e7e1/
Phaser 2.1.3: http://jsfiddle.net/nkovacs/b0g1b7nL/
It'll console.log the difference between the expected position of the sprite and the actual position.
Normally it'll be close to 0 on both. But if you start to resize the result window, the Phaser 2.3 tween will lag behind, while the 2.1.3 one doesn't.
If you put the tab in the background, the Phaser 2.1.3 tween will catch up once you bring it back in the foreground. The Phaser 2.3 tween never catches up.
This is a problem for me, because I'm working on a multiplayer game, where the server has its own timers, and expects the animations to match up. E.g. the server knows that an enemy will reach its destination in X seconds, and damage the player. Because of this lag, the player is damaged before they see the enemy reaching them.
Unlike the jsfiddle example, my game's running at only 15-20 fps for some reason, which makes the problem even worse, as the tween is continuously accumulating lag.

As a quick fix, try forcing a single update per frame. I reckon that's part
of the cause.
On May 27, 2015 5:27 AM, "nkovacs"<EMAIL_ADDRESS>wrote:

Phaser 2.3.0: http://jsfiddle.net/nkovacs/8x23e7e1/
Phaser 2.1.3: http://jsfiddle.net/nkovacs/b0g1b7nL/
It'll console.log the difference between the expected position of the
sprite and the actual position.
Normally it'll be close to 0 on both. But if you start to resize the
result window, the Phaser 2.3 tween will lag behind, while the 2.1.3 one
doesn't.
If you put the tab in the background, the Phaser 2.1.3 tween will catch up
once you bring it back in the foreground. The Phaser 2.3 tween never
catches up.
This is a problem for me, because I'm working on a multiplayer game, where
the server has its own timers, and expects the animations to match up. E.g.
the server knows that an enemy will reach its destination in X seconds, and
damage the player. Because of this lag, the player is damaged before they
see the enemy reaching them.
Unlike the jsfiddle example, my game's running at only 15-20 fps for some
reason, which makes the problem even worse, as the tween is continuously
accumulating lag.
—
Reply to this email directly or view it on GitHub
https://github.com/photonstorm/phaser/issues/1819.

http://phaser.io/docs/Phaser.Game.html/Phaser.Game.html#forceSingleUpdate
On May 27, 2015 5:27 AM, "nkovacs"<EMAIL_ADDRESS>wrote:

Phaser 2.3.0: http://jsfiddle.net/nkovacs/8x23e7e1/
Phaser 2.1.3: http://jsfiddle.net/nkovacs/b0g1b7nL/
It'll console.log the difference between the expected position of the
sprite and the actual position.
Normally it'll be close to 0 on both. But if you start to resize the
result window, the Phaser 2.3 tween will lag behind, while the 2.1.3 one
doesn't.
If you put the tab in the background, the Phaser 2.1.3 tween will catch up
once you bring it back in the foreground. The Phaser 2.3 tween never
catches up.
This is a problem for me, because I'm working on a multiplayer game, where
the server has its own timers, and expects the animations to match up. E.g.
the server knows that an enemy will reach its destination in X seconds, and
damage the player. Because of this lag, the player is damaged before they
see the enemy reaching them.
Unlike the jsfiddle example, my game's running at only 15-20 fps for some
reason, which makes the problem even worse, as the tween is continuously
accumulating lag.
—
Reply to this email directly or view it on GitHub
https://github.com/photonstorm/phaser/issues/1819.

That makes it even worse: http://jsfiddle.net/nkovacs/8x23e7e1/11/

I am also trying to profile something similar. I am not convinced it is related only to tweens. I have 6 Phaser.Image instances which I loop over on the update loop shifting their y += 5.   This lags in a similar way but the annoying thing is that I cannot narrow it down.  I cut my asset resolution in half (got rid of native retina) and yeah its faster, but the grind is still there the same so I ruled that out really.
forceSingleUpdate kind of does a little better but the gained smoothness still is invalidated by the grind so this setting is preferable but the issue remains. I have also not noticed a major difference between canvas and webgl on my ipad.  Even so, neither solves the problem.
The reason I am using my Ipad, is because It happens on my desktop as well across all browsers. From @englercj  https://github.com/englercj/phaser-debug great plugin, to the canary dev tools, to simply logging out the FPS.  Nothing is giving me any clues and nor is it tied to any specific other event that I can see. I just mean it is not related to any specific functionality but rather everything which is bringing me back to timer and/or game/render loop.
It kills me that I cannot provide useful details!

This is now fixed in the dev branch (the Tween issue anyway). In theory it shouldn't have mattered if the browser lagged, as Game will run multiple logic updates to catch-up with itself. But there is a cap on the quantity it can do, and sometimes it just isn't enough - so Tweens will now use their own internal delta timer value to ensure that they complete within the duration specified, no matter what happens to the browser. It means they're at the mercy of browser stutters visually though, but that's probably a safe price to pay.


Switch from library panic to return an Error or other invalid return

Currently jacobi_symbol panics when p is even - perhaps this should return and Error, maybe -1?
/// Returns the Jacobi symbol `(a/p)` given an odd `p`. Panics on even `p`.
pub(crate) fn jacobi_symbol<const L: usize>(a: i32, p_long: &Uint<L>) -> JacobiSymbol {

Idea being that any dependent libraries who import this function can handle errors/panics instead of the underlying library panicing.

jacobi_symbol() takes an Odd-wrapped integer since #38, so this is no longer an issue.


How to apply InvFFT to image?

Thank you for your sharing。
I apply FFT to image like this：
var afterFFT=fft.FFTImageDataRGBA(imgdata.data,img.width, img.height);
I got imgdata from a canvas，imgdata.data is consist of rgba（not ComplexArray）。
Then，I got result（ComplexArray with real and imag），it seems right。
I want to apply INVFFT to the result， I try like this：
var beforeFFT=fft.FFTImageDataRGBA(afterFFT, img.width, img.height, 1);
But I got a wrong result：all of value are 0.
How can I apply InvFFT to the afterFFT ？
I reform the fft.FFTImageDataRGBA = function(data, nx, ny) to:
fft.FFTImageDataRGBA = function(data, nx, ny, isInverse)

Looking at the lib, it seems like that's not currently supported, the FFTImageDataRGBA would have to accept complex arrays.  Should be pretty easy to implement though.


fail to process reasonable 3d dataset

Hi
I tried to process a mrc file but failed:

The dataset is a mrc file of 1756x1843x143 size, 439MB
My GPU card is an NVIDIA RTX A4000 with 16GB GPU memory

I used tile_shape parameters to reduce the GPU usage but got this error. Note that without the  tile_shape parameter I have the same error
synapse_net.run_segmentation -i 20241017_RTtomo_TS3_25kx_rec.mrc -o 20241017_RTtomo_TS3_25kx_rec_result --tile_shape 128 128 12 -m vesicles_3d
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\u00xxxx\.conda\envs\synapse-net\Scripts\synapse_net.run_segmentation.exe\__main__.py", line 7, in <module>
  File "C:\Users\u00xxxx\Documents\PROJECTS\Leuven\synapse-net\synapse_net\tools\cli.py", line 159, in segmentation_cli
    tiling = parse_tiling(args.tile_shape, args.halo, is_2d=is_2d)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\u00xxxx\Documents\PROJECTS\Leuven\synapse-net\synapse_net\inference\util.py", line 459, in parse_tiling
    default_tiling = get_default_tiling(is_2d=is_2d)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\u00xxxx\Documents\PROJECTS\Leuven\synapse-net\synapse_net\inference\util.py", line 426, in get_default_tiling
    raise NotImplementedError(f"Estimating the tile size for a GPU with {vram} GB is not yet supported.")
NotImplementedError: Estimating the tile size for a GPU with 17.170956288 GB is not yet supported.

Do you have any tips to process it ?
Best
Benjamin

I will look into it more later, but what I immediately spot: The tile shape is given in the wrong order. It needs to be passed as ZYX (python axis convention), i.e. like this:
--tile_shape 12 128 128

but the error is likely due to some error issue in our code. I will check it later and report back.

Hi @bpavie ,
there was an issue with the inference code, that prevented any GPU with less than 20 GB from working. I fixed this now (though it still won't work with less than 10GB, but I don't think there is much on the market with <10GB anymore).
I fixed this, so once you update the code the prediction should work for you. Let me know if this fixes the issue.
It should also not be necessary to specify the patch shape anymore, the default one should work now.
If you do specify the tile shape, then make sure to give it in ZYX order.
And just fyi: in case you want to segment vesicles in Cryo-ET data, you could also try the model vesicles_cryo.

Hi @constantinpape
The new version seems to process, but it doesn't seem to not output anything.
I am on Windows  11 and tried:
--output ./output/

--output "./output"

and the full path but none of them output anything.
Example of command:
synapse_net.run_segmentation -i 20241017_RTtomo_TS3_25kx_rec.mrc --tile_shape 12 128 128 --model vesicles_cryo --output "./output/"
Determined tile size: {'x': 256, 'y': 256, 'z': 32}
Processing files:   0%|                                                                                | 0/1 [00:00<?, ?it/s]Rescaling the data at 20241017_RTtomo_TS3_25kx_rec.mrc by (1.2522727792913264, 0.8162963302047164, 0.8162963302047164) to match the training voxel size (0.88, 1.35, 1.35)

file structure:
dir
 Volume in drive C is Windows

 Directory of C:\Users\u00xxxx\test

13/12/2024  11:19    <DIR>          .
13/12/2024  11:18    <DIR>          ..
10/12/2024  13:48       460.533.096 20241017_RTtomo_TS3_25kx_rec.mrc
13/12/2024  11:19    <DIR>          output

I

13/12/2024  11:19    <DIR>          output

Did you check if there is anything in this directory (output)? The result should be written in there as 20241017_RTtomo_TS3_25kx_rec_prediction.tif.
If that's not the case then we likely have some issue with processing filepaths on windows, as we haven't tested running it on windows yet. I could set up CI later to see if I can reproduce the issu.

Hi @constantinpape ,
Yes, the output dir is empty:
dir output
 Volume in drive C is Windows
 Volume Serial Number is 920B-93BA

 Directory of C:\Users\u00xxxxx\test\output

13/12/2024  11:19    <DIR>          .
13/12/2024  11:19    <DIR>          ..
               0 File(s)              0 bytes
               2 Dir(s)  83.589.132.288 bytes free


AppVeyor: remove global cache and add node 0.11

We don't install any global modules.
/CC @jugglinmike

Coverage remained the same when pulling 97a5486cab54adf1dff4ad0f50f9d5e4bc4db6ce on XhmikosR:appveyor2 into f46fe408d4e0084cd7704a9cc7ba5d28407f7ef4 on jshint:master.

Coverage remained the same when pulling<PHONE_NUMBER>6e96993ad0eceb6662d5fdb180e74d on XhmikosR:appveyor2 into f7494c84554241d068cbe899730a119272f2329b on jshint:master.

Coverage remained the same when pulling<PHONE_NUMBER>6e96993ad0eceb6662d5fdb180e74d on XhmikosR:appveyor2 into f7494c84554241d068cbe899730a119272f2329b on jshint:master.

Bump
It seems there are a few tests that fail with Node.js 0.11 on Windows.

Coverage remained the same when pulling 1c7c6ae3b3b3cb19a8c50c229869ddef01bc18f9 on XhmikosR:appveyor2 into f7494c84554241d068cbe899730a119272f2329b on jshint:master.

Coverage remained the same when pulling b39fce4e363bebe7de9df03724baf5b27e3359a9 on XhmikosR:appveyor2 into f7494c84554241d068cbe899730a119272f2329b on jshint:master.

Coverage remained the same when pulling b39fce4e363bebe7de9df03724baf5b27e3359a9 on XhmikosR:appveyor2 into f7494c84554241d068cbe899730a119272f2329b on jshint:master.

We need to get this in along with #1932 and then try to fix the remaining issues...


JoErNanO: Watch baggagestorage.com.au

JoErNanO requests the watch of the watch_keyword baggagestorage\.com\.au. See the MS search here and the Stack Exchange search here.

baggagestorage\.com\.au has been seen in 0 true positives, 0 false positives, and 0 NAAs.

Approved by tripleee in Charcoal HQ


drivers: i2c: sam0 fix bus synchronization issues and smart mode read after nack error

When in master mode when the peripheral is enabled the bus state is unknown - software needs to force it to idle before the peripheral is ready to transmit.
The I2C driver expects to be able to control the stop - however when trying to delay the stop the use of smart mode triggers another bus read after the NACK due the data register read. When using smart mode the nack and module command have to be issued at the same time to prevent this read from occurring. Smart mode has been disabled to allow the api to control the stop.
Fixes: #38878
Signed-off-by: Bryan Hunt<EMAIL_ADDRESS>

@galak @nashif Can you kick off the workflows and check the reviewer list for this PR?

Hi @bryan-hunt ,
Just to inform, it is not allowed merge commits in Zephyr. Instead create unnecessary commits, rebase your branch against the main branch and don't make commit with changes on top of original commit.  The rebase should be on top of upstream/main and don't forget to fetch upstream code.
Please, read How to Contribute to get a better view about how to upstream code to ZephyrRTOS.

Hi @bryan-hunt ,
Some fixes were made for I2C. Rebase before you continue your work.

@bryan-hunt +1 for rebase if you have the time. The i2c sam0 driver is currently broken.


[UI/UX] Implement search bar

In https://github.com/ZJUSCT/mirror-front/issues/29#issuecomment-1179562329, @lynzrand mentioned that he had proposed a search box design. We now want to implement this solution.
We expect the search box to exhibit the following behaviours:

If the search box is empty, the current layout remains unchanged.
If the search box contains any characters, the following actions occur:

Support fuzzy search.
Hide the "Popular Mirrors" and "Latest Update" components, and only display the cards within the "All Mirrors" section that match the search.
The matched characters in the mirror cards should be highlighted. In light theme, the highlight colour should be "创新红" #B01F24, one of the school colours of Zhejiang University.

On the right side of the search box, there should be a CTRL K symbol to indicate to users that they can use this shortcut to enter content within the search box. Refer to the implementation on the CERNET mirror site.

A basic search bar with fuzzy search has been implemented in https://github.com/ZJUSCT/mirror-front/commit/ce0c8eabf0230f3eae3a737c9a55da2b96c790fe. Further style tweaks are still needed.


No response from server

Hi,
For some Reason on v12.2.0 I get the following error:

This page isn’t working
<IP_ADDRESS> didn’t send any data.
ERR_EMPTY_RESPONSE

whilst the output in the terminal looks good to me:

Started the plot device on port 9000

I am using a dockerized mongodb and I also verified there is a user found in its mongo collection.
node -v  # v6.10.3
Any Ideas?

🙊  Changed the port to smth else and it worked.


[rm nixpkgs] Install same package in profile as in flake.nix

Summary
I don't really expect there to be a difference here, but for consistency, we should be installing exactly the same packages in profile and in flake.nix, even if we expect the contents to be the same.
How was it tested?
Before:
> DEVBOX_FEATURE_REMOVE_NIXPKGS=1 devbox rm hello
> DEVBOX_FEATURE_REMOVE_NIXPKGS=1 devbox add<EMAIL_ADDRESS>> cat devbox.lock
...
<EMAIL_ADDRESS>{
      "last_modified": "1970-01-01T00:00:00Z",
      "resolved": "github:NixOS/nixpkgs/eb751d65225ec53de9cf3d88acbf08d275882389#hello",
      "source": "devbox-search",
      "systems": {
        "aarch64-darwin": {
          "store_path": "/nix/store/qh8s6yb93b1wl0jxg5i0pfq2rf7qm44j-hello-2.12.1"
        },
        "x86_64-darwin": {
          "store_path": "/nix/store/ycbqd7822qcnasaqy0mmiv2j9n9m62yl-hello-2.12.1",
          "ca_store_path": "/nix/store/1jaa5hqp6pv43z7p3xc7gkck0q3rkg4n-hello-2.12.1"
        },
        "x86_64-linux": {
          "store_path": "/nix/store/8n3pr83b7aab975gwrd6hnk10pqphr59-hello-2.12.1"
        }
      }
    },
...
> ls -la .devbox/nix/profile/default/bin
total 0
dr-xr-xr-x  5 root  nixbld  160 Dec 31  1969 .
dr-xr-xr-x  5 root  nixbld  160 Dec 31  1969 ..
lrwxr-xr-x  1 root  nixbld   66 Dec 31  1969 hello -> /nix/store/ycbqd7822qcnasaqy0mmiv2j9n9m62yl-hello-2.12.1/bin/hello

> DEVBOX_FEATURE_REMOVE_NIXPKGS=1 devbox run -- tail -n 1 '$(which hello)'
exec /nix/store/1jaa5hqp6pv43z7p3xc7gkck0q3rkg4n-hello-2.12.1/bin/hello "$@"

After
> ls -la .devbox/nix/profile/default/bin
total 0
dr-xr-xr-x  5 root  nixbld  160 Dec 31  1969 .
dr-xr-xr-x  5 root  nixbld  160 Dec 31  1969 ..
lrwxr-xr-x  1 root  nixbld   66 Dec 31  1969 hello -> /nix/store/1jaa5hqp6pv43z7p3xc7gkck0q3rkg4n-hello-2.12.1/bin/hello

The installation here works well. But I'm seeing some errors when calling devbox rm hello after having installed the package using the CaPath:
resolving - in-memory: calling search..
Error: Package<EMAIL_ADDRESS>not found in profile. Skipping.

Debugging...

OK I found the problem and fixed it.
While debugging, I noticed that when we add packages to the profile by store path, then our profile looks like this:
> nix profile list --profile .devbox/nix/profile/default
0 github:NixOS/nixpkgs/22dc22f8cedc58fcb11afe1acb08e9999e78be9c#legacyPackages.x86_64-darwin.bat github:NixOS/nixpkgs/22dc22f8cedc58fcb11afe1acb08e9999e78be9c#legacyPackages.x86_64-darwin.bat /nix/store/y7qsqbjmdfgdqr9z47dj6zn5k7j22zh4-bat-0.19.0
1 github:NixOS/nixpkgs/3007746b3f5bfcb49e102b517bca891822a41b31#legacyPackages.x86_64-darwin.jq github:NixOS/nixpkgs/3007746b3f5bfcb49e102b517bca891822a41b31#legacyPackages.x86_64-darwin.jq /nix/store/glb4gpim82f0bmzcpayvklbfqc6qzvwj-jq-1.6-bin /nix/store/qb0p5mj3m8n64kpnmi8fd0f0ikazdy67-jq-1.6-man
2 - - /nix/store/ycbqd7822qcnasaqy0mmiv2j9n9m62yl-hello-2.12.1

That is, we don't get resolved/unresolved references. And, this may be a problem since devpkg.Package.Equals() relies on that, for example.
In practice, I think this won't be a problem, but I figure I'd call attention to it.

What worried me is that sometimes we attempt to resolve an empty package. Debugging logs:
resolving<EMAIL_ADDRESS>in-memory: found in lockfile!
resolving - in-memory: calling search..
resolved to github:NixOS/nixpkgs/f80ac848e3d6f0c12c52758c0f25c10c97ca3b62#-
comparing github:NixOS/nixpkgs/eb751d65225ec53de9cf3d88acbf08d275882389 and github:NixOS/nixpkgs/f80ac848e3d6f0c12c52758c0f25c10c97ca3b62
resolving<EMAIL_ADDRESS>in-memory: found in lockfile!
resolving - in-memory: found in lockfile!
Error: Package<EMAIL_ADDRESS>not found in profile. Skipping.

Well, it's treating - as a legacy package, and then just using the default nixpkgs commit hash, f80ac848e3d6f0c12c52758c0f25c10c97ca3b62, to build its reference.. so thn we use the reference github:NixOS/nixpkgs/f80ac848e3d6f0c12c52758c0f25c10c97ca3b62#-...
Anyway, I don't think we'll run into this, but I just wanted to pull on the thread a little bit.

That is, we don't get resolved/unresolved references. And, this may be a problem since devpkg.Package.Equals() relies on that, for example.
In practice, I think this won't be a problem, but I figure I'd call attention to it.

I think this does change some assumptions in the code i.e. that querying nix profile item will have a resolved reference.  So, maybe we need to add some defensive checks.
But I don't see where it is used in devpkgs.Package.Equals(). Could you point me to it?


#2704: Scatterplot: Add min/max fields for X/Y axes on plot to improve viewer's usability

Most often, only values >0 on both axes are analyzed. And when using the range slider, the graph “moves” to the right (or up for Y) and you have to return it to bring the desired area (close to 0.0) closer.
Consider implementing:

[ ] the ability to rigidly fix one of the ends of the sliders (in this case, so that when zooming in on the graph, negative parts of the axes do not appear).
[ ] set min and max values without accessing the context panel

This issue has been mirrored in Jira: https://reddata.atlassian.net/browse/GROK-15068

In this case, it might be more convenient to use zoom on a specific area of the scatter plot by using ALT+mouse drag.

This issue has been mirrored in Jira: https://reddata.atlassian.net/browse/GROK-16168


GH-38458: [Go] Add ValueLen to BinaryLike interface

Rationale for this change
Adding ValueLen to the BinaryLike interface for easy convenience of determining the length of an individual value for a Binary/String like array.
Are these changes tested?
yes

Closes: #38458

CC @frbvianna


Rfc/git 1630 signals

Take over from #1770 re: #1630

@ahopkins
Thanks for taking this over. I'm very interested in this Signals work, because it is integral for the future of Sanic-Plugins-Framework.
I'll read over the proposed implementation and see if I can offer feedback.

I think this is a really great potential win. I've been playing with the branch, and have some ideas on how to refine the implementation. I'll post on the RFC thread some changes I'm thinking about. One is to scrap the additional dependency and just roll our own signaling solution.

Coming 2021


Wakepy and pyinstaller - does it works?

Hello,
I have a python program packed with pyinstaller which uses wakepy. After packing for windows 10 .exe program is halted when wakepy with keep.running() is called. Have you tested it with pyinstaller?

Hi @folkien , there is nothing so special in wakepy which make it not possible to use it in a packaged executable. One thing to consider, though: The current (0.7.2) version uses still parts from a subpackage called _deprecated. As this has dynamical (runtime) imports, pyinstaller and alike do not see the import and cannot automatically add the files to the bundle. This means that those files need to be added to the bundle manually.
See also https://github.com/fohrloop/wakepy/issues/49
Note that the _deprecated subpackage is, as the name also implies, deprecated and will be removed in a future version of wakepy. In the future, I hope the dynamical imports can be avoided altogether.
Hope this helps!
Niko


Readme: Added run sample script

README.md: Added description of how to run a sample test.
Removed a paragraph from Boot section.

@JanPrusinowski Thanks for contribution, but please first follow these rules and make sure pre-commit passes before asking for review: https://github.com/Dasharo/open-source-firmware-validation?tab=readme-ov-file#contributing


Remove the name field from Expando.

The Expando class has a name field that is used only for toString-debugging.
That's not something we do anywhere else, and it's probably more speculative than useful.
Just remove it.

Not worth the effort to make the change.


fix(cli): context-aware import path snippet for generate command

Fixes https://github.com/prisma/prisma/issues/6409
See additional discussion in https://github.com/prisma/prisma/pull/21140
Needs tests.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 2 committers have signed the CLA.:x: fiznool:x: janpioYou have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 2 committers have signed the CLA.:x: fiznool:x: janpioYou have signed the CLA already but the status is still pending? Let us recheck it.


Exception when using attachByName on a validator chain

Tested on ZF 2.5.1
Running the code example in https://framework.zend.com/manual/2.4/en/modules/zend.input-filter.intro.html#introduction yields an exception:
Argument 1 passed to Zend\ServiceManager\AbstractPluginManager::__construct() must be an instance of Zend\ServiceManager\ConfigInterface, instance of Zend\ServiceManager\ServiceManager given, called in /opt/potwallet_web/vendor/zendframework/zend-validator/src/ValidatorPluginManager.php on line 382 and defined in /opt/potwallet_web/vendor/zendframework/zend-servicemanager/src/AbstractPluginManager.php on line 62
The problematic line is in https://github.com/zendframework/zend-validator/blob/master/src/ValidatorChain.php#L70
when the plugin manager is not initialized.
$this->setPluginManager(new ValidatorPluginManager(new ServiceManager));
the PluginManager constructor's argument is expected to implement  ConfigInterface, ServiceManager doesn't.

(Discussed offline with @mkherlakian , and agreed as a "won't fix" due to being a non-LTS version, and the latest version resolving the issue.)
If you could... could you please submit an issue against the website repository so that we can ensure a big banner is posted across those legacy pages indicating folks should look to https://docs.zendframework.com for up-to-date documentation?

Will do @weierophinney. Also closing this one out per your comment. Thanks!


Support nested actions

The verification action should search all action YAML files in the repository and validate their typings against them accordingly.
Currently only the main action's typings are verified.

Great spot. It's a bug. We should fix it with a major version bump because for some clients it may be a breaking change to add the missing validation.
We already have one action with nested actions that provide typings, which increases the urgency of this bug: https://github.com/Wandalen/wretry.action


Using Observables to cancel Observables. (Alternative to Abort Signal)

Okay, this one might sound a little strange, but I was talking with @benjamingr, and he reminded me of past discussions about just using Observable as a signal by talking about what a shame it was we had AbortSignal natively, but no Observable. Like "what if we wanted to signal something other than cancellation?".
The idea
To that end, I'd like to re-explore this idea. Currently, we have the idea of takeUntil as an operator, which is effectively cancellation of the source observable, and notification of completion downstream. But what if we allowed that cancellation notification at the point of subscription, and we also provided an observable of cancellations to the observable initializer:
These two things would be very similar, only one would't cause the completion notification:
source$.pipe(
  takeUntil(notifier$)
).subscribe({
  next: console.log,
  complete: () => console.log('complete')
});

source$.subscribe({
  next: console.log,
  complete: () => console.log('complete') // never called because it was cancelled.
}, notifier$);

Let's take the example from #5649:
    let sideEffects = 0;

    const firehose = new Observable((subscriber) => {
      let n = 0;
      while (!subscriber.closed && n < 1000) {
        sideEffects++;
        subscriber.next(n++);
      }
    });

    // Let's say this observable here is the result of some sort of operator that someone made.
    // If this is hard to understand, I guess a challenge would be to have someone "roll their own"
    // concatMap operator without using our Subscribers and having working knowledge of
    // our implementation details.
    const flatteningSyncOuter = new Observable((subscriber, teardown$) => {

      // Notice how now we are threading the signal through. Pretty straight forward.
      firehose.subscribe({
        next: value => subscriber.next(value)
      },  teardown$);

      firehose.subscribe({
        next: value => subscriber.next(value),
        complete: () => subscriber.complete()
      },  teardown$);
    });

    const unsubController = new Subject();
    const teardown$ = unsubController.asObservable(); // this is really optional

    flatteningSyncOuter.pipe(
      tap(() => {
          // Give it a 1 in 10 chance of unsubbing externally.
          if (Math.random() > 0.9) unsubController.next();
      }),
      take(300)
    ).subscribe(undefined, teardown$);

    expect(sideEffects).to.equal(3);

As you can see, the API difference is minimal.
Furthermore, we could easily provide something to get an Observable from an AbortSignal, and vice versa. AND we could even allow for users to pass AbortSignal there, if it's available, and we could do the conversion for them under the hood.
I think this solves a few problems around AbortSignal, in particular, if someone wanted to use the provided teardown$ to register teardowns, it's MUCH more ergonomic than AbortSignal:
AbortSignal
const source1$ = new Observable((subscriber, signal) => {
   const resource = new ExpensiveResource();
   signal.addEventListener('abort', () => resource.close(), { once: true });
});

Observable
const source1$ = new Observable((subscriber, teardown$) => {
   const resource = new ExpensiveResource();
   teardown.subscribe(() => resource.close());
});

It's a lot cleaner. I think the confusing bit above might be that users will wonder if they need to unsubscribe from the teardown they've subscribed to. But that's unnecessary, because teardown must happen eventually. Unless the subscription runs forever.
Pros

Solves all of the same problems that AbortSignal does
We an still provide compatibility with AbortSignal
Teardown registration is more ergonomic (if users want to do it manually)
We wouldn't need to ship any code for Subscription eventually (and we'll always need to ship observables, clearly)

Cons

Teardown registration has the confusing element of "should I unsubscribe from this too??".
We end up needing to ship the code for this to the browser in the short term
Less convenient for sharing AbortSignals with other APIs that accept them.

Other thoughts
If Promise was designed better, and allowed sync emission, it would have been better for this particular use case. Alas. We have what we have.

Hmm, this is interesting - some thoughts on this:

I think the important bit (to me) is being able to pass a single token to multiple .subscribes (alongside other non Rx APIs).

I think the most interesting (to me) use case to me is to let .subscribe take a signal (optionally).
This would simplify a bunch of my code that deals with both Rx and non Rx bits.

I think for the observable constructor - returning a disposer is fine and probably as short as it gets.
I don't particularly understand the merit of tokens as another argument (like you said - you just want to signal it happened)
I don't think I understand the use case very well.

Tangent:

If Promise was designed better, and allowed sync emission, it would have been better for this particular use case. Alas. We have what we have.

For what it's worth when promises needed to do this we used callbacks which was both the shortest and had no error handling / timing guarantees that had to be upheld.
In your case that would be:
const source1$ = new Observable((subscriber, teardown) => {
   const resource = new ExpensiveResource();
   teardown(() => resource.close());
});
// which is almost as short, and provides no benefit over the 'current'
const source1$ = new Observable((subscriber, teardown) => {
   const resource = new ExpensiveResource();
   return () => resource.close();
});

We mainly did this because cancellation (in bluebird) was an afterthought of the promise constructor which already had other guarantees and constraints.

Also:

he reminded me of past discussions about just using Observable as a signal by talking about what a shame it was we had AbortSignal natively, but no Observable

I would very much be interested in exploring Observables in the platform again (as a much simpler primitive). I had to learn a lot just to get a (pretty basic) understanding of EventTarget and it's a very complicated primitive in some surprising ways.

I like it.

I think the confusing bit above might be that users will wonder if they need to unsubscribe from the teardown they've subscribed to. But that's unnecessary, because teardown must happen eventually. Unless the subscription runs forever.

Although, for me the confusing part is: what happens if I pass a normal/unicast observable to subscribe, will RxJS "multicast it for me" or am I only supposed to pass multicasted observables to subscribe?

Interesting idea. I like it.
One observation is that using a Subject as the notifier is a little different to using an AbortSignal. To behave more similarly to AbortSignal, it would need to be a ReplaySubject(1). Passing a Subject kinda assumes that all use of the notifier within observable's implementation is going to involve synchronous calls to subscribe - i.e. no possibility of a missed notification.
What would the implementation of Observable#subscribe look like - given that (as a garden-variety observable) the notifier won't have an equivalent of Subscription#closed or AbortSignal#aborted? How would it exit early upon receipt of an already-signalled notifier?

I guess a BehaviorSubject might model that better. 🤔

Exciting! I am exploring this idea too a couple of months ago (check out the article), my thoughts:
The foundation that we currently have is that Observable has two fundamental properties: data propagation and cancelation. It is evident that data propagation is fundamental since it is what makes an Observable to be observable, it is its way to express something to the subscriber and we achieve it using a callback subscriber.next. The other property is cancelation, this may seem fundamental at first but it occurred to me that cancelation may not and it may be just an emergent property. Exploring this I've found that, cancelation is a demand from the subscriber, it is purely based on the application logic when to cancel. So it is not the cancelation is fundamental but rather the ability for an Observable to accept a demand or a request from the outside and it makes sense because, in a larger scale, a subscriber could demand other than cancelation and that could emerged somewhere (depends on the application requirement). This ability of Observable to react base on the subscriber request arrives at me to the conclusion that cancelation is indeed an Observable because sending a request is very similar to what an Observable is (a data propagation).
@benlesh will the idea of accepting another Observable inside the producer function as a way for cancelation is open for the possibility to be more generic like what you have said:

Like "what if we wanted to signal something other than cancellation?".

If so then the API that I am imagining:
Observable.CANCEL = Symbol('CANCEL');

const obs = new Observable((subscriber, talkback) => {
  const id = setInterval(() => subscriber.next('hey'), 500);

  talkback.pipe(
    filter(token => token === Observable.CANCEL),
    takeWhile(token => token !== Observable.CANCEL, true)
  ).subscribe(() => {
    clearInterval(id);
    subscriber.complete();
  });
})

const cancel = fromEvent(document.body, "click").pipe(mapTo(Observable.CANCEL))

obs.subscribe(data => console.log(data), cancel);

Then maybe we can extends the Observable to have a boilerplate encapsulation for automatic cancelation like:
// CancelObservable extends Observable
new CancelObservable((subscriber, cancel) => {
  cancel.subscribe(); // filtered subscriber request down to Observable.CANCEL only
})

I hope that RxJS team will explore this idea, it is very exciting! It will open up many opportunities.

I'd like to iterate that supporting/using AbortSignals would not mean passing the signal to the subscriber function (in the observable constructor). It would just mean subscribe accepting an optional signal second argument and if a signal is passed there invoking the unsubscribe function
const source1$ = new Observable((subscriber) => {
   const resource = new ExpensiveResource();
   // call next somewhere

   return () => resource.close();
});
const unsubscrie = source1$.subscribe(...); // today, still works
source1$.subscribe(..., signal); // I am suggesting we add support for this

This would break very little existing code while enabling interop with all other async code :]

Bare with me cause I'm probably missing the point here... But out of curiosity, why introduce a new cancellation mechanism when rx already has one?
Also, when AbortSignal is used for a Promise, it would reject with AbortError once it is signalled. Wouldn't it be consistent if something similar is done for Observable? In that case, support for cancellation signal can be handled with just a new operator similar to takeUntil, but instead of completing, it errors out with AbortError when abortSignal emits. Something like
source$.pipe(
  abortWhen(abortSignal)
  ).subscribe({
    next: console.log,
    complete: () => console.log('not called on abort'),
    error: e => console.error('called on abort', e);
  });

This way, upstream is cancelled the normal way (returned function from subscribe function is called).
Also, one can make abortWhen operator accept an observable to signal the cancel as well.

Ok I indeed missed the point. Turns out (for me) that the plan was altogether to move away from old way of cancellation to a new way. #5863. As I understand to be able to support aborting of synchronously emitting observables (like in the firehose example), you need to be able to setup cancellation ahead of time.
I'll shut up (for now 😜).

@benjamingr
I think I'd agree with you that current way of detecting a cancellation should still be supported, so existing code is not broken. But consider the following example as what I understand is the main reason for the idea of passing the abort signal to the subscriber function:
const firehose$ = new Observable((subscriber) => {
   let cancelled = false;
   let count = 0;
   while(!cancelled) {
     subscriber.next(count++);
   }
   return () => cancelled = true;
});

In this synchronous 'firehose' example, the subscriber function will never reach the point where the cancellation function is returned. Hence it can never be cancelled.
But in the case where an abort signal is passed, it can still be cancelled like this:
const firehose$ = new Observable((subscriber, signal) => {
   let count = 0;
   while(!signal.aborted) {
     subscriber.next(count++);
   }
});

This is because cancellation is setup prior to calling the synchronous subscription function. Where in the first example, cancellation is (tried to be) setup after calling it.
Still, my opinion is that cancellation in the first example should still be supported (as well), because I'd say that in more than 99% of the cases, observables are used in asynchronous ways, and as you say, this way it can be backward compatible with existing code.


use circular buffer in FiniteReplayProvider

Uses NewFiniteReplayProvider() to instantiate the provider instead of direct literal. This allows catching count misconfiguration at instantiation (typically application startup) instead as a later panic. In my opinion all panics should be replaced with returning errors, all uses of panic I looked at in the library are in the "expected errors" category. Keeping panic behaviour in Put(), as changing that is very much out of scope.
Synchronises access to the underlying buffer and head and tail state using an RWMutex. I haven't looked closely at the concurrency handling in the library as a whole, so if you're already serialising access to the replay providers this doesn't add any value. If not, a similar mutex should be added to the ValidReplayProvider.

If you don't like the NewFiniteReplayProvider/error-approach it should be removed and made consistent with ValidReplayProvider instead. Likewise I don't think that these changes should be released/merged without consistently switching to errors instead of panics.

Thank you for the PR!

if you're already serialising access to the replay providers

This is done inside the provider, replay providers are not required to be thread safe. There's no need to change that yourself, though. I wanted to have this code mainly as a reference for benchmarks and tests. Besides, the ValidReplayProvider should be changed aswell, as the underlying buffer was the same which means it also leaks memory.

all uses of panic I looked at in the library are in the "expected errors" category

My take on this would be that these panics are caused by programming issues rather than normal execution which means they are bugs. For example, dispatching a Message with no topics would be a bug, which should be caught in testing. Besides, the causes of these panics can be statically determined, I believe. Here are a few examples of what I think:

The FiniteReplayProvider is initialized at startup and usually its size is provided directly as a literal. If it would be negative it would be very apparent
A Connection is usually created with an http.Request created immediately before. If the request object would be nil, it would be very apparent

There is a case to be made about errors related to topics – while in most scenarios I do think they would be provided statically, I can imagine some where they would be pulled from a data source. I'd say that even then, if the data source gives nothing back, it means the data is corrupted and that its errors were not handled. Though it would be prudent to not crash the whole program in these cases.
Not panicking would allow for better error handling in unusual cases, allow for custom error logging and, most importantly, not crash the program. What do you think? Have you encountered in your use a scenario where you'd have preferred if errors were returned instead?

I don't think that these changes should be released/merged without consistently ...

Totally agree. The changes will be cohesive.

One could argue that it's still a bug as the programmer failed to properly validate the input 🤷 but it's definitely not a static one.

You're right here – the argument falls flat whenever there's a new layer introduced – be it some other code or an external configuration source. Otherwise pretty much every error happens because somebody didn't validate something somewhere 😃

... like forgetting to nil-check some input and causing a nil panic, unforeseen errors

That's a better definition for "programming errors" than what I had in mind. I think I'm convinced on this – I'll adapt the API to use errors and constructors. Constructors also simplify the library code, as no init logic would need to be done anymore. Given that this is a pretty big set of changes I'll probably roll them out across multiple versions (something like 0.9.0-pre.1, 0.9.0-pre.2 and so on).
Thank you for your input and time!

Great! That sounds like a sensible & responsible plan :+1:

Pushed new commit:

use replay helper function to avoid creating an intermediate buffer
TestFiniteReplayProvider had to be adjusted, as it assumed that the event with the ID 4 would still be present after pushing events {5,6,7} into a finite  replay provider with a count of 3.
TestJoe_errors is failing, but I'm a bit unsure as to what it's actually  testing.

Where is that test failing? I should be able to help with that.

--- FAIL: TestJoe_errors (0.00s)
    joe_test.go:262: callback was called after subscribe returned
2024/02/06 15:57:29 panic: panicked
goroutine 167 [running]:
runtime/debug.Stack()
	/usr/lib/go/src/runtime/debug/stack.go:24 +0x5e
github.com/tmaxmax/go-sse.(*Joe).tryReplay.func1()
	/home/hugowett/Projects/go-sse/joe.go:246 +0x93
panic({0x704580?, 0x8027d0?})
	/usr/lib/go/src/runtime/panic.go:914 +0x21f
github.com/tmaxmax/go-sse_test.(*mockReplayProvider).Replay(0xc000312500, {{0xc0003504d0, 0xc0001b1d38}, {{{0xc000350490, 0x40ebe5}, 0x80}}, {0xc000316d20, 0xc000316d20, 0xc000350480}})
	/home/hugowett/Projects/go-sse/joe_test.go:32 +0x67
github.com/tmaxmax/go-sse.(*Joe).tryReplay(0x0, {{0x805398, 0xc00031c0e8}, {{{0x0, 0x0}, 0x0}}, {0xc0003048d0, 0x1, 0x1}}, {0x805370, ...}, ...)
	/home/hugowett/Projects/go-sse/joe.go:250 +0x102
github.com/tmaxmax/go-sse.(*Joe).start(0xc0003038b0, {0x805370, 0xc000312500})
	/home/hugowett/Projects/go-sse/joe.go:216 +0x2a9
created by github.com/tmaxmax/go-sse.(*Joe).init.func1 in goroutine 166
	/home/hugowett/Projects/go-sse/joe.go:279 +0x1e6
FAIL
FAIL	github.com/tmaxmax/go-sse	0.087s
FAIL

The test verifies the behavior of Joe on client errors. At the beginning it publishes two messages so that there is something to replay when a client is subscribed – the fact that the replay provider has count 1 and 2 messages are published is also kind of a test for FiniteReplayProvider, because it is programmed in such a way to be able to replay events even when the event with the client's LastEventID is the last removed event from the buffer. Let me give an example:

a message with ID "first" is published
(in a real life scenario) the client receives the message with ID "first" and then the connection drops; LastEventID is now "first"
a message with ID "second" is published, but there is no one to receive it. The FiniteReplayProvider stores it and drops the message with ID "first", given that its capacity is 1
the client reconnects and sends the server its LastEventID, "first"
even though the message with ID "first" was removed, the replay provider knows that it was before the message with ID "second" – and thus a valid ID – and replays to the client the message with ID "second". Here is the code which ensures this behavior

I think what happens here is that in the new implementation no lastRemovedID is retained and because of this no events are replayed in that test. The callErr is still returned because the Flush method is still called on the client and that's why the error assertion doesn't fail and only the called == 1 assertion does. Also, called == 1 assertion is normally there to ensure that after the client has returned an error, the provider doesn't attempt to send any more messages – that's why there's another Publish before. In this case, given that FiniteReplayProvider behaves in an unexpected manner, called will probably be 0, as no messages have actually been attempted to be sent to the client.
To fix this the previous behavior of the internal buffer type – i.e. the logic surrounding lastRemovedID should be ported. I'm wondering though if another behavior would be better; in the way the replay provider is implemented now, if the client has the LastEventID an ID of an event which is no longer part of the buffer, nothing would be replayed. Would it be better to still replay all the events in the buffer, considering that if the client provides an ID, it's probably of an older event? Or should the provider replay all events regardless of whether the client provides a LastEventID or not? Should this be configurable? What do you think?

Yeah, that behaviour is a bit odd. Or, well, I guess it’s to provide playback for as long as it’s actually possible, but it complicates the logic around replays a bit: we can replay as long as the last event is retained OR if it was the last event to get evicted. I wonder if it’s worth that extra +1 of replay capability. It explains the behaviour in the test I changed, it didn’t assume that more than 3 events were retained.
I wonder if there’s some convention around this behaviour (replay all vs. replay none) when it comes to implementations? And is there any way to communicate to the client that the last observed event is too old for replay?

Let's leave the code as it is for now. I'll do some research to see what other libraries – in and outside the Go ecosystem – implement. As a curiosity, given that you're using this replay provider, what was the behavior you would have expected, if you've thought of it before?

I’ve mostly been concerned about how to communicate to the client that the ID is too old to support resume.
Found this issue: https://github.com/whatwg/html/issues/8297
So maybe the best thing would be a hook or config that allows sending a message with an application specific “out of sync” event type.

Hm, this is indeed important.
Perhaps there should be a specific kind of error ReplayProviders should return which indicates that for the given LastEventID nothing could be replayed. Then there could be some OnError hook on the Server which is called when a subscription fails with the error and the subscription as parameters. Inside this hook one could check if the error is that specific replay error and do something appropriate in that scenario – for example retry with a subscription without a LastEventId, send another response to the client etc.
The API could look like:
var ErrReplayInvalidID error

type Server struct {
    OnError func(r *http.Request, sub Subscription, err error) (newSub Subscription, retry bool)
}

With this hook I could also get rid of the Logger, given that OnSession and OnError cover pretty much the entire lifecycle of the request.
(note to self: the sentinel errors are kind of piling up; there may be some potential of unifying them into a single error type)
What do you think?

Looks good, and grouping the errors into a single type could allow some more convenient error checking. I'm quite partial to using error codes. It has the benefit of making the things you check (type and code) constants instead of variables, and linters can warn if you're doing a non-exhaustive switch:
type ErrorCode string

const (
	ErrorCodeInvalidID ErrorCode = "invalid_id"
	ErrorCodeNoTopic   ErrorCode = "no_topic"
)

func NewError(code ErrorCode, text string) *Error {
	return &Error{
		text: text,
		Code: code,
	}
}

func NewErrorf(code ErrorCode, format string, a ...any) *Error {
	err := fmt.Errorf(format, a...)

	return &Error{
		text:  err.Error(),
		cause: errors.Unwrap(err),
		Code:  code,
	}
}

type Error struct {
	text  string
	cause error

	Code ErrorCode
}

func (e *Error) Error() string {
	return e.text
}

func (e *Error) Unwrap() error {
	return e.cause
}

func IsErrorWithCode(err error, code ErrorCode) bool {
	var e *Error

	return errors.As(err, &e) && e.Code == code
}

func main() {
	err := NewError(ErrorCodeInvalidID, "dunno what that is")
	if IsErrorWithCode(err, ErrorCodeInvalidID) {
		println("is invalid ID")
	} else if err != nil {
		panic("waah!")
	}

	var sseErr *Error

	if !errors.As(err, &sseErr) {
		panic("unknown error!")
	}

	switch sseErr.Code {
	case ErrorCodeInvalidID:
		println("switch thinks this is an invalid id")
	case ErrorCodeNoTopic:
		println("switch thinks this is something that lacks a topic")
	}
}

...it opens up to both including root causes and adding fields carrying relevant information about the error, like the event that triggered the error.

Do you want me to do any more changes to the PR. For example the last deleted ID feature is a bit up in the air at the moment. Should I add support for it, or do you want to merge & maybe work on removing it completely before next release?

I'll remove that feature for the moment.
If you have the resources, could you add the suggested test in the code review?

Of course. I’ve never done any allocation checking in tests before, so it’ll be a good exercise. Though I don’t know if it’ll explicitly catch memory leaks, as we only track allocations, not deallocations. An infinitely growing slice would definitely show up though, so 👍

We don't really have a way to track deallocs, given that those are the GCs responsibility and it decides when to do them. Besides  the replay provider is usually part of a long-running service, so the deallocation won't really ever happen. What is important to track is, as you said, that the slice doesn't grow.

Done!
I can see why you chose to add a lastRemovedID. When working with small buffers in tests it feels strange that the first item only can be used as a reference point for replaying the items that follow it. But that's less of a thing with more realistic replay counts.

Tagged a release in our fork, and released our service with it to stage this morning, everything works as expected so far:

Let me know if there's anything you want me to fix in the PR.

That graph looks really good now! I assume the performance of the new version appears in the graph after 8 AM.

Yep, I fixed the linting and test errors.


Add jacoco coverage

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

[ ] Make sure the PR title is formatted like:
[BEAM-<Jira issue #>] Description of pull request
[ ] Make sure tests pass via mvn clean verify. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
[ ] Replace <Jira issue #> in the title with the actual Jira issue
number, if there is one.
[ ] If this contribution is large, please file an Apache
Individual Contributor License Agreement.

Changes Unknown when pulling 5537f71bd27971c85319884fe06dfc63994409f4 on swegner:coveralls-test into ** on apache:master**.

@kennknowles do you know much about our coveralls setup? I see we're reporting coverage from jenkins rather than travis. Coveralls also supports travis, and from what I can tell Travis has a couple advantages:

Infers correct pull request context to associate with coverage data.
Also runs on fork repositories since the .travis.yml config is part of source.

Our coveralls setup is in the maven config. I don't know whether PR context is inferred.

RE: PR context, I noticed in the coveralls build list, all PR's from jenkins are listed as #99999. IIRC, this is some hack @davorbonaci needed to add to make it work. The build for this PR from travis has its PR number correct.
Not sure if there are other pros- and cons- of collecting from Travis vs Jenkins. @jasonkuster, do you know?

Don't know of anything meaningful that hasn't been mentioned. From my perspective though, since Jenkins is where the bulk of our testing effort is, it would be good to keep what we can there. If you think the pros outweigh that though, feel free to push back.


Automatic PR of restapi_auto_jugeorge_master into dev

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/2319

Closing for now, in response to #1057. These changes will be incorporated shortly.


Confusing behavior: allow_stdin overrides allow_read

Hi.
I was really puzzled why reading from a file failed with a combination of  allow_stdin and allow_read and eventually found that allow_stdin negated allow_read.
I think it's confusing and the documentation should make it clear that those methods are exclusive.

Hi! Thanks for the issue.
I agree the documentation needs to be more clear about what happens when two rules have overlapping syscalls.
What's happening is that allow_stdin allows read in the specific case when the fd parameter is 0, whereas allow_read allows read syscalls unconditionally. When you pass both of these rules to libseccomp, it discards the allow_stdin specific rule and allows all reads.
I thought this behavior wasn't great for extrasafe because it's easy to accidentally enable read globally with some other ruleset but mistakenly believe (due to calling allow_stdin) that it's disabled except for the specific cases you allowed.
Do you think printing a warning to stderr would be useful here, besides improving the documentation? "A parameterized rule would have been overridden by the current rule, so the global rule is being ignored."

Thak you for your reply.
Is it possible to just return ConditionalNoEffectError in this case?
I think it is the most consistent behavior.

Do you think printing a warning to stderr would be useful here, besides improving the documentation? "A parameterized rule would have been overridden by the current rule, so the global rule is being ignored."

I don't think it's a good idea because it won't work unless stderr is allowed.

I don't think it's a good idea because it won't work unless stderr is allowed.

That's a very good point :)
Returning ConditionalNoEffectError in this case might be the best option since you're the second person to bring it up. I was also briefly exploring switching to seccompiler from libseccomp which might make it easier to control this behavior - as far as I'm aware it's not inherent to seccomp itself, just how libseccomp generates its bpf programs.
Another option would be adding a typestate <ReadParameterized, WriteParameterized, ...etc.> parameter to the SystemIO struct that would not implement calling allow_read after calling allow_stdin but would still allow allow_stdout and allow_fd etc, but that's a lot of work and would have to be duplicated for any other ruleset that works like this, and wouldn't be there by default for user-defined rulesets.
Another option might be adding a .strict_mode() that detects this case and errors only when strict mode is enabled, but I don't really like the idea of having a strict mode - it would be confusing and the current mode isn't particularly unstrict anyway.
I guess given that the user can always define their own RuleSet it's okay to error in this case as well.

Hi, I'm sorry it's taken me a year to get back to work on this project. I just pushed a branch with some cleanups I've been working on in preparation for seccompiler and landlock. It looks like there are some build errors with code coverage but I think this does solve the issue.
Also, it looks like my first comment here somehow got corrupted or maybe I was just very tired - there's an edit on it that seems strange.  ¯_(ツ)_/¯

I'm going to close this since you thumbs-up'd my previous comment.
I'm also going to push a new release to crates.io today. If you run into more issues with this feel free to reopen this issue or make a new one!
Thanks again for the report, and sorry it took me so long to get back to it.


Add ref.listenSelf

Is your feature request related to a problem? Please describe.
I'm having a StateProvider and I want to know when its value changed.
Describe the solution you'd like
Add a callback called onChanged to know when its value changed.
Describe alternatives you've considered
Create another provider to watch it.

I think this is what you are after:
https://riverpod.dev/docs/concepts/reading/#using-reflisten-to-react-to-a-provider-change

This is already a thing: ref.listen

Yes. Provider listen itself

Fair enough. I'll probably name it "listenSelf" though
And add "refreshSelf" too

Glad to see it. Thanks!

Do you remember why you decided not to implement refreshSelf?
If I have a Notifier whose provider is only read and not watched, invalidateSelf would not actually force it to be rebuilt unless I do another read, right?

Correct. Because that's generally how it's meant to behave

If we had refreshSelf though, it would be immediately rebuilt even if it's not watched or read. Could be useful when you would prefer to have the new state in the next line.

Reading ref.state would trigger the rebuild anyway

Oh wow so this is what I want, right?
void refreshSelf() {
  invalidateSelf();
  state;
}

because of the _element.flush() in get state, I see. Didn't give it a try but makes sense. Thanks!


Outdated code examples Storefront

The code examples for compound components are outdated as we renamed them a while back.
Example:
Outdated: https://eds.equinor.com/components/navigation-tabs/
Correct: https://eds-storybook-react.azurewebsites.net/?path=/docs/components-tabs--default
This was reported by one of our users in #eds-design-system on Slack.

Not sure if there's a point in fixing this in the Storefront when it can be fixed in the transition to Zeroheight.

Closed as we are deprecating the Storefront in favour of Zeroheight.


Command Line arguments?

Is it possible to preconfigure restic-browser to open a backup repo via a command line argument? I would be nice to be able to create a shortcut that opens the application and prompts the user for a password.

Hi. There's no command line support right now, but Restic Browser already picks up relevant restic ENV values. See https://github.com/emuell/restic-browser/issues/34#issuecomment-1304855874 for details.
If we'd allow command line args, there actually would be a lot of them to configure AWS, B2 and RClone repositories, such as AWS_ACCESS_KEY_ID, B2_ACCOUNT_ID, AZURE_ACCOUNT_NAME...


STYLE: Use unique_ptr for Elastix and Transformix Pimpl data members

Following C++ Core Guidelines, April 10, 2022, "A raw pointer (a T*) is non-owning":
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#r3-a-raw-pointer-a-t-is-non-owning

It appears that this updated pattern should work with the modern compilers.
Despite SimpleITK having a large number of ITK filters, this code is relatively small and homogeneous. In order to keep SimpleITK consistent, and not have this directory diverge please update the other SimpleITK Pimpl classes to this new pattern.
Also consider if these elastix Pimpl classes are actually needed. The other SITK pimple classes are base classes with derived templated classes for different type. I don't see that used here.


Manage errors at decoding

There is currently no check on the validity of the values (as in, varint decoded is bigger than actually possible to fit in the struct)
Need to do this for any kind of varint.
Impact on decode_varint (if it doesnt fit, return error) and on the transformation done to fit the final structure in pb_decode

Wiredecoder should check for the fully encoded tag, which would eliminate errors in wirevalues naturally.

Wiredecoder should check for the fully encoded tag, which would eliminate errors in wirevalues naturally.

Done indirectly with acb2a57c1612a3fb096efc383ac1b3ffa8548d93


Serving runtime not properly configured in deployed after TrustyAI service

When serving runtimes are deployed after the TrustyAI service CR is applied, in some cases it might not be properly configured.

I'll just add some information as to how I'm reproducing this from the dashboard perspective:

Create a new Data Science Project (i.e. k8s namespace)
Go to the settings tab and click Enable  TrustyAI
Create a Data Connection - I'm using the rhods-public S3 bucket
Create a Model Server
Create a Model (i.e. an InferenceService in k8s)
Curl data at the model.

When it's done in the above order the curl commands error out. If you perform this sequence but move step 2 to last, everything works as expected.


Create jumbled_number.cppCB

A number is said to be Jumbled if for every digit, its neighbouring digit differs by max 1.

Please accept my pull request.