[Release Radar Request] Jira CLI v1.0.0

Open Source Project name
Jira CLI
What is your project?
JiraCLI is an interactive command line tool for Atlassian Jira.
Version
1.0.0
Date
July 03, 2022
Description of breaking changes
Breaking changes from the previous beta releases.

Replace limit flag with paginate
Append components on edit instead of overriding
Append labels on edit instead of overriding
Move board and project list to sub-command

GitHub Repo
https://github.com/ankitpokhrel/jira-cli
Website
https://github.com/ankitpokhrel/jira-cli
Link to changelog
https://github.com/ankitpokhrel/jira-cli/releases/tag/v1.0.0
Social media
https://twitter.com/ankitpokhrel/status/1543676990821105665
Anything else to add?
No response

Congrats on being featured in the MONTHOF Release Radar 🥳

Read the blog here
Check out our GitHub Tweet
Feel free to retweet Mish's tweet mentioning you
Check out our Release Radar YouTube video for the July Release Radar promo video 😄


Consider ditching push_bytes.rs entirely

We have 400 lines of really non-trivial code in that file and it seems that the only reason for its existence is to prevent a user from accidentally pushing a push of more than 4 GB... Which sane user would accidentally do that? Also, it doesn't even guarantee that all the scripts that we create are passing the size invariant because I can push 2^32 - 1 any number of times.
The result of this "check" is that it is no longer possible to push a simple byte vector using the Builder::push_slice operation.. Which is super inconvenient. I tried .into() etc, but it seems I have to really manually wrap the vector into PushBytesBuf??
let bytes = vec![1u8; 32];
script::Builder::new().push_slice(&bytes); // not possible

Yes, you can see in rust-miniscript code of the form
                let witness_script = wsh.inner_script().to_v0_p2wsh();
                let push_bytes = <&PushBytes>::try_from(witness_script.as_bytes())
                    .expect("Witness script is not too large");
                script::Builder::new().push_slice(push_bytes).into_script()

But having said this, pushing unstructured byte slices onto scripts isn't a super common operation. Keys, hashes, and arrays you can do much more easily.

I've been trying to figure this out for tens of minutes now..
    impl TryFrom<Vec<u8>> for PushBytesBuf {
        type Error = PushBytesError;

        fn try_from(vec: Vec<u8>) -> Result<Self, Self::Error> {
            // check len
            let _: &PushBytes = vec.as_slice().try_into()?;
            Ok(PushBytesBuf(vec))
        }
    }

This method suggests there should be a TryFrom<&[u8]> impl for PushBytes but the compiler isn't letting me without yeah what you suggest, the <&PushBytes>:: prefix:
                builder = builder.push_slice(PushBytes::try_from(&buf[..]).unwrap()); // invalid

But having said this, pushing unstructured byte slices onto scripts isn't a super common operation. Keys, hashes, and arrays you can do much more easily.

That depends. I guess for people entirely submerged in our API, it wouldn't. But in many cases, a 32-byte slice will come in as a Vec<u8> and converting it first into [insert the right specific hash-type] can be just an nuisance if you can easily do if buf.len() != 32 { /* error */ } instead.. Converting to [u8; 32] is even harder.
I'm implementing an ASM parser, so arguably this is a power-user use-case, but any protocol doing message passing using protobuf or something will be passing raw bytes and always having to use the newtypes can be annoying.

any protocol doing message passing using protobuf or something will be passing raw bytes and always having to use the newtypes can be annoying.

Please provide a reference where this is actually happening. In many (most?) protocols one can more easily check the input by parsing it into its appropriate newtype rather than writing manually since there is often existing machinery for that and one doesn't forget a check by accident.
Anyway, how about having a try_push_bytes(&mut self, bytes: &[u8]) -> Result<(), PushBytesError> { self.push_slice(<&PushBytes>::try_from(bytes)?); Ok(()) } ?
Should avoid the need to import the type and do funny things with <&PushBytes>:: (should we call it reverse turbo fish?)

I have also struggled with the push_bytes API, I like the try_push_bytes() idea, also we could probable improve the docs on the module and/or the two types (not exactly sure why I've never done this myself now it comes up).


Add Backend Metrics

Description
[Describe what this change achieves]
Issues Resolved
[List any issues this PR will resolve]
Check List

[x] New functionality includes testing.

[ ] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented.

[ ] New functionality has javadoc added
[ ] New functionality has user manual doc added

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #268 (5539bf2) into develop (2395c67) will increase coverage by 12.44%.
The diff coverage is n/a.

@@              Coverage Diff               @@
##             develop     #268       +/-   ##
==============================================
+ Coverage      67.36%   79.81%   +12.44%
==============================================
  Files            134       60       -74
  Lines           4275     1645     -2630
  Branches         737      421      -316
==============================================
- Hits            2880     1313     -1567
+ Misses          1182      324      -858
+ Partials         213        8      -205

Flag
Coverage Δ

dashboards-notifications
79.81% <ø> (ø)

opensearch-notifications
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...arch/notifications/model/ChannelMessageResponse.kt

...opensearch/notifications/util/SecureIndexClient.kt

...arch/notifications/index/NotificationEventIndex.kt

...opensearch/notifications/channel/ChannelFactory.kt

...opensearch/notifications/index/EventQueryHelper.kt

...h/notifications/model/NotificationConfigDocInfo.kt

...fications/action/DeleteNotificationConfigAction.kt

.../opensearch/notifications/throttle/CounterIndex.kt

...rg/opensearch/notifications/spi/NotificationSpi.kt

.../org/opensearch/notifications/spi/utils/Helpers.kt

... and 50 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2395c67...5539bf2. Read the comment docs.

Could you describe the metrics that you are collecting? Maybe it's even better if you add/update the related document for metrics to repo?

Could you describe the metrics that you are collecting? Maybe it's even better if you add/update the related document for metrics to repo?

Will update docs in repo and PR description. Didn't even realize the PR body was default


tensorflow.map_fn support in Custom Layer.

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 Pro
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: No
TensorFlow installed from (source or binary):  binary
TensorFlow version (use command below): 2.1.0
Python version: 3.7.8
Bazel version (if compiling from source): None
GCC/Compiler version (if compiling from source): None
CUDA/cuDNN version: 10.1.243 / 7.6.5
GPU model and memory: GeForce GTX 950M 2GB

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with:

TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

Describe the current behavior
I am trying to create a custom layer that calculates the forward kinematics for a robotic arm using 'DH parameters'. In my code, I am using the 6 joint angles as the input of the custom layer (Kinematics_Physics) and I am using tensorflow.map_fn to iteratively calculate the forward kinematics of each set of angles in the input. My goal is to set the 'DH parameters' as the trainable weights and train a model to optimize the 'DH parameters'.  when I test the custom layer with tf.Variable as input it works without any error. Also, when I used the custom layer wrapped in a tensorflow.keras.layers.Lambda layers, there are no errors but obviously the weights of my custom layer is not visible to the tensorflow. as can be seen from the model.summary() output. However when I try to use the custom layer as it is in the model, tensorflow throws the error as "TypeError: 'Tensor' object cannot be interpreted as an integer".
Describe the expected behavior
There could be something wrong in my implementation of the custom layer or probably the Tensorflow API doesn't support the map_fn for layers in the Functional API for building models. Is there a workaround for this problem? Any recommendations?
Standalone code to reproduce the issue
import tensorflow as tf
print(tf.__version__)
from tensorflow import math as m
from tensorflow.keras import layers

class Kinematics_Physics(layers.Layer):

    def __init__(self,):
        super(Kinematics_Physics, self).__init__()
        # define initial variables
        self.TF_PI = tf.constant(3.1415926535897932, dtype =tf.float32)
        self.TF_180 = tf.constant(180.0, dtype =tf.float32)
        self.TINY_VALUE = tf.constant(1e-6, dtype =tf.float32)
        self.out_Pose=tf.Variable([0,0,0,0,0,0],  dtype =tf.float32, trainable = False)
        self.A_Deg = tf.Variable(0,dtype = tf.float32, trainable = False)
        self.B_Deg = tf.Variable(0,dtype = tf.float32, trainable = False)
        self.C_Deg = tf.Variable(0,dtype = tf.float32, trainable = False)
        self.sA = tf.Variable(0,dtype = tf.float32, trainable = False)
        self.cA = tf.Variable(0,dtype = tf.float32, trainable = False)
        self.b = tf.Variable(tf.zeros((4,4),dtype = tf.float32),dtype = tf.float32, trainable = False)

        # initial dh parameters: 6 x 4 parameters for a robotic arm with 6 joints.
        dh = tf.constant([  [0 ,    180.,   -650.,  0.],
                            [270.,   90.,      0.,  0.],
                            [800.,    0.,      0.,  0.],
                            [140.,   90.,   -908.,  0.],
                            [0.,    -96.,      0.,  0.],
                            [0.,    -65.,   260.,   0.]] ,dtype = tf.float32)

        # convert to trainable weights?? for further optimization
        self.dh = tf.Variable(initial_value=dh, trainable=True)

        # initialize a buffer to calculate the the modified angles
        self.dh_processed = tf.Variable(tf.zeros(self.dh.shape,dtype = tf.float32),dtype = tf.float32, trainable = False)
        # buffer for transformation matrix
        self.trans_matrix = tf.Variable(tf.zeros((4,4),dtype = tf.float32),  dtype =tf.float32, trainable = False)

    @tf.function
    def radians(self,a):
        return m.multiply(a,(tf.divide(self.TF_PI, self.TF_180 )))

    @tf.function
    def degrees(self,a):
        return m.multiply(a,(tf.divide(self.TF_180 , self.TF_PI)))

    @tf.function
    def joint_transform(self,input_values):
        '''input: format --> 1D array = [a, α, d, θ]'''

        a = input_values[0] # a
        alpha = self.radians(input_values[1]) #convert degrees to radians
        d = input_values[2] # d
        theta = self.radians(input_values[3]) #convert degrees to radians θ

        self.trans_matrix.assign(
                                [[      m.cos(theta),             -m.sin(theta),              0,                a],
                                [m.multiply(m.sin(theta),m.cos(alpha)),  m.multiply(m.cos(theta),m.cos(alpha)), -m.sin(alpha), m.multiply(-m.sin(alpha),d)],
                                [m.multiply(m.sin(theta),m.sin(alpha)),  m.multiply(m.cos(theta),m.sin(alpha)),  m.cos(alpha),  m.multiply(m.cos(alpha),d)],
                                [        0,                             0,                     0,            1        ]])
        return self.trans_matrix

    @tf.function
    def to_pose(self,T):
        # converts the transformation matrix to pose [x, y, z, alpha ]
        if(m.abs(T[1,2]) <= self.TINY_VALUE and m.abs(T[2,2]) <= self.TINY_VALUE): #% singular : B = +-90 (Gimbal Lock)
            self.C_Deg.assign(tf.constant(0, dtype =tf.float32))
            self.B_Deg.assign(self.degrees(m.atan2( T[0,2] , m.divide_no_nan(T[2,2], m.cos(self.C_Deg)) ))) #convert radians to degree
            self.A_Deg.assign(self.degrees(m.atan2( T[1,0] ,  m.divide_no_nan(T[1,1], m.cos(self.C_Deg)) ))) #convert radians to degree

        else:
            self.A_Deg.assign(self.degrees(m.atan2(-T[0,1] , T[0,0]))) #convert radians to degree
            self.sA.assign(m.sin(self.radians(self.A_Deg)))
            self.cA.assign(m.cos(self.radians(self.A_Deg)))
            self.B_Deg.assign(self.degrees(m.atan2(T[0,2] , m.multiply(self.cA,T[0,0]) - m.multiply(self.sA,T[0,1]))))#convert radians to degree
            self.C_Deg.assign(self.degrees(m.atan2(-T[1,2] , T[2,2]))) #convert radians to degree

        self.out_Pose.assign([ T[0,3], T[1,3], T[2,3],self.A_Deg, self.B_Deg , self.C_Deg])
        return self.out_Pose

    @tf.function
    def forward_kinematics(self, theta):

        # add the dh_theta to measured theta to get the final theta
        actual_theta = m.add(self.dh[:,3] ,theta)

        # create new dh_table with modified theta for forward kinematics calculation
        self.dh_processed[:,0].assign(self.dh[:,0])
        self.dh_processed[:,1].assign(self.dh[:,1])
        self.dh_processed[:,2].assign(self.dh[:,2])
        self.dh_processed[:,3].assign(actual_theta)
        b = tf.eye(4, dtype = tf.float32)
        for i in tf.range(6):
            b = (tf.linalg.matmul(b,self.joint_transform(self.dh_processed[i])))
        return self.to_pose(b)

    def __call__(self, inputs):
        return tf.map_fn(self.forward_kinematics, inputs, parallel_iterations=True, dtype=tf.float32)

    def get_config(self):
        config = super(Kinematics_Physics, self).get_config()
        return config

if __name__ == "__main__":
    # create a list of six joint angles
    ang = tf.Variable([[20,10,20,10,10,20],[15,10,20,22,20,32]], dtype = tf.float32, trainable=False)
    # Call the Kinematics_physics layer
    fk = Kinematics_Physics()
    print("trainable weights: ", len(fk.trainable_weights))
    print("non-trainable weights: ", len(fk.non_trainable_weights))
    print("trainable_weights: ", fk.trainable_weights)
    print("forward kinematic transform: ", fk(ang[:10]))
    # Everything works up to this.

    # # create a model using functional API and custom layer wrapped in Lambda layer
    in_= layers.Input(shape=(6,))
    for_kin = layers.Lambda(Kinematics_Physics())(in_)
    model = tf.keras.models.Model(inputs=in_, outputs=for_kin)
    model.summary()
    # Works but the weights are not trainable in this case

    # create a model using functional API
    in_= layers.Input(shape=(6,))
    for_kin = Kinematics_Physics()(in_)
    model = tf.keras.models.Model(inputs=in_, outputs=for_kin)
    model.summary()
    # Error: expected behavior => should create a model with 24 trainable parameters
    #        but the error happens in tf.map_fn of Kinematics_Physics().__call__ function

Other info / logs Include any logs or source code that would be helpful to
diagnose the problem. If including tracebacks, please include the full
traceback. Large logs and files should be attached.
2020-10-23 16:16:20.529396: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
2.1.0
2020-10-23 16:16:23.763749: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll
2020-10-23 16:16:23.980052: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 950M computeCapability: 5.0
coreClock: 0.928GHz coreCount: 5 deviceMemorySize: 2.00GiB deviceMemoryBandwidth: 74.65GiB/s
2020-10-23 16:16:23.980403: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
2020-10-23 16:16:23.991313: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll
2020-10-23 16:16:23.997168: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_10.dll
2020-10-23 16:16:23.999492: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_10.dll
2020-10-23 16:16:24.006531: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_10.dll
2020-10-23 16:16:24.014520: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_10.dll
2020-10-23 16:16:24.031075: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll
2020-10-23 16:16:24.035576: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-10-23 16:16:24.037553: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2020-10-23 16:16:24.041785: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 950M computeCapability: 5.0
coreClock: 0.928GHz coreCount: 5 deviceMemorySize: 2.00GiB deviceMemoryBandwidth: 74.65GiB/s
2020-10-23 16:16:24.043759: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
2020-10-23 16:16:24.044399: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll
2020-10-23 16:16:24.045042: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cufft64_10.dll
2020-10-23 16:16:24.045625: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library curand64_10.dll
2020-10-23 16:16:24.046172: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusolver64_10.dll
2020-10-23 16:16:24.046644: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cusparse64_10.dll
2020-10-23 16:16:24.065410: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll
2020-10-23 16:16:24.070724: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-10-23 16:16:25.336541: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-10-23 16:16:25.337273: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102]      0
2020-10-23 16:16:25.338396: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0:   N
2020-10-23 16:16:25.358220: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1241] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 1370 MB memory) -> physical GPU (device: 0, name: GeForce GTX 950M, pci bus id: 0000:01:00.0, compute capability: 5.0)
trainable weights:  1
non-trainable weights:  9
trainable_weights:  [<tf.Variable 'Variable:0' shape=(6, 4) dtype=float32, numpy=
array([[   0.,  180., -650.,    0.],
       [ 270.,   90.,    0.,    0.],
       [ 800.,    0.,    0.,    0.],
       [ 140.,   90., -908.,    0.],
       [   0.,  -96.,    0.,    0.],
       [   0.,  -65.,  260.,    0.]], dtype=float32)>]
2020-10-23 16:16:27.011142: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_10.dll
forward kinematic transform:  tf.Tensor(
[[ 548.9288   -118.251114 -527.5552     47.53804   -30.621014 -144.55815 ]
 [ 511.83533   -85.7232   -492.43372    49.052803  -46.484085 -145.28882 ]], shape=(2, 6), dtype=float32)
WARNING:tensorflow:
The following Variables were used a Lambda layer's call (lambda), but
are not present in its tracked objects:
  <tf.Variable 'Variable:0' shape=(6, 4) dtype=float32>
It is possible that this is intended behavior, but it is more likely
an omission. This is a strong indication that this layer should be
formulated as a subclassed Layer rather than a Lambda layer.
Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         [(None, 6)]               0
_________________________________________________________________
lambda (Lambda)              (None, 6)                 0
=================================================================
Total params: 0
Trainable params: 0
Non-trainable params: 0
_________________________________________________________________
Traceback (most recent call last):
  File "c:/Users/ABCD/Desktop/physics_layer.py", line 127, in <module>
    for_kin = Kinematics_Physics()(in_)
  File "c:/Users/ABCD/Desktop/physics_layer.py", line 99, in __call__
    return tf.map_fn(self.forward_kinematics, inputs, parallel_iterations=True, dtype=tf.float32)
  File "C:\ProgramData\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_core\python\ops\map_fn.py", line 228, in map_fn
    for elem in elems_flat]
  File "C:\ProgramData\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_core\python\ops\map_fn.py", line 228, in <listcomp>
    for elem in elems_flat]
  File "C:\ProgramData\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_core\python\ops\tensor_array_ops.py", line 1078, in __init__
    name=name)
  File "C:\ProgramData\Anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_core\python\ops\tensor_array_ops.py", line 716, in __init__
    self._tensor_array = [None for _ in range(size)]
TypeError: 'Tensor' object cannot be interpreted as an integer

Was able to reproduce the issue with TF v2.1 and TF v2.3.
However, on running the code with TF-nightly the error changes to TypeError: Could not build a TypeSpec for <KerasTensor: shape=(None, 6) dtype=float32 (created by layer 'input_2')> with type KerasTensor. Please find the attached gist. Thanks!

It appears that you are overriding the __call__ method on Layer, but you should actually be overriding call. Try to see if it fixes your issue (there may be multiple issues with your code).

@kumar10725 I think this was resolved. when I updated your code based on @fchollet suggestion, I don't see any error. Please check the gist here.
Please verify once and close the issue. Thanks!

@kumar10725 Closing this issue as this was resolved. Please feel free to reopen if I am mistaken. Thanks!


"State attributes exceed maximum size of 16384 bytes"

Thanks for all the hard work on this amazing integration! I can see great future possibilities, the roadmap and enthusiasm put into this seem very promising :)
When having a decently sized Home Assistant setup, the A.I.-agent can suggest quite a few automations. The list of this automations may be of such size, that the recorder complains about exceeding a (default) maximum size for state attributes.
Logger: homeassistant.components.recorder.db_schema
Source: components/recorder/db_schema.py:624
integration: Recorder ([documentation](https://www.home-assistant.io/integrations/recorder), [issues](https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+recorder%22))
First occurred: 15:25:21 (2 occurrences)
Last logged: 15:25:21

State attributes for sensor.ai_automation_suggestions exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored

Perhaps we need to think about a future-proof way to store the result of the generated suggestions. While it is not a breaking problem that the entity attributes will not be stored, one could assume that it is not an ideal way to (ab)use attributes. At this moment, for example the template {{ state_attr("sensor.ai_automation_suggestions", "suggestions") }} results in an output length of 34621 characters for me, exceeding the 16 kB default limit for the recorder.

Updated to resolve issues within this next version V1.08. Please read:  https://github.com/ITSpecialist111/ai_automation_suggester/releases/tag/V1.08


How to configure SLL and add https in this boilerplate?

I need to set up https and configure SSL in this boilerplate. How do I do this ? Which file do I change?

Setting up SSL and HTTPS has nothing to do with your front end code. It's about the server you use to serve it. There are tons of resources all over the web to help you with this.


Conv1D example

Add Conv1D example with simple Sequential Model and input_shape

Closing this PR as this not against master, please open a new PR against master
CC @mihaimaruseac


refactor: update try it out page

Updates Try it Out page

Review:
Not sure how to pass maxVerifications and environment (staging/production) to the widget, as I can see there is no such props.

https://user-images.githubusercontent.com/89008845/230667773-104c2612-7449-4e08-8bbc-a7d8dbfe5c9c.mp4

@paolodamico I've added try-callback page to match flow from this Figma screen. But I cant't fully test it.
I've set up url parameters here, when I use Sign In link, I can see QR code on id.worldcoin.org/authorize page, but scanning it with simulator failing. May I ask you to take a look and adjust handling at /try-callback serverSideProps after a successful connection?


Update CONTRIBUTING.md

This PR contains zero changes and no description so I'm assuming it was submitted accidentally.


Fix V8 links

This pull request swaps v8eval links with actual V8 links.

@VeNoMouS Kindly asking you why this change is not accepted. I did not know v8eval until I see this project, and it's very frustrating the link leads me to V8's website.


Suggest adding link to resume command

Document is pretty good.  However there is a companion command to this one, which is to resume the service after it's been suspended. Suggest adding a link to that somewhere in this article.
https://docs.microsoft.com/en-us/powershell/module/sharepoint-server/resume-spenterprisesearchserviceapplication?view=sharepoint-ps

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 543f3c98-c159-39db-dba3-2a5ad826dafe
Version Independent ID: 72b0dab6-70e6-e2ba-dde6-160afe0d70f9
Content: Suspend-SPEnterpriseSearchServiceApplication (SharePointServer)
Content Source: sharepoint/sharepoint-ps/sharepoint-server/Suspend-SPEnterpriseSearchServiceApplication.md
Product: sharepoint-2016-powershell
GitHub Login: @Techwriter40
Microsoft Alias: pamgreen

Thanks for your feedback @Lady-MCSE


InkWell prevents proper widget disposal in PageView.builder

Steps to reproduce

Create a PageView with multiple pages
Add a widget containing an InkWell to one of the pages
Navigate between pages in the PageView
Observe that the widget with InkWell doesn't dispose properly

Expected results
Widget should properly dispose when navigating away from its page in PageView
Actual results
Widget doesn't dispose when using InkWell, but works correctly when using GestureDetector instead.
Additional Context
Switching from InkWell to GestureDetector resolves the disposal issue:
GestureDetector( // Works correctly
  onTap: () {},
  child: Container(
    child: Text('Test'),
  ),
)

Code sample
Code sample
class PageViewExample extends StatefulWidget {
  @override
  State<PageViewExample> createState() => _PageViewExampleState();
}

class _PageViewExampleState extends State<PageViewExample> {
  final PageController _controller = PageController();
  final int pageCount = 3;

  @override
  void dispose() {
    _controller.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: PageView.builder(
        controller: _controller,
        itemCount: pageCount,
        itemBuilder: (context, index) {
          return ProblematicWidget(pageNumber: index);
        },
      ),
    );
  }
}

class ProblematicWidget extends StatefulWidget {
  final int pageNumber;

  const ProblematicWidget({required this.pageNumber});

  @override
  State<ProblematicWidget> createState() => _ProblematicWidgetState();
}

class _ProblematicWidgetState extends State<ProblematicWidget> {
  @override
  void initState() {
    super.initState();
    print('Initializing page ${widget.pageNumber}');
  }

  @override
  void dispose() {
    print('Disposing page ${widget.pageNumber}'); // Never called with InkWell
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Center(
      child: InkWell( // Problem: Prevents disposal
        onTap: () {},
        child: Container(
          padding: EdgeInsets.all(20),
          color: Colors.blue,
          child: Text(
            'Page ${widget.pageNumber}',
            style: TextStyle(color: Colors.white),
          ),
        ),
      ),
    );

    // Solution: Use GestureDetector instead
    return Center(
      child: GestureDetector(
        onTap: () {},
        child: Container(
          padding: EdgeInsets.all(20),
          color: Colors.blue,
          child: Text(
            'Page ${widget.pageNumber}',
            style: TextStyle(color: Colors.white),
          ),
        ),
      ),
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
[Paste your logs here]

Flutter Doctor output
Doctor output
flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.24.4, on macOS 15.0 24A335 darwin-arm64, locale en-DE)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 16.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2023.1)
[✓] IntelliJ IDEA Community Edition (version 2023.3.8)
[✓] VS Code (version 1.95.3)
[✓] Connected device (5 available)
[✓] Network resources

• No issues found!

@bschmalb
I think this is similar to https://github.com/flutter/flutter/issues/89212, so please follow-up in it for further updates.
Closing this as a duplicate. If you disagree, write in comments and I'll reopen it.


fix onecluster package_dir bug

In bin/config.onecluster.ini.template, package_dir is ${CMAKE_CURRENT_SOURCE_DIR}/src/tools/webstudio/app_package/local/packages, however the path of webstudio is $DSN_ROOT/webstudio.
This bug will cause package not found error and I create a symbolic link to fix it.

@jiguanglizipao,
Thanks for your contribution.
To ensure that the project team has proper rights to use your work, please complete the Contribution License Agreement at https://cla.microsoft.com.
It will cover your contributions to all Microsoft-managed open source projects.
Thanks,
Microsoft Pull Request Bot

@jiguanglizipao, thanks for signing the contribution license agreement. We will now validate the agreement and then the pull request.
Thanks, Microsoft Pull Request Bot


Exit  Code 125. The Parameter is Incorrect.  DockerDo command not recognised.

PLEASE DO NOT INCLUDE ANY PASSWORDS OR TOKENS IN YOUR ISSUE!!!
When running the output of the new-bccontainerwizard on my machine regardless of which settings I strip back I keep getting "The parameter is incorrect.  ExitCode: 125' error.  My system does not seem to recognise the DockerDo command used with the New-NavContainer.ps1 script line 1951 character 15.
Scripts used to create container and cause the issue
$containerName = 'bc215'
$credential = Get-Credential -Message 'Using UserPassword authentication. Please enter credentials for the container.'
$auth = 'UserPassword'
$artifactUrl = Get-BcArtifactUrl -type 'OnPrem' -version '21.5' -country 'gb' -select 'Latest'
$licenseFile = 'D:\NavData\Licences\2023-07\BC21.flf'
New-BcContainer `
    -accept_eula `
    -containerName $containerName `
    -credential $credential `
    -auth $auth `
    -artifactUrl $artifactUrl `
    -licenseFile $licenseFile `
    -memoryLimit 4G `
    -updateHosts

Full output of scripts
PS C:\WINDOWS\system32> C:\Users\MichaelC\Documents\bc215.ps1
BcContainerHelper is version 5.0.5
BcContainerHelper is running as administrator
HyperV is Enabled
UsePsSession is True
Host is Microsoft Windows Server 2019 Standard - 10.0.17763.4377
Docker Client Version is 24.0.4
Docker Server Version is 24.0.4
Removing entries from hosts
Removing bc215 from container hosts file
Removing bc215-* from container hosts file
Removing Desktop shortcuts
Removing d:\bccontainerhelper\Extensions\bc215
Fetching all docker images
Fetching all docker volumes
Using image mcr.microsoft.com/businesscentral:10.0.17763.4377
Creating Container bc215
Style: onprem
Multitenant: No
Version: 21.5.53619.53819
Platform: 21.0.53597.53794
Generic Tag: <IP_ADDRESS>
Container OS Version: 10.0.17763.4377 (ltsc2019)
Host OS Version: 10.0.17763.4377 (ltsc2019)
Using process isolation
Using locale en-GB
Disabling the standard eventlog dump to container log every 2 seconds (use -dumpEventLog to enable)
Using license file D:\NavData\Licences\2023-07\BC21.flf
Files in d:\bccontainerhelper\Extensions\bc215\my:
- AdditionalOutput.ps1
- license.flf
- MainLoop.ps1
- SetupVariables.ps1
- updatehosts.ps1
Creating container bc215 from image mcr.microsoft.com/businesscentral:10.0.17763.4377
a4b48cbe9487c310b5981e42dd585b45750479bfe65bf11d26159b6779f92f60
New-BcContainer Telemetry Correlation Id: a2da87f2-19b5-42ec-899e-dff555e5f74c
DockerDo : docker: Error response from daemon: hcs::CreateComputeSystem a4b48cbe9487c310b5981e42dd585b45750479bfe65bf11d26159b6779f92f60: The parameter is incorrect.
ExitCode: 125
Commandline: docker run --volume "c:\bcartifacts.cache:c:\dl" --label nav= --env isBcSandbox=N --label version=21.5.53619.53819 --label platform=21.0.53597.53794
--label country=gb --env artifactUrl=https://bcartifacts.azureedge.net/onprem/21.5.53619.53819/gb --env licenseFile="c:\run\my\license.flf" --name bc215 --hostname
bc215 --env auth=NavUserPassword --env username="michaelc" --env ExitOnError=N --env locale=en-GB --env databaseServer="" --env databaseInstance="" --volume
"d:\bccontainerhelper:d:\bccontainerhelper" --volume "d:\bccontainerhelper\Extensions\bc215\my:C:\Run\my" --isolation process --restart unless-stopped --env
filesOnly=False --memory 4G --env enableApiServices=Y --env useSSL=N --volume "c:\windows\system32\drivers\etc:C:\driversetc" --env securePassword=76492d1116743f0423413b
16050a5345MgB8AEgARgBZAEoAeQBXAGQAUQA5ADEAcQA4ADgAWgA0AGwAbwBxAHoAUgA5AEEAPQA9AHwANAAzAGMAZgA1AGIAOQBkAGMAMwA0ADEAYwBhADAANwBmADAAMwBhADMANAA1ADUAZgAxAGIANgBjADcAYQBhAGI
AYwA4ADUAMgA3AGEAYgBhADAAMgAxADQANQBmADEAOAA0AGEAMgA0ADUAMAA3ADIAZgBhADQANABhADAAYwA= --env passwordKeyFile="c:\run\my\aes.key" --env removePasswordKeyFile=Y --env
accept_eula=Y --env accept_outdated=Y --detach mcr.microsoft.com/businesscentral:10.0.17763.4377
At C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\5.0.5\ContainerHandling\New-NavContainer.ps1:1951 char:15
+ ...       if (!(DockerDo -accept_eula -accept_outdated:$accept_outdated - ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,DockerDo
...

Screenshots
N/A
Additional context
I am running Chocolately CLI v2.0.0 with its Docker Engine 24.0.4, which mimics the standard docker command but does not seem to be able to handle the DockerDo command.  Would I be to amend the scripts to replace the DockerDo commands with another equivalent?

Where do you see the command not recognised?
I also use Docker Engine (not through choco) it includes the Docker client and that works fine.
Error 125 is typically a compatibility issue with the computer or some software (often antivirus), frequently fixed by adding -isolation hyperv


Trying to pull a icloud calendar, doesnt populate

Anyone know how to fix this:
It happens when im trying to import a family calendar from icloud.

0|MagicMirror  | [2024-12-06 13:43:06.276] [LOG]   [CALEXT2] calendar:family >> Missing parameter value in 'X-APPLE-STRUCTURED-LOCATION;VALUE=URI;X-ADDRESS=200 East State Street\nMedia PA 19063\nUnited States;X-APPLE-ABUID="Dr. George Dakwar    MD’s Home"::;X-APPLE-MAPKIT-HANDLE=CAES/QEI2TIaEgm87DJXYfVDQBGh3udTothSwCJ0Cg1Vbml0ZWQgU3RhdGVzEgJVUxoMUGVubnN5bHZhbmlhIgJQQSoPRGVsYXdhcmUgQ291bnR5MgVNZWRpYToFMTkwNjNSCkUgU3RhdGUgU3RaAzIwMGIOMjAwIEUgU3RhdGUgU3RyDUNyb3plciBIZWFsdGgqDjIwMCBFIFN0YXRlIFN0Mg4yMDAgRSBTdGF0ZSBTdDIQTWVkaWEsIFBBICAxOTA2MzINVW5pdGVkIFN0YXRlczg5QARQAVonCiUSEgm87DJXYfVDQBGh3udTothSwBjZMiC30qG+m9+YlOgBkAMB;X-APPLE-RADIUS=73.28578642816773;X-APPLE-REFERENCEFRAME=1;X-TITLE=200 East State Street'
0|MagicMirror  | [2024-12-06 13:43:07.222] [LOG]   [CALEXT2] calendar:2 >> Scanned: 35, Selected: 35
0|MagicMirror  | [2024-12-06 13:44:05.915] [LOG]   [CALEXT2] calendar:test >> Scanning start with interval:60000
0|MagicMirror  | [2024-12-06 13:44:06.278] [LOG]   [CALEXT2] calendar:family >> Scanning start with interval:60000
0|MagicMirror  | [2024-12-06 13:44:06.671] [LOG]   [CALEXT2] calendar:test >> Scanned: 5, Selected: 5
0|MagicMirror  | [2024-12-06 13:44:07.327] [LOG]   [CALEXT2] calendar:family >> Missing parameter value in 'X-APPLE-STRUCTURED-LOCATION;VALUE=URI;X-ADDRESS=200 East State Street\nMedia PA 19063\nUnited States;X-APPLE-ABUID="Dr. George Dakwar    MD’s Home"::;X-APPLE-MAPKIT-HANDLE=CAES/QEI2TIaEgm87DJXYfVDQBGh3udTothSwCJ0Cg1Vbml0ZWQgU3RhdGVzEgJVUxoMUGVubnN5bHZhbmlhIgJQQSoPRGVsYXdhcmUgQ291bnR5MgVNZWRpYToFMTkwNjNSCkUgU3RhdGUgU3RaAzIwMGIOMjAwIEUgU3RhdGUgU3RyDUNyb3plciBIZWFsdGgqDjIwMCBFIFN0YXRlIFN0Mg4yMDAgRSBTdGF0ZSBTdDIQTWVkaWEsIFBBICAxOTA2MzINVW5pdGVkIFN0YXRlczg5QARQAVonCiUSEgm87DJXYfVDQBGh3udTothSwBjZMiC30qG+m9+YlOgBkAMB;X-APPLE-RADIUS=73.28578642816773;X-APPLE-REFERENCEFRAME=1;X-TITLE=200 East State Street'

I don't have an icloud. Could you provide a calendar (without any sensitive data) which also shows this problem?
Does the problem occure when you are using the default calendar?

I deleted the 3 events that were causing it before copying them to another calendar, I cant replicate it anymore. It seemed like it was caused by ' being in the location field.


Project folder structure in public assets path

I'm hoping this is just a silly mistake on my part and this is an easy fix. When building my site, React Static serves my JavaScript files from URLs that mirror my project structure. Shockingly, this goes all the way to root. Like:
<script defer type="text/javascript" src="/templates/vendors~Users/my.user.name/my.project.name/src/routes/home/Home.somehash.js"></script>

I've messed with a few of the path config settings but that only seems to change what these URLs are prefixed with. Any suggestions?

Ah, it sounds like this is known issue that will be fixed in the next major release hopefully: https://spectrum.chat/react-static/general/my-machines-local-path-is-being-added-to-assets-during-build~651032e1-5e01-4a7a-afcb-ec5d5938f913

Correct. It's a known issue. Best bet is to downgrade to 7.1.0 if you need to build right now :)

Thanks @SleeplessByte that worked for me!

Closing in favor of #1375


Segfault when viewing pixel history

Description
Checking the pixel history crashes RD.
Steps to reproduce

Open jasper.rdc.zip
navigate to draw call 330
click "History" for any selected pixel

Environment

RenderDoc version: 1.13
Operating System: Linux (Solus distro)
Graphics API: Vulkan

GPU: Intel Corporation UHD Graphics 630 (Mobile)

I can't reproduce the crash, but the capture you provided was made on an nvidia GPU not an Intel one so I'm not sure if you uploaded the wrong capture? Since the filename is intel.rdc and you said that you were using an Intel GPU in your report.
I only tested on windows/nvidia because my linux/nvidia system is heavily out of date. I'm updating it now in the background but I don't think there's a high chance that this will reproduce there given nvidia tends to use largely the same driver on both OSs.
Can you say exactly which nvidia driver you're using? The capture was made on 465.27, is that what you're using - can you try updating to a more recent driver?

Oh, sorry about the confusion! Now trying to force Intel in RD's options, it fails to launch the app -
rd-intel.log
We've also got people (like @cwfitzgerald) being able to load the RD capture and see the pixel history, but it shows nonsensical stuff like

(it says "depth clipped", but also that the texture value changes, and the depth value shouldn't be clipped there)
Here are the launch instructions for the app, in case you want to see it yourself.
# install git, rust, and cargo
git clone https://github.com/gfx-rs/wgpu
cd wgpu/player
cargo run -- <trace_folder>

Where the "trace_folder" can be extracted from
wgpu-trace.zip
This will build the app and run it, also spewing the command line you could put into RD:

Running /hub/moz/wgpu/target/debug/play /hub/bugs/jasper-bug/wgpu-trace/

I don't understand what you mean about forcing Intel. RenderDoc has no way of forcing an application to use a particular GPU, and the log you posted doesn't show any indication that a particular GPU is being forced for replay. The nvidia GPU is getting picked up as the closest match.
Jasper has reported the issue to me separately. Something in the capture is non-deterministic which breaks the pixel history as it relies on repeatable results. From what Jasper has said this happens in the application as well so it doesn't seem like a RenderDoc bug.
Please don't move the goalposts and try to change what the issue is about. Are you actually seeing a crash when running the pixel history and can you explain exactly how you get it and how to reproduce? You said that it happened with the original capture but I can't reproduce that - are you able to get a crash with the original capture yourself when running on an nvidia GPU? If so what driver are you running on? The log you posted shows a device lost error but not a crash so I'm unclear on if that's the same bug or a different one.

I'm talking about "Settings ->Replay -> GPU Selection Override" thing.

Please don't move the goalposts and try to change what the issue is about.

I don't see any goalposts moved. Clicking on pixel history crashes RD for me, it's still the original issue. The extra info I provided about nonsensical results are to help investigating this. My assumption was that RD goes into some kind of undefined behavior, which results in a crash for me but garbage on other machines.

You said that it happened with the original capture but I can't reproduce that - are you able to get a crash with the original capture yourself when running on an nvidia GPU? If so what driver are you running on? The log you posted shows a device lost error but not a crash so I'm unclear on if that's the same bug or a different one.

Yes, it crashes when doing the steps described in the body of this issue. My NV driver version in 465.27.

As indicated by the section of the settings, that override only applies to the replay and has no effect on what physical device the application selects.
The inconsistent contents of the pixel history are not related to the crash and they match up with the capture itself and indeed the application which behaves unpredictably or produces nonsense results. That's why I didn't want to confuse the issue and stick just to the crash. From talking to Jasper separately that problem looks like an nv shader compiler bug.
I was able to get my nvidia linux system updated and did repro the crash. I'm actually not sure how I was able to open it on windows (or the other person you mentioned), but this looks like the bug fixed by #2244 with pixel history on imageless framebuffers. If you update to a recent nightly build or preferably to the v1.14 released since this bug was opened that should work correctly.


io_uring_peek_batch_cqe does not populate cqe over FFI consistently!

After couple months of segfaults(over interpreter) narrowed the bug down to io_uring_peek_batch_cqe not populating cqe consistently. The first loop always runs fine after its anyone's guess as to when it will segfault from not populating cqe

while counter := ((io_uring_submit(ring) if io_uring_sq_ready(ring) else 0) + counter - cq_ready):

    # get count of how many event(s) are ready and fill `cqe`
    while not (cq_ready := io_uring_peek_batch_cqe(ring, cqe, counter)):
        io_uring_wait_cqe_nr(ring, cqe, 1)  # wait for at least `1` event to be ready.

    for i in range(cq_ready):
        # do stuff ...

What's the sequence that reproduces it? I have no issues here, and since it's not a header function, we should be running the exact same thing.
I am, once again, suspicious of the other code! But if you can boil it down to something that reproduces it reliably for you, then it'd be easier to take a look at. You can also try and re-compile liburing and add:
assert(i == count);
to src/queue.c:io_uring_peek_batch_cqe() before where it does
return count;
as that should be the only way that we'd ever return a value that doesn't match what we already looped and filled it with. In diff form, it'd look like:
diff --git a/src/queue.c b/src/queue.c
index c4360614ecf3..b74f984fa631 100644
--- a/src/queue.c
+++ b/src/queue.c
@@ -1,6 +1,7 @@
 /* SPDX-License-Identifier: MIT */
 #define _POSIX_C_SOURCE 200112L

+#include <assert.h>
 #include "lib.h"
 #include "syscall.h"
 #include "liburing.h"
@@ -182,6 +183,7 @@ again:
 		for (;head != last; head++, i++)
 			cqes[i] = &ring->cq.cqes[(head & mask) << shift];

+		assert(count == i);
 		return count;
 	}

Oh, and you'd need to configure liburing as:
$ ./configure --use-libc

for that to work.

I have already been compiling python liburing with --use-libc

Its happens over FFI though! So in C I am sure there isn't an issues with it.
I have changed the code to use io_uring_for_each_cqe vs io_uring_peek_batch_cqe and it doesn't segfault, though its not perfect yet.
while counter := ((io_uring_submit(ring) if io_uring_sq_ready(ring) else 0) + counter - cq_ready):

    # get count of how many event(s) are ready and fill `cqe`
    while not (cq_ready := io_uring_cq_ready(ring)):
        io_uring_wait_cqe_nr(ring, cqe, 1)  # wait for at least `1` event to be ready.

    io_uring_for_each_cqe(ring, cqe)

    for i in range(cq_ready):
        # do stuff ...

You can see it effect this library https://github.com/YoSTEALTH/Shakti

< I have changed the code to use io_uring_for_each_cqe vs io_uring_peek_batch_cqe and it doesn't segfault, though its not perfect yet.
That doesn't really help me, as I'd be interested in figuring out if there is an issue with io_uring_peek_batch_cqe(). If you could try what I suggested, that'd be awesome, thanks!

Added that C code and compiled, getting:
In file included from queue.c:4:
queue.c: In function ‘io_uring_peek_batch_cqe’:
queue.c:186:22: warning: comparison of integer expressions of different signedness: ‘unsigned int’ and ‘int’ [-Wsign-compare]
  186 |         assert(count == i);
      |                      ^~

Not sure if you wanting me to run some test file!. Running python test code, back to segfault. Valgrind gives:
==101814== Memcheck, a memory error detector
==101814== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
==101814== Using Valgrind-3.23.0 and LibVEX; rerun with -h for copyright info
==101814== Command: /opt/python/3.13/bin/python3 -m pytest test/event/sqe_test.py
==101814== Parent PID: 101812
==101814==
==101814== Invalid free() / delete / delete[] / realloc()
==101814==    at 0x48458CF: free (vg_replace_malloc.c:989)
==101814==    by 0x26083C: PyMem_RawFree (obmalloc.c:963)
==101814==    by 0x26083C: _PyObject_Free (obmalloc.c:2416)
==101814==    by 0x26083C: _PyObject_Free (obmalloc.c:2406)
==101814==    by 0x825AD57: Py_DECREF (object.h:947)
==101814==    by 0x825AD57: __pyx_f_6shakti_5event_3run_completion_entry (run.c:18985)
==101814==    by 0x826E36B: __pyx_f_6shakti_5event_3run_engine (run.c:19374)
==101814==    by 0x826E36B: __pyx_pf_6shakti_5event_3run_run (run.c:18604)
==101814==    by 0x826E36B: __pyx_pw_6shakti_5event_3run_1run (run.c:18404)
==101814==    by 0x1E94E3: _PyObject_MakeTpCall (call.c:242)
==101814==    by 0x34BC6D: _PyEval_EvalFrameDefault (generated_cases.c.h:813)
==101814==    by 0x1EC31F: _PyObject_VectorcallDictTstate (call.c:146)
==101814==    by 0x1EC31F: _PyObject_Call_Prepend (call.c:504)
==101814==    by 0x2922A1: slot_tp_call (typeobject.c:9400)
==101814==    by 0x1E94E3: _PyObject_MakeTpCall (call.c:242)
==101814==    by 0x34CD51: _PyEval_EvalFrameDefault (generated_cases.c.h:1500)
==101814==    by 0x1EC31F: _PyObject_VectorcallDictTstate (call.c:146)
==101814==    by 0x1EC31F: _PyObject_Call_Prepend (call.c:504)
==101814==    by 0x2922A1: slot_tp_call (typeobject.c:9400)
==101814==  Address 0x7e2320e21060 is in a rw- mapped file segment
==101814==
==101814== Invalid read of size 4
==101814==    at 0x825AAEC: Py_INCREF (object.h:824)
==101814==    by 0x825AAEC: __pyx_f_6shakti_5event_3run_completion_entry (run.c:19099)
==101814==    by 0x826E36B: __pyx_f_6shakti_5event_3run_engine (run.c:19374)
==101814==    by 0x826E36B: __pyx_pf_6shakti_5event_3run_run (run.c:18604)
==101814==    by 0x826E36B: __pyx_pw_6shakti_5event_3run_1run (run.c:18404)
==101814==    by 0x1E94E3: _PyObject_MakeTpCall (call.c:242)
==101814==    by 0x34BC6D: _PyEval_EvalFrameDefault (generated_cases.c.h:813)
==101814==    by 0x1EC31F: _PyObject_VectorcallDictTstate (call.c:146)
==101814==    by 0x1EC31F: _PyObject_Call_Prepend (call.c:504)
==101814==    by 0x2922A1: slot_tp_call (typeobject.c:9400)
==101814==    by 0x1E94E3: _PyObject_MakeTpCall (call.c:242)
==101814==    by 0x34CD51: _PyEval_EvalFrameDefault (generated_cases.c.h:1500)
==101814==    by 0x1EC31F: _PyObject_VectorcallDictTstate (call.c:146)
==101814==    by 0x1EC31F: _PyObject_Call_Prepend (call.c:504)
==101814==    by 0x2922A1: slot_tp_call (typeobject.c:9400)
==101814==    by 0x1EBC6E: _PyObject_Call (call.c:361)
==101814==    by 0x1EBC6E: PyObject_Call (call.c:373)
==101814==    by 0x34A76D: _PyEval_EvalFrameDefault (generated_cases.c.h:1353)
==101814==  Address 0x8 is not stack'd, malloc'd or (recently) free'd
==101814==
==101814==
==101814== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==101814==    at 0x4A13E44: __pthread_kill_implementation (pthread_kill.c:44)
==101814==    by 0x49BBA2F: raise (raise.c:26)
==101814==    by 0x49BBADF: ??? (in /usr/lib/libc.so.6)
==101814==    by 0x825AAEB: __pyx_f_6shakti_5event_3run_completion_entry (run.c:19046)
==101814==
==101814== HEAP SUMMARY:
==101814==     in use at exit: 5,416,878 bytes in 5,884 blocks
==101814==   total heap usage: 19,576 allocs, 13,693 frees, 38,609,088 bytes allocated
==101814==
==101814== 1 bytes in 1 blocks are definitely lost in loss record 5 of 1,768
==101814==    at 0x4842788: malloc (vg_replace_malloc.c:446)
==101814==    by 0x277D94: PyMem_RawMalloc (obmalloc.c:940)
==101814==    by 0x277D94: _PyObject_Malloc (obmalloc.c:2132)
==101814==    by 0x25618C: PyModule_ExecDef (moduleobject.c:470)
==101814==    by 0x392F2A: exec_builtin_or_dynamic (import.c:873)
==101814==    by 0x392F2A: _imp_exec_dynamic_impl (import.c:4773)
==101814==    by 0x392F2A: _imp_exec_dynamic (import.c.h:513)
==101814==    by 0x2537DE: cfunction_vectorcall_O (methodobject.c:512)
==101814==    by 0x34A76D: _PyEval_EvalFrameDefault (generated_cases.c.h:1353)
==101814==    by 0x1E993D: _PyObject_VectorcallTstate (pycore_call.h:168)
==101814==    by 0x1E993D: object_vacall (call.c:819)
==101814==    by 0x1EAEF4: PyObject_CallMethodObjArgs (call.c:880)
==101814==    by 0x397E61: import_find_and_load (import.c:3704)
==101814==    by 0x397E61: PyImport_ImportModuleLevelObject (import.c:3786)
==101814==    by 0x341EC3: builtin___import___impl (bltinmodule.c:277)
==101814==    by 0x341EC3: builtin___import__ (bltinmodule.c.h:107)
==101814==    by 0x253C6E: cfunction_vectorcall_FASTCALL_KEYWORDS (methodobject.c:441)
==101814==    by 0x1E9BB3: _PyObject_VectorcallTstate (pycore_call.h:168)
==101814==    by 0x1E9BB3: _PyObject_CallFunctionVa (call.c:552)
==101814==
==101814== 2 bytes in 2 blocks are definitely lost in loss record 8 of 1,768
==101814==    at 0x4842788: malloc (vg_replace_malloc.c:446)
==101814==    by 0x277D94: PyMem_RawMalloc (obmalloc.c:940)
==101814==    by 0x277D94: _PyObject_Malloc (obmalloc.c:2132)
==101814==    by 0x25618C: PyModule_ExecDef (moduleobject.c:470)
==101814==    by 0x392F2A: exec_builtin_or_dynamic (import.c:873)
==101814==    by 0x392F2A: _imp_exec_dynamic_impl (import.c:4773)
==101814==    by 0x392F2A: _imp_exec_dynamic (import.c.h:513)
==101814==    by 0x2537DE: cfunction_vectorcall_O (methodobject.c:512)
==101814==    by 0x34A76D: _PyEval_EvalFrameDefault (generated_cases.c.h:1353)
==101814==    by 0x355A8C: _PyEval_EvalFrame (pycore_ceval.h:119)
==101814==    by 0x355A8C: _PyEval_Vector (ceval.c:1818)
==101814==    by 0x355A8C: PyEval_EvalCode (ceval.c:598)
==101814==    by 0x344968: builtin_exec_impl (bltinmodule.c:1145)
==101814==    by 0x344968: builtin_exec (bltinmodule.c.h:556)
==101814==    by 0x253C6E: cfunction_vectorcall_FASTCALL_KEYWORDS (methodobject.c:441)
==101814==    by 0x34A76D: _PyEval_EvalFrameDefault (generated_cases.c.h:1353)
==101814==    by 0x1E993D: _PyObject_VectorcallTstate (pycore_call.h:168)
==101814==    by 0x1E993D: object_vacall (call.c:819)
==101814==    by 0x1EAEF4: PyObject_CallMethodObjArgs (call.c:880)
==101814==
==101814== 600 bytes in 1 blocks are possibly lost in loss record 1,091 of 1,768
==101814==    at 0x4842788: malloc (vg_replace_malloc.c:446)
==101814==    by 0x277D94: PyMem_RawMalloc (obmalloc.c:940)
==101814==    by 0x277D94: _PyObject_Malloc (obmalloc.c:2132)
==101814==    by 0x3855B4: _PyObject_MallocWithType (pycore_object_alloc.h:46)
==101814==    by 0x3855B4: gc_alloc (gc.c:2022)
==101814==    by 0x3855B4: _PyObject_GC_NewVar (gc.c:2061)
==101814==    by 0x285067: PyTuple_New (tupleobject.c:76)
==101814==    by 0x285067: PyTuple_New (tupleobject.c:70)
==101814==    by 0x3AD28B: r_object (marshal.c:1243)
==101814==    by 0x3AD4F8: r_object (marshal.c:1428)
==101814==    by 0x3AD2B3: r_object (marshal.c:1249)
==101814==    by 0x3AD4E2: r_object (marshal.c:1425)
==101814==    by 0x3AF5D7: read_object (marshal.c:1556)
==101814==    by 0x3AF766: marshal_loads_impl (marshal.c:1899)
==101814==    by 0x3AF766: marshal_loads (marshal.c.h:332)
==101814==    by 0x34DB04: _PyEval_EvalFrameDefault (generated_cases.c.h:1215)
==101814==    by 0x1E993D: _PyObject_VectorcallTstate (pycore_call.h:168)
==101814==    by 0x1E993D: object_vacall (call.c:819)
==101814==
==101814== 1,608 bytes in 3 blocks are possibly lost in loss record 1,570 of 1,768
==101814==    at 0x4842788: malloc (vg_replace_malloc.c:446)
==101814==    by 0x277D94: PyMem_RawMalloc (obmalloc.c:940)
==101814==    by 0x277D94: _PyObject_Malloc (obmalloc.c:2132)
==101814==    by 0x3855B4: _PyObject_MallocWithType (pycore_object_alloc.h:46)
==101814==    by 0x3855B4: gc_alloc (gc.c:2022)
==101814==    by 0x3855B4: _PyObject_GC_NewVar (gc.c:2061)
==101814==    by 0x285067: PyTuple_New (tupleobject.c:76)
==101814==    by 0x285067: PyTuple_New (tupleobject.c:70)
==101814==    by 0x3AD28B: r_object (marshal.c:1243)
==101814==    by 0x3AD4F8: r_object (marshal.c:1428)
==101814==    by 0x3AD2B3: r_object (marshal.c:1249)
==101814==    by 0x3AD4E2: r_object (marshal.c:1425)
==101814==    by 0x3AD2B3: r_object (marshal.c:1249)
==101814==    by 0x3AD4E2: r_object (marshal.c:1425)
==101814==    by 0x3AF5D7: read_object (marshal.c:1556)
==101814==    by 0x3AF766: marshal_loads_impl (marshal.c:1899)
==101814==    by 0x3AF766: marshal_loads (marshal.c.h:332)
==101814==
==101814== 2,552 bytes in 1 blocks are possibly lost in loss record 1,609 of 1,768
==101814==    at 0x4842788: malloc (vg_replace_malloc.c:446)
==101814==    by 0x277D94: PyMem_RawMalloc (obmalloc.c:940)
==101814==    by 0x277D94: _PyObject_Malloc (obmalloc.c:2132)
==101814==    by 0x28B7DF: _PyObject_MallocWithType (pycore_object_alloc.h:46)
==101814==    by 0x28B7DF: _PyType_AllocNoTrack (typeobject.c:1953)
==101814==    by 0x28B7DF: PyType_GenericAlloc (typeobject.c:1982)
==101814==    by 0x28BBF8: type_call (typeobject.c:1897)
==101814==    by 0x1E94E3: _PyObject_MakeTpCall (call.c:242)
==101814==    by 0x34BC6D: _PyEval_EvalFrameDefault (generated_cases.c.h:813)
==101814==    by 0x355A8C: _PyEval_EvalFrame (pycore_ceval.h:119)
==101814==    by 0x355A8C: _PyEval_Vector (ceval.c:1818)
==101814==    by 0x355A8C: PyEval_EvalCode (ceval.c:598)
==101814==    by 0x344968: builtin_exec_impl (bltinmodule.c:1145)
==101814==    by 0x344968: builtin_exec (bltinmodule.c.h:556)
==101814==    by 0x253C6E: cfunction_vectorcall_FASTCALL_KEYWORDS (methodobject.c:441)
==101814==    by 0x34A76D: _PyEval_EvalFrameDefault (generated_cases.c.h:1353)
==101814==    by 0x1E993D: _PyObject_VectorcallTstate (pycore_call.h:168)
==101814==    by 0x1E993D: object_vacall (call.c:819)
==101814==    by 0x1EAEF4: PyObject_CallMethodObjArgs (call.c:880)
==101814==
==101814== 4,608 bytes in 2 blocks are possibly lost in loss record 1,677 of 1,768
==101814==    at 0x4842788: malloc (vg_replace_malloc.c:446)
==101814==    by 0x4004A8D: malloc (rtld-malloc.h:56)
==101814==    by 0x4004A8D: _dlfo_mappings_segment_allocate (dl-find_object.c:217)
==101814==    by 0x4004A8D: _dl_find_object_update_1 (dl-find_object.c:671)
==101814==    by 0x4004A8D: _dl_find_object_update (dl-find_object.c:805)
==101814==    by 0x400CD1F: dl_open_worker_begin (dl-open.c:756)
==101814==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==101814==    by 0x400C0DF: dl_open_worker (dl-open.c:803)
==101814==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==101814==    by 0x400C533: _dl_open (dl-open.c:905)
==101814==    by 0x4A0D9E3: dlopen_doit (dlopen.c:56)
==101814==    by 0x4001522: _dl_catch_exception (dl-catch.c:237)
==101814==    by 0x4001678: _dl_catch_error (dl-catch.c:256)
==101814==    by 0x4A0D4C2: _dlerror_run (dlerror.c:138)
==101814==    by 0x4A0DA9E: dlopen_implementation (dlopen.c:71)
==101814==    by 0x4A0DA9E: dlopen@@GLIBC_2.34 (dlopen.c:81)
==101814==
==101814== 5,288 bytes in 1 blocks are possibly lost in loss record 1,685 of 1,768
==101814==    at 0x4842788: malloc (vg_replace_malloc.c:446)
==101814==    by 0x277D94: PyMem_RawMalloc (obmalloc.c:940)
==101814==    by 0x277D94: _PyObject_Malloc (obmalloc.c:2132)
==101814==    by 0x3855B4: _PyObject_MallocWithType (pycore_object_alloc.h:46)
==101814==    by 0x3855B4: gc_alloc (gc.c:2022)
==101814==    by 0x3855B4: _PyObject_GC_NewVar (gc.c:2061)
==101814==    by 0x284903: _PyTuple_FromArray.part.0 (tupleobject.c:380)
==101814==    by 0x351ACA: _PyEval_EvalFrameDefault (generated_cases.c.h:1374)
==101814==    by 0x1EC258: _PyObject_VectorcallDictTstate (call.c:135)
==101814==    by 0x1EC258: _PyObject_Call_Prepend (call.c:504)
==101814==    by 0x298F01: slot_tp_init (typeobject.c:9646)
==101814==    by 0x28BC66: type_call (typeobject.c:1909)
==101814==    by 0x1E94E3: _PyObject_MakeTpCall (call.c:242)
==101814==    by 0x34BC6D: _PyEval_EvalFrameDefault (generated_cases.c.h:813)
==101814==    by 0x355A8C: _PyEval_EvalFrame (pycore_ceval.h:119)
==101814==    by 0x355A8C: _PyEval_Vector (ceval.c:1818)
==101814==    by 0x355A8C: PyEval_EvalCode (ceval.c:598)
==101814==    by 0x344968: builtin_exec_impl (bltinmodule.c:1145)
==101814==    by 0x344968: builtin_exec (bltinmodule.c.h:556)
==101814==
==101814== LEAK SUMMARY:
==101814==    definitely lost: 3 bytes in 3 blocks
==101814==    indirectly lost: 0 bytes in 0 blocks
==101814==      possibly lost: 14,656 bytes in 8 blocks
==101814==    still reachable: 5,402,219 bytes in 5,873 blocks
==101814==         suppressed: 0 bytes in 0 blocks
==101814== Reachable blocks (those to which a pointer was found) are not shown.
==101814== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==101814==
==101814== ERROR SUMMARY: 9 errors from 9 contexts (suppressed: 0 from 0)
==101814==
==101814== 1 errors in context 1 of 9:
==101814== Invalid read of size 4
==101814==    at 0x825AAEC: Py_INCREF (object.h:824)
==101814==    by 0x825AAEC: __pyx_f_6shakti_5event_3run_completion_entry (run.c:19099)
==101814==    by 0x826E36B: __pyx_f_6shakti_5event_3run_engine (run.c:19374)
==101814==    by 0x826E36B: __pyx_pf_6shakti_5event_3run_run (run.c:18604)
==101814==    by 0x826E36B: __pyx_pw_6shakti_5event_3run_1run (run.c:18404)
==101814==    by 0x1E94E3: _PyObject_MakeTpCall (call.c:242)
==101814==    by 0x34BC6D: _PyEval_EvalFrameDefault (generated_cases.c.h:813)
==101814==    by 0x1EC31F: _PyObject_VectorcallDictTstate (call.c:146)
==101814==    by 0x1EC31F: _PyObject_Call_Prepend (call.c:504)
==101814==    by 0x2922A1: slot_tp_call (typeobject.c:9400)
==101814==    by 0x1E94E3: _PyObject_MakeTpCall (call.c:242)
==101814==    by 0x34CD51: _PyEval_EvalFrameDefault (generated_cases.c.h:1500)
==101814==    by 0x1EC31F: _PyObject_VectorcallDictTstate (call.c:146)
==101814==    by 0x1EC31F: _PyObject_Call_Prepend (call.c:504)
==101814==    by 0x2922A1: slot_tp_call (typeobject.c:9400)
==101814==    by 0x1EBC6E: _PyObject_Call (call.c:361)
==101814==    by 0x1EBC6E: PyObject_Call (call.c:373)
==101814==    by 0x34A76D: _PyEval_EvalFrameDefault (generated_cases.c.h:1353)
==101814==  Address 0x8 is not stack'd, malloc'd or (recently) free'd
==101814==
==101814==
==101814== 1 errors in context 2 of 9:
==101814== Invalid free() / delete / delete[] / realloc()
==101814==    at 0x48458CF: free (vg_replace_malloc.c:989)
==101814==    by 0x26083C: PyMem_RawFree (obmalloc.c:963)
==101814==    by 0x26083C: _PyObject_Free (obmalloc.c:2416)
==101814==    by 0x26083C: _PyObject_Free (obmalloc.c:2406)
==101814==    by 0x825AD57: Py_DECREF (object.h:947)
==101814==    by 0x825AD57: __pyx_f_6shakti_5event_3run_completion_entry (run.c:18985)
==101814==    by 0x826E36B: __pyx_f_6shakti_5event_3run_engine (run.c:19374)
==101814==    by 0x826E36B: __pyx_pf_6shakti_5event_3run_run (run.c:18604)
==101814==    by 0x826E36B: __pyx_pw_6shakti_5event_3run_1run (run.c:18404)
==101814==    by 0x1E94E3: _PyObject_MakeTpCall (call.c:242)
==101814==    by 0x34BC6D: _PyEval_EvalFrameDefault (generated_cases.c.h:813)
==101814==    by 0x1EC31F: _PyObject_VectorcallDictTstate (call.c:146)
==101814==    by 0x1EC31F: _PyObject_Call_Prepend (call.c:504)
==101814==    by 0x2922A1: slot_tp_call (typeobject.c:9400)
==101814==    by 0x1E94E3: _PyObject_MakeTpCall (call.c:242)
==101814==    by 0x34CD51: _PyEval_EvalFrameDefault (generated_cases.c.h:1500)
==101814==    by 0x1EC31F: _PyObject_VectorcallDictTstate (call.c:146)
==101814==    by 0x1EC31F: _PyObject_Call_Prepend (call.c:504)
==101814==    by 0x2922A1: slot_tp_call (typeobject.c:9400)
==101814==  Address 0x7e2320e21060 is in a rw- mapped file segment
==101814==
==101814== ERROR SUMMARY: 9 errors from 9 contexts (suppressed: 0 from 0)

io_uring_for_each_cqe kinda works! But there are certain time where cqe doesn't get populated as well. Its just running bunch of io_uring_prep_nop.  I am not sure how to reproduce this in C its clearly C data not reaching Python side (sometimes).
As for the wrapper, there isn't much to it:
cdef class io_uring_cqe:
    cdef __io_uring_cqe* ptr

it just a pointer and Liburing does all the assigning memory, data, removal.

Not sure why but io_uring_peek_cqe(ring, cqe) returns<PHONE_NUMBER> vs 0 on success and -EAGAIN on failure

Finally!!! after 2 whole months of dealing with segfault, solution was to use different Liburing functions?!
    while counter := ((io_uring_submit(ring) if io_uring_sq_ready(ring) else 0)+counter-cq_ready):
        cq_ready = 0
        while io_uring_peek_cqe(ring, cqe) == -EAGAIN:
            io_uring_wait_cqe_nr(ring, cqe, 1)  # wait for at least `1` event to be ready.

        for index in range(io_uring_for_each_cqe(ring, cqe)):
            # do stuff ...

Not sure why io_uring_peek_batch_cqe was glitching out like that. Sure needs an investigation.
Today was the last day I could work on fixing this bug, glad I burnt the midnight oil to get er done :)

Not sure why but io_uring_peek_cqe(ring, cqe) returns<PHONE_NUMBER> vs 0 on success and -EAGAIN on failure

4294967285 is -EAGAIN, it sounds like a casting issue on your side where it becomes unsigned.

Not sure what was happening with<PHONE_NUMBER> same code working fine now, maybe a glitch in the matrix.
Real issues is to figure out whats happening with io_uring_peek_batch_cqe, I must have compiled (gcc, clang) near to thousand times, many different codes, I had over 10GB+ core dump files ...

Looking at your earlier segfaults, are you sure it isn't some random memory issues you are seeing? Eg a bug in the other code, or something like that. They don't look related to liburing, but they do look like use-after-free issues, or double frees.

io_uring_for_each_cqe kinda works! But there are certain time where cqe doesn't get populated as well.

Things like that make me very suspicious, because while the peek batch API is probably not that widely used, io_uring_for_each_cqe() certainly is. And I find it very hard to believe there's a bug in that.

Looking at your earlier segfaults, are you sure it isn't some random memory issues you are seeing? Eg a bug in the other code, or something like that. They don't look related to liburing, but they do look like use-after-free issues, or double frees.

These could certainly be true, I could have goofed somewhere! For now same code io_uring_peek_cqe & io_uring_for_each_cqe works, and io_uring_peek_batch_cqe is a glitch.

Things like that make me very suspicious

This is actually my fault, I was calling io_uring_for_each_cqe without calling io_uring_peek_cqe first, thus the weirdness.

O, I forgot to mention.
In my code I have it so sqe->user_data can not be set to 0, if so it will raise error.
While using cq_ready = io_uring_peek_batch_cqe(...) many times I noticed cq_ready would return value like 10 later processing cqe[i].user_data only say 5 of those 10 would have valid user_data and rest would be 0.
Either cq_ready would give higher ready cqe count while in reality only few were ready, or all ready cqe weren't inserted into array?
Another problem was when I would io_uring_cq_advance(ring, cq_ready) this would segfault, maybe because cq_ready was trying to free more cqe than actually available?

I still think this is exceedingly likely an issue in your code or interface. Given that there are demonstrable signedness issues etc, I have to say I don't have a lot of faith in the stability of it.
In terms of completions, you can do:
# echo 1 > /sys/kernel/debug/tracing/events/io_uring/enable

and then view a trace with:
# cat /sys/kernel/debug/tracing/trace

which will tell you what completions came in.
io_uring_cq_advance() only adds an integer to the shared CQ ring head, there's no way that can segfault unless you overwrote some memory, for example.

Ok, trying with trace enabled.
For both same exact code, only thing changed are:
Segfault:
    # event manager
    while counter := ((io_uring_submit(ring) if io_uring_sq_ready(ring) else 0)+counter-cq_ready):
        # get count of how many event(s) are ready and fill `cqe`
        while not (cq_ready := io_uring_peek_batch_cqe(ring, cqe, counter)):
            io_uring_wait_cqe_nr(ring, cqe, 1)  # wait for at least `1` event to be ready.

        for index in range(cq_ready):
            res, user_data = cqe.get_index(index)
            if (ptr := <PyObject*><uintptr_t>user_data) is NULL:
                raise RuntimeError('`engine()` - received `NULL` from `user_data`')

# cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 34/34   #P:24
#
#                                _-----=> irqs-off/BH-disabled
#                               / _----=> need-resched
#                              | / _---=> hardirq/softirq
#                              || / _--=> preempt-depth
#                              ||| / _-=> migrate-disable
#                              |||| /     delay
#           TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
#              | |         |   |||||     |         |
         python3-8831    [001] .....  4383.112306: io_uring_create: ring 000000001a5a30d7, fd 3 sq size 1024, cq size 2048, flags 0x10000
         python3-8831    [001] .....  4383.112357: io_uring_submit_req: ring 000000001a5a30d7, req 00000000d8323653, user_data 0x72576c6665c0, opcode NOP, flags 0x10, sq_thread 0
         python3-8831    [001] .....  4383.112359: io_uring_queue_async_work: ring 000000001a5a30d7, request 00000000d8323653, user_data 0x72576c6665c0, opcode NOP, flags 0x40010, normal queue, work 00000000f99d6224
         python3-8831    [001] .....  4383.112405: io_uring_submit_req: ring 000000001a5a30d7, req<PHONE_NUMBER>34bb66, user_data 0x72576c666500, opcode NOP, flags 0x10, sq_thread 0
         python3-8831    [001] .....  4383.112406: io_uring_queue_async_work: ring 000000001a5a30d7, request<PHONE_NUMBER>34bb66, user_data 0x72576c666500, opcode NOP, flags 0x40010, normal queue, work<PHONE_NUMBER>5de4eb
         python3-8831    [001] .....  4383.112406: io_uring_submit_req: ring 000000001a5a30d7, req<PHONE_NUMBER>f87a3a, user_data 0x72576c5f0dc0, opcode NOP, flags 0x10, sq_thread 0
         python3-8831    [001] .....  4383.112407: io_uring_queue_async_work: ring 000000001a5a30d7, request<PHONE_NUMBER>f87a3a, user_data 0x72576c5f0dc0, opcode NOP, flags 0x40010, normal queue, work<PHONE_NUMBER>0c4c78
         python3-8831    [001] .....  4383.112407: io_uring_submit_req: ring 000000001a5a30d7, req 000000004e1b7af5, user_data 0x72576c5f0460, opcode NOP, flags 0x10, sq_thread 0
         python3-8831    [001] .....  4383.112407: io_uring_queue_async_work: ring 000000001a5a30d7, request 000000004e1b7af5, user_data 0x72576c5f0460, opcode NOP, flags 0x40010, normal queue, work<PHONE_NUMBER>d91cea
         python3-8831    [001] .....  4383.112407: io_uring_submit_req: ring 000000001a5a30d7, req 000000009ac3c392, user_data 0x72576c5f0fa0, opcode NOP, flags 0x10, sq_thread 0
         python3-8831    [001] .....  4383.112408: io_uring_queue_async_work: ring 000000001a5a30d7, request 000000009ac3c392, user_data 0x72576c5f0fa0, opcode NOP, flags 0x40010, normal queue, work<PHONE_NUMBER>8922c1
         python3-8831    [001] .....  4383.112408: io_uring_submit_req: ring 000000001a5a30d7, req 00000000cc5b0bf8, user_data 0x72576c5f1000, opcode NOP, flags 0x10, sq_thread 0
         python3-8831    [001] .....  4383.112408: io_uring_queue_async_work: ring 000000001a5a30d7, request 00000000cc5b0bf8, user_data 0x72576c5f1000, opcode NOP, flags 0x40010, normal queue, work 00000000e31f5469
         python3-8831    [001] .....  4383.112408: io_uring_submit_req: ring 000000001a5a30d7, req 00000000dbbc1472, user_data 0x72576c5f1060, opcode NOP, flags 0x10, sq_thread 0
         python3-8831    [001] .....  4383.112409: io_uring_queue_async_work: ring 000000001a5a30d7, request 00000000dbbc1472, user_data 0x72576c5f1060, opcode NOP, flags 0x40010, normal queue, work 000000006dcad57f
         python3-8831    [001] .....  4383.112409: io_uring_submit_req: ring 000000001a5a30d7, req 000000009b5895d6, user_data 0x72576c5f10c0, opcode NOP, flags 0x10, sq_thread 0
         python3-8831    [001] .....  4383.112409: io_uring_queue_async_work: ring 000000001a5a30d7, request 000000009b5895d6, user_data 0x72576c5f10c0, opcode NOP, flags 0x40010, normal queue, work<PHONE_NUMBER>a2d9f8
         python3-8831    [001] .....  4383.112411: io_uring_submit_req: ring 000000001a5a30d7, req<PHONE_NUMBER>f98703, user_data 0x72576c4a6500, opcode NOP, flags 0x10, sq_thread 0
         python3-8831    [001] .....  4383.112411: io_uring_queue_async_work: ring 000000001a5a30d7, request<PHONE_NUMBER>f98703, user_data 0x72576c4a6500, opcode NOP, flags 0x40010, normal queue, work<PHONE_NUMBER>13aa14
         python3-8831    [001] .....  4383.112411: io_uring_submit_req: ring 000000001a5a30d7, req 00000000e4e79765, user_data 0x72576c4a6560, opcode NOP, flags 0x10, sq_thread 0
         python3-8831    [001] .....  4383.112411: io_uring_queue_async_work: ring 000000001a5a30d7, request 00000000e4e79765, user_data 0x72576c4a6560, opcode NOP, flags 0x40010, normal queue, work 000000008b7003de
         python3-8831    [001] .....  4383.112415: io_uring_cqring_wait: ring 000000001a5a30d7, min_events 1
    iou-wrk-8831-8833    [009] ...1.  4383.112478: io_uring_complete: ring 000000001a5a30d7, req 00000000d8323653, user_data 0x72576c6665c0, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-8831-8833    [009] ...1.  4383.112485: io_uring_complete: ring 000000001a5a30d7, req<PHONE_NUMBER>34bb66, user_data 0x72576c666500, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-8831-8833    [009] ...1.  4383.112486: io_uring_complete: ring 000000001a5a30d7, req<PHONE_NUMBER>f87a3a, user_data 0x72576c5f0dc0, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-8831-8833    [009] ...1.  4383.112487: io_uring_complete: ring 000000001a5a30d7, req 000000004e1b7af5, user_data 0x72576c5f0460, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-8831-8833    [009] ...1.  4383.112488: io_uring_complete: ring 000000001a5a30d7, req 000000009ac3c392, user_data 0x72576c5f0fa0, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-8831-8833    [009] ...1.  4383.112489: io_uring_complete: ring 000000001a5a30d7, req 00000000cc5b0bf8, user_data 0x72576c5f1000, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-8831-8833    [009] ...1.  4383.112490: io_uring_complete: ring 000000001a5a30d7, req 00000000dbbc1472, user_data 0x72576c5f1060, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-8831-8833    [009] ...1.  4383.112492: io_uring_complete: ring 000000001a5a30d7, req 000000009b5895d6, user_data 0x72576c5f10c0, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-8831-8833    [009] ...1.  4383.112493: io_uring_complete: ring 000000001a5a30d7, req<PHONE_NUMBER>f98703, user_data 0x72576c4a6500, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-8831-8833    [009] ...1.  4383.112493: io_uring_complete: ring 000000001a5a30d7, req 00000000e4e79765, user_data 0x72576c4a6560, result 0, cflags 0x0 extra1 0 extra2 0
         python3-8831    [001] .....  4383.112493: io_uring_task_work_run: tctx 00000000b66e999c, count 6, loops 1
         python3-8831    [001] .....  4383.112495: io_uring_task_work_run: tctx 00000000b66e999c, count 4, loops 1

No problems:
    # event manager
    while counter := ((io_uring_submit(ring) if io_uring_sq_ready(ring) else 0)+counter-cq_ready):
        cq_ready = 0
        if io_uring_peek_cqe(ring, cqe) == -EAGAIN:
            io_uring_wait_cqe_nr(ring, cqe, 1)  # wait for at least `1` event to be ready.

        for index in range(io_uring_for_each_cqe(ring, cqe)):
            res, user_data = cqe.get_index(index)
            if (ptr := <PyObject*><uintptr_t>user_data) is NULL:
                raise RuntimeError('`engine()` - received `NULL` from `user_data`')
            cq_ready += 1

# cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 33/33   #P:24
#
#                                _-----=> irqs-off/BH-disabled
#                               / _----=> need-resched
#                              | / _---=> hardirq/softirq
#                              || / _--=> preempt-depth
#                              ||| / _-=> migrate-disable
#                              |||| /     delay
#           TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
#              | |         |   |||||     |         |
         python3-9116    [019] .....  4526.441262: io_uring_create: ring<PHONE_NUMBER>32cb5a, fd 3 sq size 1024, cq size 2048, flags 0x10000
         python3-9116    [019] .....  4526.441325: io_uring_submit_req: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>288e5f, user_data 0x71785ba665c0, opcode NOP, flags 0x10, sq_thread 0
         python3-9116    [019] .....  4526.441327: io_uring_queue_async_work: ring<PHONE_NUMBER>32cb5a, request<PHONE_NUMBER>288e5f, user_data 0x71785ba665c0, opcode NOP, flags 0x40010, normal queue, work 00000000b6665afe
         python3-9116    [019] .....  4526.441371: io_uring_submit_req: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>90a3eb, user_data 0x71785ba66500, opcode NOP, flags 0x10, sq_thread 0
         python3-9116    [019] .....  4526.441371: io_uring_queue_async_work: ring<PHONE_NUMBER>32cb5a, request<PHONE_NUMBER>90a3eb, user_data 0x71785ba66500, opcode NOP, flags 0x40010, normal queue, work 00000000ab7055aa
         python3-9116    [019] .....  4526.441372: io_uring_submit_req: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>6a43f0, user_data 0x71785b9f0dc0, opcode NOP, flags 0x10, sq_thread 0
         python3-9116    [019] .....  4526.441372: io_uring_queue_async_work: ring<PHONE_NUMBER>32cb5a, request<PHONE_NUMBER>6a43f0, user_data 0x71785b9f0dc0, opcode NOP, flags 0x40010, normal queue, work<PHONE_NUMBER>f0a9ea
         python3-9116    [019] .....  4526.441372: io_uring_submit_req: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>2a093c, user_data 0x71785b9f0460, opcode NOP, flags 0x10, sq_thread 0
         python3-9116    [019] .....  4526.441372: io_uring_queue_async_work: ring<PHONE_NUMBER>32cb5a, request<PHONE_NUMBER>2a093c, user_data 0x71785b9f0460, opcode NOP, flags 0x40010, normal queue, work 00000000d429ca3b
         python3-9116    [019] .....  4526.441373: io_uring_submit_req: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>97a369, user_data 0x71785b9f0fa0, opcode NOP, flags 0x10, sq_thread 0
         python3-9116    [019] .....  4526.441373: io_uring_queue_async_work: ring<PHONE_NUMBER>32cb5a, request<PHONE_NUMBER>97a369, user_data 0x71785b9f0fa0, opcode NOP, flags 0x40010, normal queue, work 00000000cdf133ed
         python3-9116    [019] .....  4526.441373: io_uring_submit_req: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>9f27ce, user_data 0x71785b9f1000, opcode NOP, flags 0x10, sq_thread 0
         python3-9116    [019] .....  4526.441373: io_uring_queue_async_work: ring<PHONE_NUMBER>32cb5a, request<PHONE_NUMBER>9f27ce, user_data 0x71785b9f1000, opcode NOP, flags 0x40010, normal queue, work 00000000ac14df27
         python3-9116    [019] .....  4526.441374: io_uring_submit_req: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>2bb775, user_data 0x71785b9f1060, opcode NOP, flags 0x10, sq_thread 0
         python3-9116    [019] .....  4526.441374: io_uring_queue_async_work: ring<PHONE_NUMBER>32cb5a, request<PHONE_NUMBER>2bb775, user_data 0x71785b9f1060, opcode NOP, flags 0x40010, normal queue, work<PHONE_NUMBER>ed3eeb
         python3-9116    [019] .....  4526.441374: io_uring_submit_req: ring<PHONE_NUMBER>32cb5a, req 00000000f59b29d9, user_data 0x71785b9f10c0, opcode NOP, flags 0x10, sq_thread 0
         python3-9116    [019] .....  4526.441374: io_uring_queue_async_work: ring<PHONE_NUMBER>32cb5a, request 00000000f59b29d9, user_data 0x71785b9f10c0, opcode NOP, flags 0x40010, normal queue, work 000000008d6e0cbe
         python3-9116    [019] .....  4526.441376: io_uring_submit_req: ring<PHONE_NUMBER>32cb5a, req 000000006b26aef7, user_data 0x71785b8a6500, opcode NOP, flags 0x10, sq_thread 0
         python3-9116    [019] .....  4526.441376: io_uring_queue_async_work: ring<PHONE_NUMBER>32cb5a, request 000000006b26aef7, user_data 0x71785b8a6500, opcode NOP, flags 0x40010, normal queue, work<PHONE_NUMBER>825a78
         python3-9116    [019] .....  4526.441376: io_uring_submit_req: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>26ecfe, user_data 0x71785b8a6560, opcode NOP, flags 0x10, sq_thread 0
         python3-9116    [019] .....  4526.441376: io_uring_queue_async_work: ring<PHONE_NUMBER>32cb5a, request<PHONE_NUMBER>26ecfe, user_data 0x71785b8a6560, opcode NOP, flags 0x40010, normal queue, work 00000000a2ab8519
         python3-9116    [019] .....  4526.441379: io_uring_cqring_wait: ring<PHONE_NUMBER>32cb5a, min_events 1
    iou-wrk-9116-9118    [012] ...1.  4526.441415: io_uring_complete: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>288e5f, user_data 0x71785ba665c0, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-9116-9118    [012] ...1.  4526.441422: io_uring_complete: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>90a3eb, user_data 0x71785ba66500, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-9116-9118    [012] ...1.  4526.441423: io_uring_complete: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>6a43f0, user_data 0x71785b9f0dc0, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-9116-9118    [012] ...1.  4526.441424: io_uring_complete: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>2a093c, user_data 0x71785b9f0460, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-9116-9118    [012] ...1.  4526.441425: io_uring_complete: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>97a369, user_data 0x71785b9f0fa0, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-9116-9118    [012] ...1.  4526.441426: io_uring_complete: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>9f27ce, user_data 0x71785b9f1000, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-9116-9118    [012] ...1.  4526.441426: io_uring_complete: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>2bb775, user_data 0x71785b9f1060, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-9116-9118    [012] ...1.  4526.441427: io_uring_complete: ring<PHONE_NUMBER>32cb5a, req 00000000f59b29d9, user_data 0x71785b9f10c0, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-9116-9118    [012] ...1.  4526.441428: io_uring_complete: ring<PHONE_NUMBER>32cb5a, req 000000006b26aef7, user_data 0x71785b8a6500, result 0, cflags 0x0 extra1 0 extra2 0
    iou-wrk-9116-9118    [012] ...1.  4526.441429: io_uring_complete: ring<PHONE_NUMBER>32cb5a, req<PHONE_NUMBER>26ecfe, user_data 0x71785b8a6560, result 0, cflags 0x0 extra1 0 extra2 0
         python3-9116    [019] .....  4526.441433: io_uring_task_work_run: tctx 000000009c113459, count 10, loops 1

@isilence Since you seem to know Python would you mind looking at this bug as well? I created a simple test here: https://github.com/YoSTEALTH/Liburing/blob/master/test/queue/get_cqe_test.py#L25-L42 ring and cqe is initialized here https://github.com/YoSTEALTH/Liburing/blob/master/test/conftest.py#L36-L50

@YoSTEALTH, everyone knows python, though I know little of FFI nuances, but the real question is whether it's really a liburing issue. And skimming through the discussion there is a higher chance it's not.
Can you do a little bit of debugging? The easiest thing you can do, it to compile a custom liburing and print all CQEs it gets in io_uring_peek_batch_cqe(). If python would also prints what it gets on its side, you can compare outputs and hopefully say what's wrong. Or just paste the output here.

It took me over a month of debugging to narrow down the cause of segfault to that function, thus I created that test.
Python's liburing is custom compiled (does not use os liburing).
There is no way to print after io_uring_peek_batch_cqe() is called and it goes into segfault.
Even one of Cython's core dev ran that test and couldn't figure out what was causing the segfault.
Its weird since the wrap itself is just 1 line of code https://github.com/YoSTEALTH/Liburing/blob/master/src/liburing/queue.pyx#L278-L284, so for it to be ffi related is really low as well! Also why does same code works fine for io_uring_for_each_cqe?

It took me over a month of debugging to narrow down the cause of segfault to that function, thus I created that test.
Python's liburing is custom compiled (does not use os liburing).
There is no way to print after io_uring_peek_batch_cqe() is called and it goes into segfault.

Where does it segfaults? In the io_uring_peek_batch_cqe call itself? Or after, when you're looking at CQEs you got from it?

Assume you are running https://github.com/YoSTEALTH/Liburing/blob/master/test/queue/get_cqe_test.py#L25-L42 test, if the loop = 2 or some small value it will pass, if the loop is higher value is when segfault happens. Its hard to tell, since any of python errors will raise exception. I would assume Cython might segfault as well if there is type difference or for example unsigned returning -1 or something like that.

Do you have a call trace from python? If the library / test framework hides it, can you run without it and get a stack trace?

Not exactly sure what you mean, I do have faulthandler enabled and output:
============================= test session starts ==============================
platform linux -- Python 3.13.0b1, pytest-8.2.2, pluggy-1.5.0 -- /opt/python/3.13/bin/python3
cachedir: .pytest_cache
rootdir: /.../Liburing/Python/Liburing
configfile: pyproject.toml
collecting ... collected 1 item

test/queue/get_cqe_test.py::test_peek_batch free(): invalid pointer
Fatal Python error: Aborted

Current thread 0x00007e4e7b26c740 (most recent call first):
  File "/.../Liburing/Python/Liburing/test/queue/get_cqe_test.py", line 36 in test_peek_batch
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/python.py", line 162 in pytest_pyfunc_call
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/python.py", line 1632 in runtest
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/runner.py", line 173 in pytest_runtest_call
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/runner.py", line 241 in <lambda>
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/runner.py", line 341 in from_call
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/runner.py", line 240 in call_and_report
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/runner.py", line 135 in runtestprotocol
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/runner.py", line 116 in pytest_runtest_protocol
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/main.py", line 364 in pytest_runtestloop
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/main.py", line 339 in _main
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/main.py", line 285 in wrap_session
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/main.py", line 332 in pytest_cmdline_main
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/opt/python/3.13/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/config/__init__.py", line 178 in main
  File "/opt/python/3.13/lib/python3.13/site-packages/_pytest/config/__init__.py", line 206 in console_main
  File "/opt/python/3.13/lib/python3.13/site-packages/pytest/__main__.py", line 7 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main

Extension modules: liburing.error, liburing.queue, liburing.time, liburing.socket, liburing.socket_extra, liburing._test, liburing.version, liburing.futex, liburing.probe, liburing.common, liburing.register, liburing.file, liburing.other, liburing.statx, liburing.poll, liburing.os, liburing.syscall, liburing.helper, liburing.xattr (total: 19)


Research running evosuite in parallel

Description
Research how intellij and evosuite work when multiple test generation jobs are triggered in parallel

EvoSuite processes run fine in parallel, the only limitation is the way we currently handle coverage visualization/test results. They are not really designed with multiple editors in mind


paint_y_axis() target_y index out of bounds

When running spiral_cube_cube.stl from this thingiverse download, I am getting the error IndexError: index 102 is out of bounds for axis 0 with size 100 when paint_y_axis() in perimeter.py is called.

Hi @SeymourGabe I just pushed a change which should fix your issue.
Please let me know if you find any other problems.


[Shell] Provide proper error message related to worker implementations

Description:

Consider following example

With the new parser changes we are providing an proper error messages related to module level worker implementations as follows.


@fylgja/controls - Unable to change border style for disabled radios

Right now there's no way to change the border style for a disabled radio input with @fylgja/controls.
An example would be the material-style radio button in combination with a dashed border-style, which looks a bit funky.
Cool, thanks, peace out.

Fixed in v2


Image plot PyTorch

I'm trying to use the image plot function on a PyTorch DeepExplainer but I'm not sure if it's working correctly. I'm using the MNIST code from https://github.com/pytorch/examples/tree/master/mnist.
Then, to make the image plot I do:
import shap
import numpy as np

# select a set of background examples to take an expectation over
background = next(iter(train_loader))[0][:100]
# explain predictions of the model on four images
e = shap.DeepExplainer(model, background)

shap_values = e.shap_values(next(iter(test_loader))[0][:5])

sv_transposed = [shap_value.transpose(0, 2, 3, 1) for shap_value in shap_values]

shap.image_plot(sv_transposed, -next(iter(train_loader))[0][:5].permute(0, 2, 3, 1).numpy())

The result:

This doesn't seem right, does it? If you compare it to the plots on the shap GitHub example.
Thanks for the help!

It does seem less intuitive than what I have seen the in past. Could be an odd model. To debug it is helpful to check and make sure the sum of the SHAP values equals the model output they are explaining (plus the expected_value base). Another thing to consider is if the 100 background samples are actually random (if they are all from the same class that would change things).


Add support for IMF Application 2E 4ED (2022)

DRAFT until the specification is published.
PCD available at https://github.com/SMPTE/st2067-21
Closes #310

@fschleich See refactored Application2E2021 image validation method


Integrate TINT into tobac

Finally getting around to starting the pull request for integrating TINT into tobac.

Thanks for the pull request, and apologies for getting round for this. Would it be possible to make a new pull request to the branch 'v2.0-dev'? Just to avoid merging directly into the main branch


Fix file arguments in nextflow_schema.json

Our file parameters are declared as string, but I think they should also have a "format": "file-path". They also lack mime types and icons. There are a icons for a few typical biological file types, I think, the rest can have far fa-file-code. In my phyloplace pipeline I've used declarations such as this:
                "hmmfile": {
                    "type": "string",
                    "format": "file-path",
                    "mimetype": "text/plain",
                    "description": "HMM file. If provided, will be used to align both the reference and query sequences.",
                    "fa_icon": "far fa-file-code"
                }

I'm not sure what the guidelines say...

I can check the guidelines about it


lakefsfs versions are not maintained in git

Currently, we have no way of knowing which ref a lakefsfs version is based on.
Suggest adding tags for future versions (and possible past)
Add release process documentation
Maintain a version changelist

This was closed by the clients release checklist by @lynnro314


Live

[Go Live][Jan 08]

Learn Build status updates of commit e93c336:
:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel


Fixed Hyperlink in BrayCurtis Distance

Updated conf.py.in and braycurtis.rst to reflect changes in the hyperlink of the BrayCurtis Distance in the reference  section of
https://www.shogun-toolbox.org/examples/latest/examples/distance/braycurtis.html

@vigsterkr Is there any further change required in this?

@karlnapf shall I close this then?


final changes before merge

Updated PR with changes to the other notebooks.

The other notebooks were already up to date in origin.


be able to create SQL editor "group" or "collection", and link to DB connection

Is your feature request related to a problem? Please describe.
Having different SQL editors that are associated with different DB connections available in editor pane can result in accidentally running a query in wrong editor with catastrophic results.
Describe the solution you'd like
It would be nice to be able to automatically or manually group two or more editors into a group and link that group to a DB connection. Then it would be nice to be able to "auto close/hide" all the editors in that group simultaneously either manually or automatically when the DB connection is closed.
Describe alternatives you've considered
Having different editor panes for different connections (ala MySQL workbench)

thanks for suggestion

Hello. In my work, I use several connections to different databases and different schemas, while a lot of folds with sql editors are open. It would be great to be able to filter tabs based on connection or have a combobox at the top as a filter so you can filter tabs with editors


GridInterface bin support for Histogram and QuadMesh

This PR adds support for binned coordinates to the GridInterface addressing https://github.com/ioam/holoviews/issues/547. This is one of the final conversions of Elements to a Dataset type.

[x] Get unit tests passing
[ ] Add __setstate__ for pickle compatibility
[ ] Move Histogram and QuadMesh tests into testbinneddataset.py or similar
[ ] Add examples of irregular meshes to reference documentation and gallery

Several nice to haves, which don't necessarily have to hold up merging this PR:

[x] Support for xarray Datasets
[x] Support for irregular meshes (i.e. 2D array coordinates)

I've now managed to also support irregular meshes, which are a commonly requested feature particularly for GeoViews. Support for datashading quadmeshes will come in a later PR (probably after TriMesh is merged). Here are some irregular meshes:

And all the usual machinery also works to explore high-dimensional datasets, e.g. here is an example of a multi-dimensional irregularly gridded dataset (requested in https://github.com/ioam/geoviews/issues/57 and https://github.com/ioam/geoviews/issues/73).
lon, lat = np.meshgrid(np.linspace(-20, 20, 5), np.linspace(0, 30, 4))
lon += lat/10
lat += lon/10

da = xr.DataArray(np.arange(40).reshape(2, 4, 5), dims=['z', 'y', 'x'],
                  coords = {'lat': (('y', 'x'), lat),
                            'lon': (('y', 'x'), lon),
                            'z': [0, 1]}, name='A')

ds = hv.Dataset(da, ['lat', 'lon', 'z'])
ds.to(hv.QuadMesh, groupby=['z'])

Here is an overview of the new functionality.

Nice!

@jbednar One question I have is how we can properly support datashading a QuadMesh, currently I see two options:

We can treat them like points which means the resulting output cannot have a density higher than the lowest density portion of the mesh because otherwise the output will have sampling artifacts.

We can treat them like rasters, which I think means that the areas will not be appropriately weighted and I haven't figured out how I'd go about generating appropriate coordinates for each pixel after regridding.

What about slicing every bin in half and rendering it as a TriMesh?

Neat idea!

Now implemented datashader rasterization of QuadMesh types, based on your suggestion. It's fairly fast once precompute is enabled.

@jbednar @jlstevens Requesting review.

Looks fine and happy to merge once you've addressed my questions above.

After discussing 'uneven' versus 'irregular', I think an irregular mesh example where the sample positions have a small (but obvious) random jitter would make the meaning of 'irregular' a lot clearer.

Tests have passed. Ready to merge?

Yes, let's merge, should get as much testing and exercising as possible before the next release.

Merged!