Support for loading the full hierarchy in a model in case it exists

This patch supports loading the complete hierarchy if present in a model. The major changes that have  taken place are in GVRf/Framework/src/org/gearvrf/GVRAssimpImporter.java and GVRf/Framework/jni/engine/importer/assimp_importer.cpp to allow loading of the entire hierarchy via recursion.
We've also added a sample app that loads the public domain "Crytek Sponza" model that is available from the Crytek website. The sample app is located in GVRf/Sample/gvrmodelloader.
Please let us know if this complies with your coding standards and requirements. Since this is our first contribution, please do pardon us for any inadvertent errors. We welcome constructive criticism. Once this one is checked in, we'll start creating pull requests for other features that we have ready.
Thanks
Soumyajit & Lokesh

Please ignore the commit comment that mentions frustum culling. That has been removed from this pull request.

Awesome!  Thanks for the pull request.  We're in the process of reviewing.  It's a lot of files, so it may take some time to review.  Thanks!

Thanks for the submission. I have a number of stylistic issues - we have tried pretty hard to maintain a consistent style in the code base, and this doesn't match in a number of ways - and a smaller number of semantic issues. I've called them out in in-line notes, but the most important are probably

I think we need to be loading scene objects, not scenes.
GVRContext.loadBitmap() is dangerous and expensive, and using it makes possibly invalid assumptions about where assets are located.

Beyond that: This is a huge submission. It took me several hours to review it; because GitHub doesn't seem to support the notion of 'patch sets' or have any way to show the changes between an initial push and a revised push, it will take me just as long to re-review it. I see that this Pull Request is actually composed of 18 discrete commits; I would much prefer more granular  Pull Requests, one per commit, that will make it much easier to track changes.

With regard to code formatting/styling, I thought we had something up on the wiki before the release, but i can't seem to find it now.  Let's get with Patrick on Monday and make sure we put something up there including the eclipse file to import.

We've made the following changes/corrections based on the comments received

All the tabs has been converted to spaces and the codes written has followed the convention for GVRF.

The naming covention followed at present is the snake_case type. I have also removed the abbreviations with more descriptive name.

Now ShaderType.UNLIT_SHADER is used instead of static_castMaterial::ShaderType(0). At the begining I did this without going for the first option without any preference.

All magic numbers were an oversight. They have been removed/replaced.

The FindClass has been replaces by using the gvr_context class directly though its class. The repeated call problem has also been fixed.

Issues with GL_CLAMP_TO_EDGE/GL_REPEAT in gl_texture.h have been modified to take into account the actual texture coordinates in the model. The code now switches to GL_REPEAT when required.

We've removed the reference to loadBitmap with a different customized function that uses GVRAndroidResource

Other minor issues pointed out have been fixed.

Thanks! I'm out of town on a family emergency, so I don't think we will thoroughly review & merge this until next week, but Chao, Jason, or Tom may have time to do a first pass before I get back

Just leave some comments there. I just tried the model loader app. I am not quite sure if GVRF has difficulty handling complicated model or there is any other issue. The side effect is not that significant without goggle. But if you wear a goggle and test with the app. You can see there is something like lagging or frames jumping. It looks really weird to me.

Hi Chao,
I was looking into the performance issue in the model loading app. I could see some frame skips in the surfaceflinger and at those times there were increased load on the GPU. Are you aware of an profiling tool which we can use for profiling the gvrf apps? This is the screenshot we got from Systrace.

Also we will try loading various 3d models and check the performance. We will update you with the results.

Good old Traceview does a decent job ...

I tried to take the gl trace and found out that the glCompileShader(...) is being called more than once. This might be causing the lag and frame drop.

It appears that glCompileShader is being called at the mesh level for each mesh multiple times. This regular shader recompilation might end up slowing down the GPU as it takes a significant amount of time. We are still investigating why this occurs so frequently. It also seems to be happening for each mesh in the hierarchy. Ideally this should only happen once per geometry that has materials assigned at model load time.
-Soumyajit

glCompileShader is being called at the mesh level for each mesh multiple times.

This is strange, as GVRF seems to do a pretty good job of managing compiled shaders. I'll refrain from re-reviewing until you find the issue and re-submit.

We've removed the reference to loadBitmap with a different customized function that uses GVRAndroidResource

Good. Imho, you should have an overload that takes a texture loading callback. (The current method(s) could just use a default callback that maps names to asset-relative files.) This would let the app handle any mapping from "texture_name" to R.drawable.texture_name and would let the app maintain its own cache, so that (say) a set of models that all use the "bricks_and_mortar" texture all use the same GVRTexture.

We now have two functions - loadScene() and loadModelAsSceneObject() for loading the 3d model as a scene or a scene object.

I'm all for nice long, descriptive names, but a method that returns a GVRSceneObject probably doesn't need to be called loadModelAsSceneObject() - GVRSceneObject loadModel(GVRAndroidResource); would seem to be nice and clear.

loadScene() and loadModelAsSceneObject()

Sorry, should have mentioned this sooner: What's your async strategy?
Loading a simple mesh is already pretty slow; recursively loading and loading textures is bound to be even slower. Seems to me like you should (at least) be using the new GVRContext. loadFutureTexture( GVRAndroidResource ) API; it would probably make sense to support both synchronous and asynchronous APIs, the way we do elsewhere:
GVRScene loadScene(...);
Future<GVRScene> loadFutureScene(...);
GVRSceneObject loadModel(...);
Future<GVRSceneObject> loadFutureModel();

Thanks @PatrickTW for your valuable suggestions . I will be more careful in the future, about the comments. :-)

Lokesh,
I was asked to review comments for code submittals.
Yours are in the A++ category.  The things I mentioned are well within the range of normal communication.
Adhering to those guidelines makes good writing even better.
Patrick
From: Lokesh Chandra Basu<EMAIL_ADDRESS>Sent: Wednesday, April 29, 2015 1:49 AM
To: Samsung/GearVRf
Cc: Patrick Townsend-Wells
Subject: Re: [GearVRf] Support for loading the full hierarchy in a model in case it exists (#18)
Thanks @PatrickTWhttps://github.com/PatrickTW for your valuable suggestions . I will be more careful in the future, about the comments. :-)
—
Reply to this email directly or view it on GitHubhttps://github.com/Samsung/GearVRf/pull/18#issuecomment-97355121.

@soumyajitdeb, are you still working on the re-compilation bug? Is there anything we can do to help, here in Mountain View?

Many thanks for the offer of help. Yes we are still looking at this. We found that some of the shader recompilations are happening within oculus sdk itself.
SRI-B was closed today because of a public holiday. We shall check and get back to you on Monday regarding a resolution and help if required. Many apologies for the delay.
Thanks
Soumyajit

We finally managed to fix the issue of stuttering. It turns out shader compilation was not the issue. The stuttering was strangely due to lack of mipmapping and too many large textures being pulled over the memory bus. We've enabled mipmaps in GVRF via the glGenerateMipmap call and also modified the corresponding glTexParameter calls to do trilinear filtering. Now the demo runs absolutely smoothly.
-Soumyajit

The name of the method has been changed from loadModelAsSceneObject(…) to loadModel(…).
The new method to load the bitmap handles both the resource filename as well as the filename from the asset folder.
The rendering delay has been fixed by the use of mipmap in the base texture.
The loadModel(...) and loadScene(...) loads all the resources and textures asynchronously since internally textures & resources are loaded via the GVRAsynchronousResourceLoader.

Soumyajit & Lokesh

Sounds good! @chenchao1407, would you please re-test and re-read the code? If it looks good to you, I'll make the time for a line-by-line re-review.

Hi,
We have created a separate group for changes made by SRI-B. So once this pull request is confirmed to be working, we can then make all these changes through just one commit from the https://github.com/SRIB-GRAPHICS/GearVRf. It will keep things simple and well organized. For all future feature addition we will be using https://github.com/SRIB-GRAPHICS/ account.

Hi,
Actually  I don't think you have to re-org 24 commits into 1. In my opinion it may be cleaner but it may make us lose the capability to trace back each change.
I agree that you can re-org the commits using something like git rebase -i and then combine or we can say eliminate some commit like testing push, testing over. But I think it will be better to keep the valid commits information which may help us debug in the future.

Hi
Reviewed the three new commits and they look good to me. Left a comment on the default texture. Tried and everything looks much better now.
Thanks

Thanks, @chenchao1407 - I'll do a line-by-line first thing tomorrow.

@JonShemitz
Hi Jon, maybe it's your turn to review the codes now. Just FYI, you can just click into the three new commits to see what they changed recently to avoid reading the large amount of codes again. :-)

I'll go through the new commits and add some line notes (this afternoon, after I run some errands) but I thought it would be better to add a global response, first.

Loading bitmaps is slow; loading models is even slower. Doing either on the GL thread is a recipe for long startup times or animation stutters ... or both. We've added high and low level async methods for loading meshes and textures; they improve startup times by parallelizing loads, and they avoid stutter by doing expensive operations on background threads. Your new methods are even more expensive than the synchronous methods we have warned can take a very long time ... yet you have no async versions of the model loading methods.
It's quite possible that an app will want to load several models that share textures. (For example, a robot army might all share the same carbon-fiber, stainless-steel and glowing-eye textures.) This code would use a different texture in each model (incurring load costs for each).

These are big issues, and I've mentioned them before. Either one would be enough to for me to block this; I would like to see both addressed.
There's also a new issue with this code. As per issue https://github.com/Samsung/GearVRf/issues/24, we keep finding problems with the GVRF reference counting mechanism; Tom has asked me to go ahead and replace it with a more conventional finalize() based mechanism, where the Java objects own the native objects and the native objects are only deleted when the Java objects are garbage collected. This will eliminate a whole class of bugs that happen in 'real' apps, where objects may be removed from the scene graph and later added back; it will also eliminate the need for 'aliasing' tricks (see GVRShaders.setMainTexture()) or the GVRHybridObject.setKeepWrapper() hack.
This affects your code, because it's a huge wodge of JNI code that basically returns a single GVRScene object: there are no Java objects corresponding to the meshes or textures or even scene objects. (Yes, I do see that the loadModel() method creates wrappers for the scene objects.) As far as I can see, the only way your model loading work and my memory management work are going to be compatible is if your scene-recursion code is moved out of the JNI and into the Java, creating Java classes that own each GVRF-internal object in the 'normal' way.
Someone's going to have to do this port. If your scene loading code were fine otherwise, I'd see this as just another piece of my huge task; given the other issues with your scene loading code, I'd sure like to ask you guys to do the port.

Thank you for the comments! Regarding 1&2, we agree that the ideal scenario would be to have the entire code block moved to Java so that the parsing of the hierarchy creates java objects for each node. This would allow the new memory management system to function correctly. Unfortunately this means a complete rewrite of the entire code.
This pull request was originally meant for the version of GVRF which was present at the repo at the time of authoring and as such was never designed to follow an asynchronous loading strategy. Now that the framework has evolved and is more efficient, we'd definitely want to support the changes. However, the primary problem we are facing is that we'd need to be able to access the references of the assimp objects on the java side to be able to parse the complete hierarchy. Currently no such functionality exists within the java code of GVRf. This involves either writing JNI wrappers for all required critical functionality (rather time consuming) or use jassimp (https://github.com/assimp/assimp/tree/master/port/jassimp) for making the functionality visible at the java layer.
We wish to know if it is okay to include jassimp with our codebase - this will be a much faster process and since jassimp is an open source project, this layer would get updated alongwith any changes to assimp itself. If we can't use jassimp, we'd end up spending a lot of time writing JNI glue code.
Thanks
Soumyajit

Jassimp sounds interesting and seems like the way to go.  Is it up to date with Assimp 3.1?  Brazil (SIDIA) needs fbx support, which is included in 3.1.  Also, they were having trouble loading some fbx files.  Is that something you could diagnose?  If so, I’ll privately send you the file.
Btw, when can we expect a pull-request for frustum culling?  I expect that can be done separate from the Jassimp work.
-tom
From: soumyajitdeb<EMAIL_ADDRESS>Sent: Tuesday, May 12, 2015 1:37 AM
To: Samsung/GearVRf
Cc: Tom Flynn
Subject: Re: [GearVRf] Support for loading the full hierarchy in a model in case it exists (#18)
Thank you for the comments! Regarding 1&2, we agree that the ideal scenario would be to have the entire code block moved to Java so that the parsing of the hierarchy creates java objects for each node. This would allow the new memory management system to function correctly. Unfortunately this means a complete rewrite of the entire code.
This pull request was originally meant for the version of GVRF which was present at the repo at the time of authoring and as such was never designed to follow an asynchronous loading strategy. Now that the framework has evolved and is more efficient, we'd definitely want to support the changes. However, the primary problem we are facing is that we'd need to be able to access the references of the assimp objects on the java side to be able to parse the complete hierarchy. Currently no such functionality exists within the java code of GVRf. This involves either writing JNI wrappers for all required critical functionality (rather time consuming) or use jassimp (https://github.com/assimp/assimp/tree/master/port/jassimp) for making the functionality visible at the java layer.
We wish to know if it is okay to include jassimp with our codebase - this will be a much faster process and since jassimp is an open source project, this layer would get updated alongwith any changes to assimp itself. If we can't use jassimp, we'd end up spending a lot of time writing JNI glue code.
Thanks
Soumyajit
—
Reply to this email directly or view it on GitHubhttps://github.com/Samsung/GearVRf/pull/18#issuecomment-101188906.

We'll go ahead and use jassimp in that case for model loading. The frustum culling update has been sent as a pull request by Lokesh. I shall be done with the more efficient frustum+occlusion culling system by tomorrow.
As for the fbx issue, please pass us the file. We'll see what we can do at our end. I've handled fbx files before but because of their proprietory nature, they can be a bit painful to load at times.

so, how is it going?

Hi,
To make the full use of async implementation in GVRF, we started with jassimp to get all the assimp data in the Java side and handle it with existing Asynchronous implementation. The initial integration is already done(jassimp-integration branch  of  https://github.com/SRIB-GRAPHICS/GearVRf). The functionality is working fine for models with less number of materials but crashes for larger models with the error [ Failed adding to JNI local ref table (has 512 entries) ]. We have tried all the solutions available to fix this issue without any luck. Any help regarding this issue will speed things up as I personally do not have any solution as of now. I'm attaching the logcat file for the test when sponza.dae model was used. The integration is working fine with the sample application [gvrjassimptest].
Thanks.
.
.
.
E/gvrf    (11560):    converting property $clr.diffuse ...
E/gvrf    (11560):    converting property $clr.specular ...
E/gvrf    (11560):    converting property $clr.emissive ...
E/gvrf    (11560):    converting property $clr.transparent ...
E/gvrf    (11560):    converting property $clr.reflective ...
E/gvrf    (11560):    converting property $mat.shininess ...
E/gvrf    (11560):    converting property $mat.reflectivity ...
E/gvrf    (11560):    converting property $mat.refracti ...
E/dalvikvm(11560): JNI ERROR (app bug): local reference table overflow (max=512)
W/dalvikvm(11560): JNI local reference table (0x7e93e1e8) dump:
W/dalvikvm(11560):   Last 10 entries (of 512):
W/dalvikvm(11560):       511: 0x418babb8 java.lang.Class<java.lang.Float>
W/dalvikvm(11560):       510: 0x4230c2c0 java.lang.String "$mat.refracti"
W/dalvikvm(11560):       509: 0x4230c278 org.util.jassimp.AiMaterial$Property
W/dalvikvm(11560):       508: 0x4230c268 java.lang.Float
W/dalvikvm(11560):       507: 0x422cc820 java.util.ArrayList
W/dalvikvm(11560):       506: 0x4230c210 java.lang.String "$mat.reflectivit... (17 chars)
W/dalvikvm(11560):       505: 0x4230c1c8 org.util.jassimp.AiMaterial$Property
W/dalvikvm(11560):       504: 0x423ec8a8 java.lang.Float
W/dalvikvm(11560):       503: 0x422cc820 java.util.ArrayList
W/dalvikvm(11560):       502: 0x423ec858 java.lang.String "$mat.shininess"
W/dalvikvm(11560):   Summary:
W/dalvikvm(11560):        20 of java.lang.Class (3 unique instances)
W/dalvikvm(11560):       110 of java.lang.String (110 unique instances)
W/dalvikvm(11560):       156 of java.nio.DirectByteBuffer (156 unique instances)
W/dalvikvm(11560):        14 of java.lang.Float (14 unique instances)
W/dalvikvm(11560):        21 of java.lang.Integer (21 unique instances)
W/dalvikvm(11560):        67 of java.util.ArrayList (4 unique instances)
W/dalvikvm(11560):        38 of java.nio.ByteBuffer[] (8 elements) (38 unique instances)
W/dalvikvm(11560):        18 of org.util.jassimp.AiColor (18 unique instances)
W/dalvikvm(11560):        64 of org.util.jassimp.AiMaterial$Property (64 unique instances)
W/dalvikvm(11560):         1 of org.util.jassimp.AiScene
W/dalvikvm(11560):         3 of org.util.jassimp.AiMaterial (3 unique instances)
E/dalvikvm(11560): Failed adding to JNI local ref table (has 512 entries)
.
.
.

  http://stackoverflow.com/questions/17033438/android-expand-jni-local-reference-table may help with your JNI issue.

  I don’t think AiScene getAssimpScene(GVRAndroidResource androidResource) is the right prototype for your new GVRContext method:

a.       It should return a GVRSceneObject, not an AiScene.
b.      Imho, there still needs to be a resource loading callback. I’m adding caching (so that if you load R.drawable.foo and it’s still in memory, it won’t get reloaded) but there still needs to be a way for apps to intercept an attempt to load (say) the “bricksAndMortar” texture and map that to R.drawable.bricks_and_mortar or to “assets/models/textures/bricksAndMortar.png”.
From: Lokesh Chandra Basu<EMAIL_ADDRESS>Sent: Sunday, May 31, 2015 9:32 PM
To: Samsung/GearVRf
Cc: Jon Shemitz
Subject: Re: [GearVRf] Support for loading the full hierarchy in a model in case it exists (#18)
Hi,
To make the full use of async implementation in GVRF, we started with jassimp to get all the assimp data in the Java side and handle it with existing Asynchronous implementation. The initial integration is already done(jassimp-integration branch of https://github.com/SRIB-GRAPHICS/GearVRf). The functionality is working fine for models with less number of materials but crashes for larger models with the error [ Failed adding to JNI local ref table (has 512 entries) ]. We have tried all the solutions available to fix this issue without any luck. Any help regarding this issue will speed things up as I personally do not have any solution as of now. I'm attaching the logcat file for the test when sponza.dae model was used. The integration is working fine with the sample application [gvrjassimptest].
Thanks.
.
.
.
E/gvrf (11560): converting property $clr.diffuse ...
E/gvrf (11560): converting property $clr.specular ...
E/gvrf (11560): converting property $clr.emissive ...
E/gvrf (11560): converting property $clr.transparent ...
E/gvrf (11560): converting property $clr.reflective ...
E/gvrf (11560): converting property $mat.shininess ...
E/gvrf (11560): converting property $mat.reflectivity ...
E/gvrf (11560): converting property $mat.refracti ...
E/dalvikvm(11560): JNI ERROR (app bug): local reference table overflow (max=512)
W/dalvikvm(11560): JNI local reference table (0x7e93e1e8) dump:
W/dalvikvm(11560): Last 10 entries (of 512):
W/dalvikvm(11560): 511: 0x418babb8 java.lang.Class
W/dalvikvm(11560): 510: 0x4230c2c0 java.lang.String "$mat.refracti"
W/dalvikvm(11560): 509: 0x4230c278 org.util.jassimp.AiMaterial$Property
W/dalvikvm(11560): 508: 0x4230c268 java.lang.Float
W/dalvikvm(11560): 507: 0x422cc820 java.util.ArrayList
W/dalvikvm(11560): 506: 0x4230c210 java.lang.String "$mat.reflectivit... (17 chars)
W/dalvikvm(11560): 505: 0x4230c1c8 org.util.jassimp.AiMaterial$Property
W/dalvikvm(11560): 504: 0x423ec8a8 java.lang.Float
W/dalvikvm(11560): 503: 0x422cc820 java.util.ArrayList
W/dalvikvm(11560): 502: 0x423ec858 java.lang.String "$mat.shininess"
W/dalvikvm(11560): Summary:
W/dalvikvm(11560): 20 of java.lang.Class (3 unique instances)
W/dalvikvm(11560): 110 of java.lang.String (110 unique instances)
W/dalvikvm(11560): 156 of java.nio.DirectByteBuffer (156 unique instances)
W/dalvikvm(11560): 14 of java.lang.Float (14 unique instances)
W/dalvikvm(11560): 21 of java.lang.Integer (21 unique instances)
W/dalvikvm(11560): 67 of java.util.ArrayList (4 unique instances)
W/dalvikvm(11560): 38 of java.nio.ByteBuffer (38 unique instances)
W/dalvikvm(11560): 18 of org.util.jassimp.AiColor (18 unique instances)
W/dalvikvm(11560): 64 of org.util.jassimp.AiMaterial$Property (64 unique instances)
W/dalvikvm(11560): 1 of org.util.jassimp.AiScene
W/dalvikvm(11560): 3 of org.util.jassimp.AiMaterial (3 unique instances)
E/dalvikvm(11560): Failed adding to JNI local ref table (has 512 entries)
.
.
.
—
Reply to this email directly or view it on GitHubhttps://github.com/Samsung/GearVRf/pull/18#issuecomment-107302427.

Hi,

  http://stackoverflow.com/questions/17033438/android-expand-jni-local-reference-table may help with your JNI issue.

  >> I already tried those hacks and it still fails.

  I don’t think AiScene getAssimpScene(GVRAndroidResource androidResource) is the right prototype for your new GVRContext method:

a.       It should return a GVRSceneObject, not an AiScene.
b.      Imho, there still needs to be a resource loading callback. I’m adding caching (so that if you load R.drawable.foo and it’s still in memory, it won’t get reloaded) but there still needs to be a way for apps to intercept an attempt to load (say) the “bricksAndMortar” texture and map that to R.drawable.bricks_and_mortar or to “assets/models/textures/bricksAndMortar.png”.
>>  About point 2, that is already in pipeline once we get around this JNI issue. This was just a initial commit to at least get things working and later we can modify it according to our need as per the requirements which have been mentioned by you.

Perhaps, then, you should try to do less in the JNI code.
From: Lokesh Chandra Basu<EMAIL_ADDRESS>Sent: Wednesday, June 03, 2015 8:42 PM
To: Samsung/GearVRf
Cc: Jon Shemitz
Subject: Re: [GearVRf] Support for loading the full hierarchy in a model in case it exists (#18)
Hi,

http://stackoverflow.com/questions/17033438/android-expand-jni-local-reference-table may help with your JNI issue.

I already tried those hacks and it still fails.

I don’t think AiScene getAssimpScene(GVRAndroidResource androidResource) is the right prototype for your new GVRContext method:

a. It should return a GVRSceneObject, not an AiScene.
b. Imho, there still needs to be a resource loading callback. I’m adding caching (so that if you load R.drawable.foo and it’s still in memory, it won’t get reloaded) but there still needs to be a way for apps to intercept an attempt to load (say) the “bricksAndMortar” texture and map that to R.drawable.bricks_and_mortar or to “assets/models/textures/bricksAndMortar.png”.

About point 2, that is already in pipeline once we get around this JNI issue. This was just a initial commit to at least get things working and later we can modify it according to our need as per the requirements which have been mentioned by you.

—
Reply to this email directly or view it on GitHubhttps://github.com/Samsung/GearVRf/pull/18#issuecomment-108707392.

So, given the jassimp pull-request, this one could be closed, correct?

Closing this pull request. Please refer to [https://github.com/Samsung/GearVRf/pull/138] for the same functionality.

Closing this request as there is another one for jassimp loader [#138]


Fix not respecting target of mouseup events.

Fixes #3466, which was also reported in #3790.
The issue was not looking at the event target on our window listeners for mouseup/pointerup. This change looks at that and emits mouseupoutside/pointerupoutside as appropriate.
For the MockPointer changes, it turns out that target is not a parameter for Events, and is a read only property. It is configurable, however, which is why I am setting the target in a roundabout way.

Looks good. Is there a fiddle that you can demo a before and after?

Before: https://jsfiddle.net/sbxoya8x/
After: https://jsfiddle.net/e7ujs4a0/

These both seem to be using 4.4.4

Ugh, unsaved edits. Link should be fixed now.


chore(package.json): allow repository link for npm

That's all a mystery to me.  MERGED!

@gdi2290
do you need me to publish a new version?

it doesn't affect the package only discovery so no worries. I also added this to bower for you as angular-360-no-scope
https://www.npmjs.com/package/angular-360-no-scope


100 geospatial view nodes do not support datetime column types time date timestamp zoned timestamp

@wybert please review the code and merge if ok with you.
Thanks
Tobias

The code overall looks good to me. Just one concern, maybe it's better to show all the fields by default as Geopandas dose


feat: make tables resizeable

Test stand: https://nda.ya.ru/t/xCSdBmdn75ueXj
All tables are resizeable. Resize values are stored to localStorage. Exception - tables without predefined set of columns (query result, preview, hotkeys), their resize values are stored only while component is mounted
Closes #807

What do you think about making a bigger minWidth for columns, which contain tablets? For now I may shrink this columns for some reason, but it will be quite a tricky to enlarge it to previous size.

Is it possible to add action for double click on resize panel? It will be handy if double click restores initial column width.

Why didn't you make resizable columns with Vdisks and Pdisks in Storage page?

Is it possible to add action for double click on resize panel? It will be handy if double click restores initial column width.

It's possible, but it requires changes to react-data-table. Currently, we replace initial value with saved value, so initial value isn't accessible.
I made an issue for this: https://github.com/ydb-platform/ydb-embedded-ui/issues/824

Why didn't you make resizable columns with Vdisks and Pdisks in Storage page?

They have their own styles with overflow: visible. Reasons:

Disks could have donors, donors displayed in stack, when expanded it's height bigger, than the row's height
There may be a lot of disks, up to 25 VDisks and 144 PDisks, they expand column width

We need some rework, if we want to change it. For now I decided to left it as it was. Also It's the last column, so its width isn't a big issue since it doesn't affect other columns and always have the width of the content


Add first_value and last_value aggregations and allow winsorization.

We allow last_value aggregations and will soon allow first_value. Adjust the validation scripts here to allow those agg types.

Thank you! I think you need to change the schema here https://github.com/Eppo-exp/eppo-metrics-sync/blob/main/eppo_metrics_sync/schema/eppo_metric_schema.json#L159

Thanks @cedomir!


Support skewness(x) in Aggregation function

Which issue does this PR close?

Part of #12254
Closes #12249
Implemented by referring to https://github.com/duckdb/duckdb/blob/main/src/core_functions/aggregate/distributive/skew.cpp
Rationale for this change

What changes are included in this PR?
Added skewness UDAF.
Ref: https://duckdb.org/docs/sql/functions/aggregates#skewnessx
Are these changes tested?

Are there any user-facing changes?

Planning to add more logic tests.

Planning to add more logic tests.

I think we can port tests from duckdb https://github.com/duckdb/duckdb/blob/main/test/sql/aggregate/aggregates/test_skewness.test

Basically I'm feeling we have too much of user .md files, duplicating the functions description and poorly connected to each other which confuses a lot.
@alamb WDYT about having a single file for all SQL functions/scalars/expressions like in https://spark.apache.org/docs/latest/api/sql/index.html with description, examples
Btw Spark holds an internal meta repository that allows such doc to be regenerated from the code, which can be next step

Btw Spark holds an internal meta repository that allows such doc to be regenerated from the code, which can be next step

I agree
What I actually think would be ideal is exactly as @comphead describes and put all the documentation about the functions in the code, and then have a post processing step that auto generates the documentation
This is how the configuration setting documentation is created. Here is the script that does it:
https://github.com/apache/datafusion/blob/main/dev/update_config_docs.sh
BTW I think this is the direction @findepi  is heading with https://github.com/apache/datafusion/pull/12266. I also think GlareDB did something similar, but I can't find the link now.

I also wonder if we should consider where to draw the line on features to include (i.e. should we include skewness by default in the core?) -- especially as all aggregate functions use the same API, this is the kind of feature we could very easily keep outside -- either in its own crate or even own repo -- and then have other code integrate it in -- e.g. #11979

Closing this PR. Will move this to https://github.com/datafusion-contrib/datafusion-functions-extra as suggested


How should we handle project planning

We should come up with a way to plan the project tasks.
What tools should we use?
Individual tracking and team tracking will be done separately, we should decide on how we will uniformly report these at the end of semester.

Project management decided as follows: https://github.com/AyranIsTheNewRaki/Herodot/wiki/Project-and-Development-Management
Group communications will be here: https://github.com/AyranIsTheNewRaki/Herodot/wiki/Group-Communication
Contributing guidelines here: https://github.com/AyranIsTheNewRaki/Herodot/wiki/Contributing-Guidelines
These are all we decided, so I am closing this issue


Golden-section search

This is my first version of Golden-section search. I tested in our enterprise project and it's as good as my hardcoded version, but a little slower.
I know that there's no test, example, bench, nothing. I just want you to tell me what I did wrong. My own critic is that there's no way to choose the precision. If I understand correctly, everything is forced to f64 with O: ArgminOp<Param = f64, Output = f64>, so the user can't actually choose with
type Param = f32/f64;
type Output = f32/f64;

Our program runs GSS two times, they take around 26ms and 23ms. When I use this version, it's around 31ms and 29ms. Nothing terrible, but maybe just annoying that the same algo is suddenly 20% slower. I didn't activate any special features, no logging, no observer, no key-value, nothing. I don't understand what's taking 5-6ms more under the hood.

I appreciate any feedback regarding the ergonomics when implementing a solver

I had difficulty implementing this simple algo. Mostly because of the types. At first I thought there would be a single type F for precision and that's it. Then I saw O::Param and O::Output, and almost only f64 looking at your own code. At that time, I was really confused. And I saw no way to cast from one type to the other so I was even more confused. My other concerns were discussed in the linked issue.
As a user (and maybe contributor) of this lib, what matter is the speed, so disabling the stuff I don't need and choosing the float precision. I port a lot of Python and C++ code to Rust and it's sometime optimization algorithm, so we may have other methods to contribute, or maybe try to optimize those you already coded. For example, there's a lasso algorithm, ported from SPAMS. We don't want it in our codebase, but it probably doesn't fit here? I must warn you though, I'm not a mathematician at all. I sometime don't even understand what I'm porting, but the results are identical or really close and it's often faster. That's all we can hope for :)

Our program runs GSS two times, they take around 26ms and 23ms. When I use this version, it's around 31ms and 29ms. Nothing terrible, but maybe just annoying that the same algo is suddenly 20% slower. I didn't activate any special features, no logging, no observer, no key-value, nothing. I don't understand what's taking 5-6ms more under the hood.

There is quite some stuff going on under the hood in the executor, I suppose that might be the reason for the 5-6ms. If it stays at 5-6ms for problems of any complexity and for any solver, than I guess that's fine, but we've seen in #10 that it may be worse. Performance is certainly an issue which needs to be addressed.

I had difficulty implementing this simple algo. Mostly because of the types.

Being generic over all kinds of types is difficult and I'm aware that ergonomics suffers from it (particularly for a developer of solvers). I could have just supported ndarray, but in my own research, I often had to deal with "special" parameter vectors which required certain operations on them to be implemented manually. I didn't want parameter vectors to be limited to vectors of floats.
But that's independent of the problem with f32 and f64 where I totally agree that it should be generic. I will address this in more detail in the comment above.

As a user (and maybe contributor) of this lib, what matter is the speed, so disabling the stuff I don't need and choosing the float precision.

I think we need to start to excessively benchmark the code and identify the time consuming parts. I'm sure there are lots of places where things can be improved, and in cases where it is not possible, we may be able to put those things behind feature gates.

I port a lot of Python and C++ code to Rust and it's sometime optimization algorithm, so we may have other methods to contribute, or maybe try to optimize those you already coded. For example, there's a lasso algorithm, ported from SPAMS. We don't want it in our codebase, but it probably doesn't fit here?

I don't see why it shouldn't fit here, unless there are licensing issues. I really appreciate every contribution, particularly because it creates feedback like this which I desperately need to improve the library.

I must warn you though, I'm not a mathematician at all. I sometime don't even understand what I'm porting, but the results are identical or really close and it's often faster. That's all we can hope for :)

I'm not a mathematician either and this is also how I work, so there is no need to worry ;)

With PR #50 it should be possible to choose the float precision. Ergonomics definitely suffers from it (for developers of solvers), because one can't just use 1.0, but instead has to write F::from_f64(1.0).unwrap(). However, I don't think this has a performance impact, because the compiler should optimize it all away.

PR #50 is merged and you could be the first to give it a try ;)

I gave it a try. Imo, this is a more sensible design for simple methods like Brent and GSS. I like it.

Thanks again for your contribution and the feedback! :)


fix: makes configuration at environment group level for multiple apps…

… installed to the same environment possible. Does so, by using the directory structure in the "prototypes" folder to look for helm and ytt configuration at env group level.
closes #109

@Zebradil I have had another stab at this feature. I have now added the extension of prototypes at environment group level with a folder called _proto. This can be used optionally on top of the existing extension with _app whereby _app wins if both are being used.
The new integration test setup is great. Thanks @kbudde .


Fee estimation

For on-chain and lightning it would be nice to have a fee estimate, but especially rn for on-chain. @TonyGiorgio said it might require reworking send flow a bit to have a psbt I can review.
blocking https://github.com/MutinyWallet/mutiny-web/issues/57

Onchain fee estimation done, lightning is harder


Master

Unity prefab instances will now be exported as Babylon.js mesh instances.  The Babylon.js mesh will be the Unity prefab.  Details from the first instance (position, rotation, scale) are copied into the Babylon.js mesh then that first instance is removed from the list of instances

Woot! Live it! I'll check it this week end.
Thank you for this great contrib
Do not forget to update the what's new for 2.3:)

Awesome!  This isn't very flexible though.  You can't change materials or components, but its still a useful feature.

Yes instances are performant at the cost of flexibility

so is this considered a "major update" or just an "update"?  I'm in the process of updating the What's New file now


chore(main): release 0.13.0

:robot: I have created a release beep boop
0.13.0 (2024-01-20)
Features

rework triggers and deployments (#268) (4b7b096)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/nader-eloshaiker/screen-geometry-app/releases/tag/v0.13.0 :sunflower:


TabBarTestId is always a not found element

const MoreStack = createStackNavigator({
MoreScreen: {
screen: MoreScreen,
navigationOptions: {
headerTitle: ,
headerStyle: styles.headerCentered,
},
},
MoreStack.navigationOptions = {
tabBarIcon: ({ tintColor }) => (),
TabBarTestId: 'Child',
};
const PrimaryNav = createBottomTabNavigator({
HomeStack,
NotificationsStack,
InboxStack,
MoreStack,
},
it('should open second menu on tabbar', async () => {
await element(by.id('Child')).tap();
});

what error do you have? I had the same problem but solved using element(by.id('TabBarID')).atIndex(0)

@juand89  what is the version of react-navigation you have used at that case

@hajermay version 2.6.2

@juand89  im using a little bit old project with version 2.0.0  of react navigation i can guess the problem it comes from that reason.

@hajermay yes you are right! I changed the version of react navigation to 2.0.0 and the test fails

But my problem when i upgrade to 2.2.5 for exemple it always show me
navigation.getChildNavigation is not a function i have just create this
issue on react-navigation
On Fri, Sep 14, 2018, 9:57 AM juand89<EMAIL_ADDRESS>wrote:

@hajermay https://github.com/hajermay yes you are right! I changed the
version of react navigation to 2.0.0 and the test fails
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/wix/detox/issues/935#issuecomment-421281048, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AVASSnVNp2WpDjIU1yEGAx6Bm8TmaitBks5ua29ngaJpZM4Wm8qm
.


Adding  '**' operator for exponential

It is shameless, but I thought this would help noobs like me. It is quasi copy and paste your remark on my twitter:
'When you run "brisk meh" it creates a new directory called "meh", and puts a project in there.
This did help me a lot so I thought why not put it in there too.. 😅

Thank you!


Please add an option to force use different bitrate/quality settings

Hallo,
thx for this beautiful app.
It would be nice to have an option to force use different bitrate/quality settings for reasons:

when a video from mediathek isn't available in the high bitrate, but in medium or low ones (it will not even play when trying to load in high bitrate)
forcing low bitrate whenever possible when using metered internet to limit traffic no matter which global Android LAN settings are used. For when you can't tell Android it's a metered WLAN. Please make another slider regardless of used WLAN.
force low bitrate when you have a slow WLAN

Please make an option to stream mediathek in low bitrate also, as there is in the download section.
There's also an app called Bradybound (com.oxplot.bradybound.8.apk), which can limit the dl speed (but will ignore the upload speed).
This could be helpful for live streaming, if it automatically switches to lower qualities, which otherwise isn't able to load in low bitrate/quality. I just tried Bradybound set to ca. 100-300 KB/s with streaming from mediathek. It doesn't lower the bitrate sadly like it would when streaming from the website or mediathekviewweb.de. It instead stops, then caches and plays the next chunks as possible.
With live TV streaming this Bradybound limiting method works (you can see the difference in the channel logos), but this method is limiting the whole Android bandwith until you change it again.
Is there another way to force live streaming to use lowest settings by default?
Thx in advance!

Thank you very much for your suggestion!
As a temporary workaround for mediathek shows you can share indiviual qualities to your favourite video player.

it will not even play when trying to load in high bitrate

This should never happen. Can you provide an example video that does not play back correctly?
Live streaming has a completely diferrent technical setup than mediathek shows. It should already use adaptive streaming and use the correct bitrate, framerate and video size depending on network quality, screen size and device capabilities.
Mediathek shows consist of up to three separate mp4 files. There is currently no elegant way to seamlessly sand adaptively switch between them
I see a view improvements we can make to address your issues:

Provide a way to play back single mediathek show videos using an user selected quality (overriding the one selected in settings)
or expose a player menu for mediathek show videos where the user can temporarily select another quality (overriding the one selected in settings)
Expose a player menu for live streams where the user can temporarily select another quality (like the youtube player)


Add extra gpu operator

Description
The goal of this PR is to add the deployment of the GPU operators in the Spokes clusters.
Fixes # (issue)
Type of change
Please select the appropriate options:

[ ] Bug fix (non-breaking change which fixes an issue)
[ x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ x] This change requires a documentation update
[ ] This change is a documentation update

Testing
Deploy the spokes pipelines and see that the GPU operator is deployed in the spokes clusters.

[ ] Test A
[ ] Test B

Test Configuration:

Versions:
Hardware:

Checklist

[ ] I have performed a self-review of my own code
[ ] If a change is adding a feature, it should require a change to the README.md and the review should catch this.
[ ] If the change is a fix, it should have an issue. The review should make sure the comments state the issue (not just the number) and it should use the keywords that will close the issue on merge.
[ ] A change should not be merged unless it passes CI or there is a comment/update saying what testing was passed.
[ ] PRs should not be merged unless positively reviewed.

Please @danielchg  if you are working on the conflicts change it to draft until finished

:tada: This PR is included in version 1.1.13 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[CI/CD] dependabot CI runs for both push and pull (effectively double the processing)

Example here: https://github.com/project-chip/connectedhomeip/pull/30697
We have separate runs on "push on branch" and pr creation, with the intent of "run on master" as well as "run on user PRs".
Dependeabot seems to create branches on the main repo, so the result is that we run 2x the CI. We should only run one

Chris wanted me to look into this; will update as I dig into what needs to be done here

Issue: What the issue is: most of our CI runs with the on: push: trigger, which will trigger for every commit to the repository. Since contributor changes are performed on forks, on: push: does not trigger CI. Instead, CI triggers for changes made in forks through on: pull_request: when pull requests from those forks are created. However, dependabot creates branches directly in the main repository instead of being a real user with a fork on its profile. This will trigger on: push: whenever it commits and on: pull_request: when that branch is pushed to master.
Solution:

We can find all CI workflows with on: push: and modify it to on: push: branches-ignore: - 'dependabot/**', excluding dependabot branches from triggering CI
We can instead use on: push: branches: - 'master' to fulfill

with the intent of "run on master"

but we will need to confirm that we don't ever want CI to run on branches.

Vatsal and I split this task up. The two PRs cover the relevant CI that was run twice on the most recent dependabot branch.
To verify, I will create a fake "dependabot/test" branch and commit to it then PR to master. No duplicate CI runs means success. Additionally, I will verify that all CI with on: push: triggers either has the master limitation or some other which prevents dependabot runs.

Some files were missed / newly added that need this fix:
Build example - NXP
Build example - TI CC13XX_26XX
Darwin Tests
Test TV Casting Example
Build example - RW61X


Bluetooth doesn't connect automatically when the service starts

Upon startup, Bluetooth doesn't connect automatically with paired devices on startup. This is very annoying when using BalenaSound in the car as you need to navigate to phone's Bluetooth settings and locate the Bluetooth connection entry.
Steps to reproduce the issue:

Power on the Raspberry Pi with BalenaSound installed.
Connect a phone to BalenaSound Bluetooth service.
Restart the Raspberry Pi (unplug and plug back in).

Expected: Bluetooth automatically re-connects.
Actual: Bluetooth doesn't re-connect automatically, manual connection is needed.
Environment:
iPhone XS Max iOS 13.0 (haven't checked Android).
Raspberry Pi 3B
balenaOS 2.38.0+rev1
bluetooth-audio rev f623957

I am having kinda the same issue
When you unplug and plug the Pi back in i have to remove it from "My Devices" and then connect to it from scratch or it will error on connection
Enviroment:
iPhone XS iOS 12.4
Raspberry Pi Zero WH
balenaOS 2.32.0+rev1
Balena-Sound master v0.0.11

It seems to me hard to bring this into balena-sound. But when you give me the MAC adress  of your mobile can i write a script which connects you automatically per bluetoothctl to your RPI 3. You need then to put the updated file into your copy of balena-sound and then build it again. If you dont know how you get rhe MAC adress i can try to give you instructions, but please try it first yourself.

Hello, I've implemented reconnection to last known device on PR #57 . If you guys get a chance give it a try and let me know what you think.

Closing issue as PR #57 was merged into master. Please let me know if you have issues with this. Thanks!


Make scale test point at staging

Make output filename match what's read as input below, change step name to match staging.

Not completely. The filename I changed in an earlier PR didn't match the filename a few lines below the change. This fixes that.


update skylib to 1.0.3, fix stardoc name

Fixes: #120

This entire testing process is too fragile.


fetchRemoteConfige compltionhander not called sometime

[REQUIRED] Step 2: Describe your environment

Xcode version: 9.3
Firebase SDK version: 3.17.0
Firebase Component:Auth, Core, RemoteConfig, Messaging, FirebaseAnalytics, FirebaseInstanceID))

[REQUIRED] Step 3: We have Implemented firebase  RemoteConfig.
while testing we have noticed some time compltionhandler of fetchremoteConfig not called.
Steps to reproduce:
we don't facing this everytime but it occurs randomly
Relevant Code:
// TODO(you): code here to reproduce the problem

var expirationDuration = 3600
var remoteConfig: RemoteConfig!
remoteConfig = RemoteConfig.remoteConfig()
let remoteConfigSettings = RemoteConfigSettings(developerModeEnabled: false)
remoteConfig.configSettings = remoteConfigSettings!
remoteConfig.fetch(withExpirationDuration: TimeInterval(expirationDuration)) { (status, error) -> Void in
if status == .success {
self.remoteConfig.activateFetched()
completion()
} else {
completion()
}
}

Does this only happen on subsequent fetches? Looks like your expiration behavior may be causing infrequent refreshes.

Does this only happen on subsequent fetches?
Looks like your expiration behavior may be causing infrequent refreshes.
-> Yes, we are calling multiple times this function.

Try setting the expirationDuration to 0 and seeing if you get refreshes immediately.
If not, please enable debug logging via FIRConfiguration.sharedInstance().setLoggerLevel(.debug) and post any relevant logs here.


RoleManager documentation and sample code

It is very hard to find something developers can hold on to start using this class.  It will be great if we can find code sample on how to use and instantiate this class.  I tried to create a razor page where I can CRUD roles, but when I try to instantiate this class in the confines of this razor page I get an error. I tried to use dependency injection, but it did not work.  The problem is there is no documentation or guide on how is the proper place, proper way of using this class. You are left to try things you may now know is the right way or recommended way to use this class. For example I found the following description of the class but without a proper code sample or more explanation is hard to know where and how I have to use it.
public RoleManager (Microsoft.AspNetCore.Identity.IRoleStore store, System.Collections.Generic.IEnumerable<Microsoft.AspNetCore.Identity.IRoleValidator> roleValidators, Microsoft.AspNetCore.Identity.ILookupNormalizer keyNormalizer, Microsoft.AspNetCore.Identity.IdentityErrorDescriber errors, Microsoft.Extensions.Logging.ILogger<Microsoft.AspNetCore.Identity.RoleManager> logger);

See #1910


chore: Widget select logic

Removes the widget selection when dropping a widget on the canvas
Removes the widget search auto focus
Fixes: #30578

Summary by CodeRabbit

Refactor

Optimized widget selection and dragging logic within the layout editor.

User Experience

Updated the behavior of the search input in the widget sidebar to enhance user interaction.

/build-deploy-preview skip-tests=true

/ok-to-test<EMAIL_ADDRESS>@tag.Widget, @tag.Sanity"

@marks0351 Could you review this? You have the most context on the files that have been modified.

Code changes look good.
widget selection is done as soon as a widget is moved/added to show property pane of respective widget. In this PR we are removing widget selection when a widget is added. I think most DnD based test cases might fail coz they anticipate respective property pane to show up once a widget is added. pls do check for those cases.
cc: @riodeuno @somangshu

/ok-to-test<EMAIL_ADDRESS>@tag.Widget, @tag.Sanity"

@marks0351 is a similar change handled in Anvil already?

@marks0351 is a similar change handled in Anvil already?

yep we do select the newly added widget via sagas(anvilDraggingSagas/addWidgetsSaga).

/ok-to-test<EMAIL_ADDRESS>@tag.Widget, @tag.Sanity"

/ok-to-test<EMAIL_ADDRESS>


Expose .IsWiping and .IsSetting as public properties/methods ?

Left internal - useful for unit testing.
An async Wipe() method could be added


chore: create a single CLI download image

The Konflux team has made changes to the build-source-image task, which previously failed when adding more than just cosign, gitsign and rekor-cli to the image. When we added ec to it, the task began to fail due to an overly large results string, which Konflux could not handle. That limitation in Konflux has been eliminated.
The change in this commit is to create a single CLI download image, which can be used by the operator, instead of the multiple currently in place. Once this change is committed on the main branch, a new component can be created in Konflux to test the build. If it is successful, we can remove the following components from the cli Application in Konflux, and subsequently remove those Dockerfiles from this repo.
The following components can be removed after this build is successful.

cli/client-server-cg
cli/client-server-ec
cli/client-server-f

With this are we planning on using this image to server the binary's via Apache? As in adding a web server to this image. Currently we copy these images into a web server via init containers, where as before we had to do this, this image would have included the web server.
If we decide to add the web server to this image, I believe the binary's need to be placed on a specific path to be server correctly "/var/www/html/clients/"
https://github.com/securesign/secure-sign-operator/blob/main/internal/clidownload/component.go#L124-L166

@JasonPowr I've modified this image to include the httpd server and placed the binaries in /var/www/html. PTAL


bug：InternalError (see above for traceback): cudnn PoolForward launch failed

Hello, I encountered such an error when running the code. I think it should be an environmental problem. Can you tell me the version of cuda and cudnn you use? Or do you have a better suggestion?Thanks！

Sorry to disturb you, I just saw the same problem in the closed issue.


[Drag event] Hard/impossible to access all edge pixels in some situations

Affected hardware version
Bangle 2
Your firmware version
2v16.45
The bug
When using Dragboard or Touch Keyboard (with the one-to-one input setting enabled) I have noticed I will pretty often have a hard time accessing characters closest to the edge of the screen when draging.
On Dragboard for example I sometimes find it's hard to access the bottom red rectangle to select letters if draging from further up the screen. This doesn't happen if I haven't calibrated the screen, but if I have it can become impossible to drag back down to the red rectangle to select letters. (calibrated like this while testing now: touchX1: -2, touchY1: 9, touchX2: 195, touchY2: 177)
I think I remember my problem with Touch Keyboard was present even on uncalibrated screen. I'll try again and come back with an update.
I've written this script below for seeing how the drag event acts close to the edges.
// Script for documenting the first/last pixel reported as touched on a drag from outside a screen edge, in on the screen and exiting the screen via the same edge.
//Upload to RAM in Web IDE and see the output in the console window

var lastB = 0;
var enterX, enterY, exitX, exitY, deltaX, deltaY;

function dragHandler(e) {
  if (lastB==0){
    print('enter via edge:');
    print(e);
    enterX = e.x;
    enterY = e.y;
  }

  if (e.b==0) {
    print('exit via same edge:');
    print(e);
    exitX = e.x;
    exitY = e.y;

    deltaX = exitX - enterX;
    deltaY = exitY - enterY;
    print('Delta x:', deltaX);
    print('Delta y:', deltaY);
    print('\n');
  }

  // EDIT: Added lines to get direct visual feedback.
  g.clear();
  g.setColor(1,e.b,e.b); // White when touching screen, red when screen released.
  g.drawLine(0,e.y,g.getWidth(),e.y);
  g.drawLine(e.x,0,e.x,g.getHeight());

  lastB=e.b;
}

Bangle.on('drag', dragHandler);

My findings are roughly (deltas given without sign):

Top edge seems to not have too much problem, delta close to zero.
Bottom edge I get a delta between 10-17 most of the time
Left edge I get a delta between 6-10 most of the time
Right edge I get a delta between 6-15 most of the time

I did the measurements with the touch screen uncalibrated.
I don't know exactly what solution I'm after here. But:

I'd like to make sure Dragboard and Touch Keyboard works better. Maybe I can achieve this in Dragboard by moving the touch boundary between the red and green rectangles.
Ideally something could be done so that deltaX and deltaY respectively are close to zero when entering and exiting an edge, when running the code above. But I suspect this is outside of espruino so could be hard.

Here is an output where I did top edge, bottom edge, left edge and right edge in that order (EDIT: I redid these with uncalibrated screen):
enter via edge:
{ "x": 102, "y": 9, "b": 1, "dx": 0,
  "dy": 0 }
exit via same edge:
{ "x": 101, "y": 12, "b": 0, "dx": 0,
  "dy": 0 }
Delta x: -1
Delta y: 3

enter via edge:
{ "x": 99, "y": 187, "b": 1, "dx": 0,
  "dy": 0 }
exit via same edge:
{ "x": 92, "y": 177, "b": 0, "dx": 0,
  "dy": 0 }
Delta x: -7
Delta y: -10

enter via edge:
{ "x": 11, "y": 93, "b": 1, "dx": 0,
  "dy": 0 }
exit via same edge:
{ "x": 9, "y": 70, "b": 0, "dx": 0,
  "dy": 0 }
Delta x: -2
Delta y: -23

enter via edge:
{ "x": 191, "y": 114, "b": 1, "dx": 0,
  "dy": 0 }
exit via same edge:
{ "x": 179, "y": 112, "b": 0, "dx": 0,
  "dy": 0 }
Delta x: -12
Delta y: -2

Installed apps
No response

Updated the script above to also give visual feedback (vertical/horizontal lines) on the watch when testing.

Here is what it looks like for me when the screen is uncalibrated (entering and exiting edges in order top, bottom, left, right):

Here is what it looks like for me when the screen is calibrated touchX1: -3, touchY1: 19,   touchX2: 192, touchY2: 176 (entering and exiting edges in order top, bottom, left, right):

For enter/exit where the value is ~15px from the edge of the screen when calibrated - I'd say that's totally acceptable and you just need to cope with that in code.
If we calibrate the screen so it matches with your finger position then probably the 'edge' isn't going to be exactly on 0px, and I don't see there's a way around that.
But for the right-hand side, what happens if you just draw the coordinates?
Bangle.on('drag', e => {
  g.setPixel(e.x,e.y);
});

This is what I get when going off the edge of the screen:

I tried your code here and it works fine for me, I see stuff like:
enter via edge:
{ "x": 161, "y": 65, "b": 1, "dx": 0,
  "dy": 0 }
exit via same edge:
{ "x": 158, "y": 84, "b": 0, "dx": 0,
  "dy": 0 }
Delta x: -3
Delta y: 19

So maybe you could just try recalibrating? It's entirely likely you just didn't press right on the coordinates, and so the screen is now slightly uncalibrated so you can't reach the RHS

For enter/exit where the value is ~15px from the edge of the screen when calibrated - I'd say that's totally acceptable and you just need to cope with that in code.

I can buy that. But it's weird the watch can register closer to the edge when entering and not be able to access the same pixels when draging back off the same edge..

But for the right-hand side, what happens if you just draw the coordinates?

Like previously with my code in the prntscreens, if the screen is uncalibrated I can get closer to the edge (go past it for some edges), but if calibrated via settings I will generally not be able to get close to all edges.

So maybe you could just try recalibrating? It's entirely likely you just didn't press right on the coordinates, and so the screen is now slightly uncalibrated so you can't reach the RHS

I tried recalibrating via settings some times also deliberately hitting the crosshairs off center to try different stuff out. It was hit and miss I would say and I had a hard time getting something pretty good.
However I tried the Touchscreen Calibration app now again, and I seem to get a better calibration that way at least with regards to accessing edge pixels. It's possible it doesn't calibrate as good in the y-direction well in on the screen where it might register a little below where I intend to touch.

The solution I guess then is to have sufficiently large borders where input is not needed for the apps functionality. These could be hardcoded or depend on the users Touch screen calibration.

As on https://github.com/espruino/BangleApps/pull/2558 - I don't think the solution is to remove touch calibration as for most users this is needed to make the keyboard usable.
As far as I can tell, in kbtouch the default is to use dragging to select the right key, so actually the fact you can't get to the edge doesn't matter at all? It's only if you deliberately set it up with 'one to one' and you calibrate the screen and only then on your watch - so I don't think that's really such a big deal?
But for dragboard  it seems we could pretty trivially increase the padding so it works fine? I think that is worth doing.

But it's weird the watch can register closer to the edge when entering and not be able to access the same pixels when draging back off the same edge?

I think this is probably just how the touch controller tries to work out where the finger is - it's effectively got a bitmap of capacitance and it's trying to guess a central point for it. Probably when you have dragged on and it knows how big your finger is, it's more willing to say it's offscreen when only half of it is. Going on, it doesn't know if it's looking at a small or a big finger.
back to what you mentioned on #2558:
Looking at your calibration, the values are way off what I would expect. The defaults (uncalibrated) are 0,0,160,160 and you are 6,9,193,176 - so it is very much as if everything is shifted.
I know you've got your code that draws the X/Y lines - if you update that for every drag event, do those lines really appear to line up with where your finger is when it's calibrated?
I guess the other option is for the calibration to deliberately 'zoom' the coordinates slightly, to ensure that even if calibration was off, you could still reach closer to the edges.

As far as I can tell, in kbtouch the default is to use dragging to select the right key, so actually the fact you can't get to the edge doesn't matter at all? It's only if you deliberately set it up with 'one to one' and you calibrate the screen and only then on your watch - so I don't think that's really such a big deal?

No, it's probably not a very big deal, agreed. The one-to-one input is my preferred way of using kbtouch, but probably not many other use it. And I jump between keyboards relatively frequently.

But for dragboard it seems we could pretty trivially increase the padding so it works fine? I think that is worth doing.

I'll probably do that :+1:

Thanks for those pics. What happens if you try pressing towards the bottom right of the screen?
Maybe try:
Bangle.on('drag',e => {
  g.clear(1).drawLine(0,e.y, 180,e.y).drawLine(e.x,0,e.x,180);
});

and then just try with dragging your finger?
Those two presses are relatively central on the screen but even so it looks like the one slightly to the bottom right is off center.
I just tried here after calibration and mine feels a bit 'off' too. I think it's the calibration that's broken. I just tried adding a mild 'zoom' to it and it does seem better.
Try this:
// disable touchscreen calibration (passed coords right through)
Bangle.setOptions({touchX1: 0, touchY1: 0, touchX2: g.getWidth(), touchY2: g.getHeight() });

var P = 32;
var corners = [
  [P,P],
  [g.getWidth()-P,P],
  [g.getWidth()-P,g.getHeight()-P],
  [P,g.getHeight()-P],
];
var currentCorner = 0;
var currentTry = 0;
var pt = {
  x1 : 0, y1 : 0, x2 : 0, y2 : 0
};

function showTapSpot() {
  var spot = corners[currentCorner];
  g.clear(1);
  g.drawLine(spot[0]-32,spot[1],spot[0]+32,spot[1]);
  g.drawLine(spot[0],spot[1]-32,spot[0],spot[1]+32);
  g.drawCircle(spot[0],spot[1], 16);
  var tapsLeft = (1-currentTry)*4+(4-currentCorner);
  g.setFont("6x8:2").setFontAlign(0,0).drawString(tapsLeft+/*LANG*/" taps\nto go", g.getWidth()/2, g.getHeight()/2);
}

function calcCalibration() {
  g.clear(1);
  // we should now have 4 of each tap in 'pt'
  pt.x1 /= 4;
  pt.y1 /= 4;
  pt.x2 /= 4;
  pt.y2 /= 4;
  print(pt);
  // work out final values
  var calib = {
    x1 : Math.round(pt.x1 - (pt.x2-pt.x1)*P/(g.getWidth()-P*2)) + 16,
    y1 : Math.round(pt.y1 - (pt.y2-pt.y1)*P/(g.getHeight()-P*2)) + 16,
    x2 : Math.round(pt.x2 + (pt.x2-pt.x1)*P/(g.getWidth()-P*2)) - 16,
    y2 : Math.round(pt.y2 + (pt.y2-pt.y1)*P/(g.getHeight()-P*2)) - 16
  };
  print(calib);
  Bangle.setOptions({
    touchX1: calib.x1, touchY1: calib.y1, touchX2: calib.x2, touchY2: calib.y2
  });
  g.setFont("6x8:2").setFontAlign(0,0).drawString(/*LANG*/"Calibrated!", g.getWidth()/2, g.getHeight()/2);

  Bangle.on('drag',e => {
    g.clear(1).drawLine(0,e.y, 180,e.y).drawLine(e.x,0,e.x,180);
  });
}

function touchHandler(_,e) {
  var spot = corners[currentCorner];
  // store averages
  if (spot[0]*2 < g.getWidth())
    pt.x1 += e.x;
  else
    pt.x2 += e.x;
  if (spot[1]*2 < g.getHeight())
    pt.y1 += e.y;
  else
    pt.y2 += e.y;
  // go to next corner
  currentCorner++;
  if (currentCorner>=corners.length) {
    currentCorner = 0;
    currentTry++;
    if (currentTry==2) {
      Bangle.removeListener('drag', dragHandler);
      return calcCalibration();
    }
  }
  showTapSpot();
}

var dragx=0, dragy=0, dragc=0;
function dragHandler(e) {
  if (e.b) {
    dragx+=e.x;
    dragy+=e.y;
    dragc++;
  } else {
    if (dragc) touchHandler(null, { x : dragx/dragc, y : dragy/dragc});
    dragx=0, dragy=0, dragc=0;
  }
}
Bangle.on('drag', dragHandler);

showTapSpot();

It doesn't save calibration but after it's done you can then press and drag to get an idea of the positioning...

Thanks for those pics. What happens if you try pressing towards the bottom right of the screen?
Maybe try: (...)

I feel like the accuracy of the touches and drags is pretty much the same for the area I can access when calibrated. I still can't access the edges with that short test code, as before.

I just tried here after calibration and mine feels a bit 'off' too. I think it's the calibration that's broken. I just tried adding a mild 'zoom' to it and it does seem better.
Try this: (...)

With this zoomed calibration I can access virtually the whole screen area (I'm just a few pixels away from bottom and right border). But for me it comes at the expense of accuracy on the area I could reach also with the old calibration. I feel this is to be expected with adding the zoom step? However, it may prove to be a good compromise!
I think this is a bit hard to capture with a video, but if you would want one maybe I can manage to shoot one with a tripod or something :P

Regarding this:

I feel this is to be expected with adding the zoom step? However, it may prove to be a good compromise!

I guess some clever math could zoom mainly "towards the bottom right" and not towards top left, and that may be the best thing for my watch and my way of pressing/draging on it. But that is probably overkill :P

I finally took a stab at this and now have a solution that works with my watch and calibration:
{
  let touchX2 = Bangle.getOptions();
  let touchY2 = touchX2.touchY2;
  touchX2 = touchX2.touchX2;
  let zoomX = (touchX2 + 10)/g.getWidth();
  let zoomY = (touchY2 + 15)/g.getWidth();

  let zoomDrag = (e)=>{
    if (e.zoomed) return;
    E.stopEventPropagation();
    e.zoomed = true;
    e.x = Math.round(e.x*zoomX);
    e.y = Math.round(e.y*zoomY);
    Bangle.emit("drag", e);
  };

  Bangle.prependListener("drag", zoomDrag);
}

I add this at the end of the boot code scripts by specifying a high boot number.
With this I can now access virtually all 176x176  pixels also when I've calibrated the screen.
I don't know how well it would work for other people. It's currently not a truly general solution.

I'm a little iffy about hard-coded values - while it may work for you it could do with a bunch of testing to see if it's better for everyone.
Also it feels like if some other code did Bangle.prependListener before yours, it'd get called twice, once with zoomed and once without zoomed coordinates - which if it's using drag for scrolling could make scroll twice as fast?
I guess part of me is thinking this issue is 18 months old, and you're the only one to comment on it - so I wonder if it's worth putting too much work into solving this for everyone, as it might be something that's only really an issue for you. If it works for you that's great, and I guess if we can make it an app that can just be installed then that makes it easy for everyone, but I don't think this is something we should be trying to build in yet.

Makes total sense :)
If I get to a more general solution that still works well for me, then I'll do a PR for an app.

I realized that applying the zoom affects the accuracy improvements of the previous calibration. So maybe it makes sense to avoid zooming for most of the screen area.
Below is another solution that only applies magnification near the edges I can't reach otherwise (right and bottom). I haven't decided on which solution I like best.
{
  let zoomDrag = (e)=>{
    "ram";
    let reachableX = 153;
    let startZoomX = 130;
    let reachableY = 153;
    let startZoomY = 130;
    if (e.zoomed) return;
    E.stopEventPropagation();
    e.zoomed = true;
    if (e.x >= startZoomX){
      print(e.x);
      e.x = startZoomX + Math.round((e.x-startZoomX)*(175-startZoomX)/(reachableX-startZoomX));
    }
    if (e.y >= startZoomY){
      print(e.y);
      e.y = startZoomY + Math.round((e.y-startZoomY)*(175-startZoomY)/(reachableY-startZoomY));
    }

    Bangle.emit("drag", e);
  };

Bangle.prependListener("drag", zoomDrag);
}

Also it feels like if some other code did Bangle.prependListener before yours, it'd get called twice, once with zoomed and once without zoomed coordinates - which if it's using drag for scrolling could make scroll twice as fast?

Is it possible to listen for prependListener registrations? If so I could listen to such registrations and reinitialize my zoomDrag handler prepended again.
But I guess this would open up for infinite loops of re-registering listeners if two or more were to do this.

Is it possible to listen for prependListener registrations?

Yes, i guess you could override prependListener on Bangle.js.
However this all got me thinking... We actually do all this zooming stuff as part of the calibration already: https://github.com/espruino/Espruino/blob/master/libs/banglejs/jswrap_bangle.c#L1992-L1993
So you don't need any of this. Literally all you need is after the touchscreen is calibrated, you take the touchX1/etc values from settings, and you move them slightly closer together - like add 10px to touch1 and take 10px from touch2.
Then it's sorted - so actually your boot code could just be:
var o = Bangle.getOptions();
Bangle.setOptions({
  touchX1:o.touchX1+10,
  touchY1:o.touchY1+10,
  touchX2:o.touchX2-10,
  touchY2:o.touchY2-10});

... but actually I look back now and this is what I was basically suggesting in https://github.com/espruino/BangleApps/issues/2457#issuecomment-1422327622 anyway (tweaking the calibration)...
So basically we 100% do not want something that overrides drag, etc. We have the calibration code for exactly this kind of thing - it's just figuring out what numbers we should use for calibration

Ok, thanks for the links and example! I'll try a variation of that for a while.

Just leaving a note.
So what's working best for me is my solution where I manually determine how far I can reach and then intercept and modify the drag event.

So basically we 100% do not want something that overrides drag, etc. We have the calibration code for exactly this kind of thing - it's just figuring out what numbers we should use for calibration

I guess my watch (and/or way of using it) is so special that there is no reason to tweak the way it's done for everyone else. If someone else shows up here saying they have similar troubles - maybe we reconsider then.

what's working best for me is my solution where I ...  modify the drag event

Ok, it would be interesting to know why really - what doesn't changing the options do right?
Because changing the options should provide you with better per-pixel accuracy.
I looks like the way you're doing it, you're not cropping the pixel x/y to the screen bounds - could that be it? I'm pretty sure I didn't used to do that in Bangle.js either, but then someone complained so I had to start cropping - so just be aware that some apps may have issues with out of bounds taps.

what's working best for me is my solution where I ...  modify the drag event

Ok, it would be interesting to know why really - what doesn't changing the options do right?
Because changing the options should provide you with better per-pixel accuracy.

It felt like the accuracy decreased on the parts of the screen that already did work with calibration before (basically the square 0<=x<=153, 0<=y<=153). So I'd rather keep that calibration on most of the screen. But then take some of the rightmost and bottom parts of the reachable area and stretch them out so I can reach the borders.

I looks like the way you're doing it, you're not cropping the pixel x/y to the screen bounds - could that be it?

I'm not entirely sure what you mean. My solution doesn't let me go beyond the lcd pixels. I take x-values [130, ... , 153] and transform them to [130, ... , 175] (and the same for y-values).

I'm pretty sure I didn't used to do that in Bangle.js either, but then someone complained so I had to start cropping - so just be aware that some apps may have issues with out of bounds taps.

As I say I'm not sure what you mean here. But I wonder if undoing that cropping would help my situation?
My assumption for my ground problem is that the LCD pixels and touch pixels are too misaligned on my watch - and that there are not enough touch pixels to the right of the LCD screen to use when having done the calibration.

Ahh, ok, gotcha - that makes a lot more sense. So the issue is really that your touchscreen appears nonlinear?
I don't know if this helps but I re-read what you write at the start, about your calibration values: touchX1: -2, touchY1: 9, touchX2: 195, touchY2: 177
The default is 160 for touchX/Y2, so the calibration will absolutely be making it difficult to reach the right and bottom edges (although at least it won't crop the values!). I guess calibration got the weird values precisely because there's something nonlinear with your touchscreen.
So personally, I think this is one of those things where it's probably just your device that needs this - so I'll close for now. If you want to make it an app that's great, but it should make it clear in the description that the first port of call should be just using the normal calibration function.

Thanks for helping me out 😊

Just leaving a note.
I've since bought another Bangle.js 2 second hand and I see the same behavior on this one as well. It seems unlikely I'd get two watches with the same manufacturing "defect" mismatching the lcd and digitizer - I'll chalk it up to my subjective expectation and/or way my fingers and the touch screen interact.
With that said I'm a little more motivated to turn my script into an app so I might do that - no promises.

Interesting - I guess it may well be something about how you press the screen, as the touchscreen controller will just report the midpoint of the area touching the screen.
It would be handy for me if you could make an app where you can display the point at which you press (maybe horiz/vert lines), and then maybe make a video of what you see?
If there's some way we could characterise this in the touchscreen calibration tool and add a correction factor into the firmware then I guess that would be perfect.
But if you want to make an app to fix it, I'd strongly suggest setting touchX/Y1 to 0 and touchX/Y2 to 176 as then the built-in touchscreen calibration will pass the raw coordinates through and you'll have the most 'range' to play with


The wrn prototxt file in cls

deploy_wrn50-2.prototxt doesn't seems like the real ‘wrn’ network，its just the nomal resnet-50 i
think..

Check carefully, it is really wider than resnet50

@soeaver WRN use dropout between two 3*3 conv in a  residual block, but in this prototxt I just see nomal bottleneck conv with bn.

Just for small dataset, please read the paper and source code again.

@soeaver  I see.. Its quite interesting that two 3*3 suits small datases well but Imagenet needs bottleneck..


[Feature] Use Python assert in smart contract

It would be neat if we could replace something like:
if not CheckWitness(Owner):
    print("Unauthorized user")
    return False

with:
assert CheckWitness(Owner), "Unauthorized user"

I'm moving to just implement this in the following way in my smart contracts:
OnError = RegisterAction('error', 'message')
...
sc_assert(CheckWitness(Owner), "Unauthorized user")

def sc_assert(condition, msg):
    if not condition:
        print(msg) # for neo-python console
        OnError(msg) # for neo-cli ApplicationLog
        throw_if_null(0)

but it would be cool if it were just natively supported with the existing assert keyword in Python.
(I noticed that the neo-cli implementation ofgetapplicationlog returns the notification events that triggered before the contract faulted, so the sc_assert function I wrote takes advantage of that - I see this as being a huge help for people trying to debug contract failures after-the-fact. So if this suggestion were to be implemented as a built-in it would be nice to replicate that feature too.)

I have not looked at what getapplicationlog does, but notification events that triggered before the contract faulted can be logged/displayed with neo-python via config sc-debug-notify. Does that help?
sc-debug-notify - toggle printing smart contract Notify events on execution failure

notification events that triggered before the contract faulted can be logged/displayed with neo-python via config sc-debug-notify. Does that help?

Yes, that could eliminate the need for the extra print statement.

One other issue I am having with my bespoke assert implementation is that it appears to be impossible to have a void return type for a function. So unless I use
ret = sc_assert(something_is_true(), "Not true!")

I end up with an extra value on the evaluation stack at the end of execution, which is undesirable. But adding the unused variable assignment is also kind of inelegant.

I think I found the solution and it doesn't require any changes to neo-boa, just a little knowledge of Python internals:
OnError = RegisterAction('error', 'message')

def Main(operation, args):
    assert args[0] == True, 'Got false argument!'
    return True

def AssertionError(msg):
    OnError(msg)
    throw_if_null(0)


Add timestamp to messages

Adding a timestamp to every MQTT message would make it easy to see, if the message shows the current state of the sensor or if the sensor was unreachable.

#20


Network State reports unreachable on IPV6 only network

I would be happy to submit a pull request, the changes are very simple (Need to check for address of type AddressFamily.InterNetworkV6 in addition to InterNetwork), but the change is trivial.
As I state below, iOS is the biggest problem, but I couldn't for the life of me figure out what the random offline issue was for android, I assume it's the same, but may not be.
Please check all of the platforms you are having the issue on (if platform is not listed, it is not supported)
I assume it affects all platforms, but its the biggest issue on iOS, my app was rejected because of failure to support ipv6 due to this issue.

[x] iOS
[x] Android
[ ] UWP
[ ] .NET Standard

Version of OS(s) listed above with issue
Any
Version of this library
5.0.5
Expected Behaviour
Network State should support ipv6 only networks, apple started requiring support for this in June 2016, Reference: https://developer.apple.com/support/ipv6/
Actual Behavior
When connected to an ipv6 only network, network state is reported as unreachable. (No exception)
Code of Conduct

No Library Update Requests
No Timeline Requests
Please read through the documentation and FAQs
Please be respectful of my time. I put a lot of effort into my libraries.  I try to support the issue requests within a day.  I'm not psychic, so please take the time to fill out the above form as complete as possible

I would be happy to accept any PR you could provide.  I will look into this for the next release, but that could be a few weeks away.


Stop polling on prepare_for_draining callback

From Broadway v1.1.0 changelog:

No longer set demand to :accumulate when draining, for compatibility
with GenStage v1.2+. This means that any polling implementation must
implement the prepare_for_draining callback and stop polling
messages. You can check how BroadwaySQS tackles this problem as an
example

The changes in this commit are indeed directly based on the changes from BroadwaySQS: https://github.com/dashbitco/broadway_sqs/commit/5b8f18a78e4760b5fcc839ad576be8c63345add0

Hey @kzemek, Thanks for the contribution and sorry for such a long delay!
I have merged and cut the release v0.9.0 🎉


feat(frontend): Update how it works modal to new design

Motivation
Adjust the how it works modal to match the new Design.
Changes

Update cover image
Update text
Make button primary to match design

Before:

New:

Same feedback as in #2936

Downscaling the image makes the text unreadable. Will leave it as is.


Add documentation about deploying

This is a first stab at documenting the deployment process.
I've made a slight change to the process while writing this document. We would previously add details of the changes to the calendar event itself but it seems more appropriate to use GitHub's releases functionality.

Looks great to me! :+1: :+1:

Rebased and force pushed ready to merge.


[FR] Support for Cloud Run gen1 execution environment

should use latest minor release of v1.9.*:

https://www.tcpdump.org/release/libpcap-1.9.1.tar.gz

tracking implementation at ft/run-gen1:
https://github.com/gchux/cloud-run-tcpdump/blob/ft/run-gen1/base-image/Dockerfile

tests were successful using:

libpcap v1.9.1
tcpdump v4.99.5

build pipeline also working OK, tcpdumpw build without problems.

new PCAP sidecar base images available at: https://github.com/gchux/cloud-run-tcpdump/tree/ft/run-gen1/base-image

see commit: https://github.com/gchux/cloud-run-tcpdump/commit/e4dcdf4847b7fae8cbdd7d9ea3bbc9ee894f0d61

keeping the build process DRY: https://github.com/gchux/cloud-run-tcpdump/commit/3ada5fd78740c11b6f85aab9f5cdde0e395291e1

in general, both libpcap and tcpdump are fetched from: https://www.tcpdump.org/release/
and then a Debian base image with the shared objects and binaries will be created to be used as-is or to import assets from it.
The base image is then used to create both:

the sidecar ubuntu:22.04 base image used to build the sidecar,
and the golang base image used to build tcpdumpw

This approach should be simple enough to have full control over the underlying dependencies and still keep the build simple.

available at ghcr.io/gchux/cloud-run-tcpdump:v1.0.62-RC1
relevant commits:

https://github.com/gchux/cloud-run-tcpdump/commit/4a27b7aeaea5be2a3edff25bca7884b74f2d9ba4
https://github.com/gchux/pcap-cli/commit/45d1cc241ef2cd38402e4cdddd576ea0cb07db27

https://github.com/gchux/pcap-cli/commit/45d1cc241ef2cd38402e4cdddd576ea0cb07db27 is of particular interest as it improves handling of SIGTERM by the PCAP packet transformer so that:

if SIGTERM is received while any number of translations are in process, it will drop them ( not write the into writers ) into STDERR instead.
PCAP writers are now context aware, and if not std{out|err} those will be closed on SIGTERM
prevent leaking of goroutines by translations consumers ( 1 per writer per NIC ); this is also related to SIGTERM handling improvements.
In order to operate in the least blocking possible way, PCAP transformer commits to translate and write every packet delivered by the gopacket engine if the context is not done by the time PCAP transformer received the packet; commitment here means that PCAP transformer is expected to generate 1 packet translation and to have that 1 packet translation written into all writers

transformer also waits for all translations it has committed to fulfill before it stops.
this commitment is disrupted by SIGTERM, which as mentioned may happen:

while packet translations are already in progress
before the translation process is started ( while translation workers are waiting from a gouroutine in the pool to become available )

this means that if the transformer has already committed to translate and write a packet before SIGTERM is received, it must also take care of:

not starting queued translation workers if context is done before a goroutine is available; which means that it must rollback its commitment by calling done on the sync.WaitGroup

this also applies when goroutine pool operates in FIFO mode

overall we should be losing close to 0 JSON packet translations captured before SIGTERM is trapped: either those packets will be fully dumped into their respective PCAP writers or into STDERR, provided that all the work is completed within 2s.

@thomasmburke I think that we should define a strategy to have X amount of the PCAP sidecar flavors based on the combination of libpcap and tcpdump versions.
Maybe we can provide 2:

bleeding edge: latest version for both dependencies.
Cloud Run gen1 compatible combination.

see: https://github.com/gchux/cloud-run-tcpdump/commit/1ab445c7e413bc112a65c136b6f1dc8a0774dfa1

base image for all builders: pcap-base:libpcap-v${LIBPCAP_VERSION}_tcpdump-v${TCPDUMP_VERSION}
base image for golang builders: pcap-golang:libpcap-v${LIBPCAP_VERSION}_tcpdump-v${TCPDUMP_VERSION}
base image for sidecar builder: pcap-sidecar:libpcap-v${LIBPCAP_VERSION}_tcpdump-v${TCPDUMP_VERSION}

PCAP sidecar now comes in to flavors: gen1 and gen2

ghcr.io/gchux/cloud-run-tcpdump:v1.0.62-gen1-RC1

ghcr.io/gchux/cloud-run-tcpdump:v1.0.62-gen2-RC1

Both Docker image flavors now contain labels to identify the version of libpcap and tcpdump used to build the base images.

Build process for any flavor no longer installs libpcap nor tcpdump from ubuntu repos, it fetches and builds specific versions from https://www.tcpdump.org/release/ ( this allows the PCAP sidecar to have an independent release track from Canonical ).

docker_build script now requires to specific runtime environment which allows to pin versions of libpcap and tcpdump per runtime ( i/e: Cloud Run, App Engine, Kubernetes Engine, etc... ).

we are now almost good to call this FR done.

no that the docker_build required the environment to build the PCAP sidecar for, we need to update the Cloud Build branch and the README

images for v1.1.0

Cloud Run gen2: ghcr.io/gchux/cloud-run-tcpdump:v1.1.0-gen2

alias ghcr.io/gchux/cloud-run-tcpdump:newest
contains newest versions of libpcap (v1.10.5) and tcpdump (v4.99.5)

Cloud Run gen1: ghcr.io/gchux/cloud-run-tcpdump:v1.1.0-gen1

alias ghcr.io/gchux/cloud-run-tcpdump:latest
contains versions of libpcap (v1.8.1) and tcpdump (v4.9.3) that work with both Cloud Run gen1 and gen2


org.json dependency is of a very old version

current version: 20090211.
consumers of the package which are using a newer version of it can end up having conflicts, at it seems that newer versions have implicit casting when reading Kusto's response, which breaks the current implementation of the library:
                JSONArray resultsRows = table0.getJSONArray("Rows");
                ArrayList<ArrayList<String>> values = new ArrayList<>();
                for (int i = 0; i < resultsRows.length(); i++) {
                    JSONArray row = resultsRows.getJSONArray(i);
                    ArrayList<String> rowVector = new ArrayList<>();
                    for (int j = 0; j < row.length(); j++) {
                        rowVector.add(row.getString(j)); <--- this breaks with newer versions, as the JSONArray implicitly casts numerics
                    }
                    values.add(rowVector);

solved in https://github.com/Azure/azure-kusto-java/pull/125


fix(accessibility): add missing alt property to img

Following accessibility principles, alt is a necessary attribute in the img tag as it helps screen readers.

thanks!!


[confcom] updating docs to include new genpolicy info

There is a new version of an included dependency, genpolicy, so we need to make a new version that includes those changes. The new version should get picked up as a part of the build process.

This checklist is used to make sure that common guidelines for a pull request are followed.
Related command
az confcom
General Guidelines

[x] Have you run azdev style <YOUR_EXT> locally? (pip install azdev required)
[x] Have you run python scripts/ci/test_index.py -q locally? (pip install wheel==0.30.0 required)

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update src/index.json automatically.
You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify src/index.json.

confcom

Please fix CI issues.

thanks for letting me know they weren't passing. The tests and styling are fixed now


bug: labware creator test pipettes list P1000 Multis

Overview
The Labware Test Protocol section of Labware Creator currently lists GEN1 and GEN2 P1000 multichannel pipettes as options in the drop down list.
Steps to reproduce

Load this existing labware definition into LC
Scroll to the labware test section and click the pipette dropdown

Current behavior
LC shows GEN1 P1000 Multi and GEN2 P1000 Multi as options in the pipette dropdown
Expected behavior
GEN1 P1000 Multi and GEN2 P1000 Multi should be removed as options from LC

New JIRA ticket: https://opentrons.atlassian.net/browse/RAUT-202


Add requireWakeEnv

For environment-based wake modules, requireWakeEnv can replace usage of .wakeignore

I think I learned my lession about trying to edit things on github directly, that was error prone haha


Update ROM spec to correct owner key flow

As identified by @varuns-nvidia in https://github.com/chipsalliance/caliptra-sw/issues/222#issuecomment-1530770019.
The ROM spec has a flowchart that includes details on owner key verification. These details should be updated:

If owner key fuses are not populated, the owner key is not compared with them.

Many devices will not have these fuses populated, to support circular economy.

In addition, we should consider allowing an edge where the image has no owner key, in which case no such verification is performed.

@mhatrevi should we change this for the 1.0 spec?

In addition, we should consider allowing an edge where the image has no owner key, in which case no such verification is performed.

Just noting that this edge was never implemented; the image must always be signed by an owner key.


iOS 13 TabBar position is invalid after resuming device from lock

Describe the bug
First of all I would like to thank you again for this library, I use it in my apps and it makes the user experience much more pleasurable.
I've been updating my app for iOS 13 and I've noticed an invalid behaviour of the Scrolling NavBar, TabBar to be exact.
If the device is locked and the NavBar/TabBar is collapsed, then its position is invalid, when the device wakes back up. The problem can be seen in the gif I've attached.
The problem appears when also presenting other VCs onto the VC that has the ScollingNavBar, such as when presenting a SideMenu, in my case.
To Reproduce
Steps to reproduce the behavior:

Make the TabBar collapse.
Lock the device/simulator
Unlock the device, the TabBar will be visible, but in the .collapsed state, so once you scroll up, the TabBar moves above its normal position.
(Tested on iPhone 11 Pro Max iOS 13 Simulator, which didn't seem to happen on iOS 12 and below after fixing #321 )

Expected behavior
The TabBar should either be collapsed when waking, or have the correct state so it does not go above its normal position
Screenshots
Here's the gif of the problem. gif
After taking a look at the source code, I believe that the scrollWithDelta function gets called before the didBecomeActive function, thus making the previousState param always .expanded before the didBecomeActive checks its value. All of these are in the ScrollingNavigationController class.
I'll do some more digging and report here if I find a fix, any ideas are appreciated

For now I worked around the issue by calling .showNavBar in my .applicationWillResignActive. However I'm seeing the misplaced TabBar issue on random occasions when scrolling too fast, or too slow, which is not something I can consistently reproduce.


[iOS] Audio is not playing in iOS 17.1.1 if a callkit is active

I am noticing that in iOS 17.1.1, I am unable to play any sounds during an active CallKit call - cannot play outgoing call sound, pick up, hang up sounds.
This is not observed on older versions of iOS or before entering a CallKit call - because everything works then. I've got two physical phones, one is an Iphone 15 Pro with latest iOS (17.1.1), the other one is an Iphone 11 with iOS 16.6.1, everything works perfectly on the 11, while on the 15, I can play sounds until I mark CallKit as having begun the call, then nothing, no sounds are played. Are there any work arounds/solutions to this issue or is this related to changes that Apple has made and there's nothing that can be done from the lib's end?
Expected behaviour
The audio from asset source is correctly played during an active callkit call.
Steps to reproduce
Play sound from asset
Observe sounds being played correctly
Start an active callkit call
Play sound from asset
Sounds are not being played
@hiennguyen92 Thank you very much for the work you have done, please help

Experiencing this as well, have you found any work around to this?
@Tim-byte-afk @hiennguyen92


Wrong kind of API reference in Bicep.

Onboard an Azure virtual machine, Template file for Bicep includes
    settings: {
      workspaceId: reference(WorkspaceResourceId, '@2021-12-01-preview').customerId
      azureResourceId: VmResourceId
      stopOnMultipleConnections: true
    }
    protectedSettings: {
      workspaceKey: listKeys(WorkspaceResourceId, '@2021-12-01-preview').primarySharedKey
    }

It should not have '@' before version.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 4776c9f5-e298-678b-c5f6-64c9ed0d082e
Version Independent ID: 223a92c8-5baa-0999-1ace-0c49b0071743
Content: Resource Manager template samples for VM insights - Azure Monitor
Content Source: articles/azure-monitor/vm/resource-manager-vminsights.md
Service: azure-monitor
Sub-service: virtual-machines
GitHub Login: @bwren
Microsoft Alias: bwren

@Jvekka Thanks for your comment! We will review and provide an update as appropriate.

@Jvekka We have verified and updated the document as required. The update will go live in less than 24 hours. Thanks again!


Less restrictive ROCm+GRU/LSTM fallback logic

A previous PR #17111 added some logic to use fallback implementations of GRU and LSTM on ROCm in situations where padded i/o is needed (since ROCm does not support padded i/o).
That logic turns out to be too restrictive - it chooses the fallback path in cases where it is not really needed, which may result in significant performance degradations.
This PR resolves the problem.

would it be acceptable to cherry-pick this one to the 2.12 branch as well?

Yes -- please open a PR against the r2.12 branch.

Thanks! I opened https://github.com/keras-team/keras/pull/17591 against r2.12.


Documented to disable UWM telemetry writer in disconnected envs

This is a manual cherry-pick of https://github.com/openshift/hypershift/pull/3380
/assign jparrill

/jira cherrypick OCPBUGS-22399

/test e2e-kubevirt-aws-ovn

/test e2e-kubevirt-aws-ovn

/retest

/retest

/override "Red Hat Trusted App Test / mce-24-txk4k / tektok-runner-enterprise-contract-mce-2.4"
/approve
/lgtm

/jira refresh

/retest-required

/lgtm

/jira refresh


eliminate most 'F' order warnings

in a memory efficient way and improve memory usage behavior in benchmarks.
Also proactively deletes some objects.
Found these to help with memory related reliability issues when running benchmarks.
One 'F' order warning persists, but I think it may be due to 1-d arrays being both 'C' and 'F' ordered and the check in cuml mnmg code not handling this correctly.

Looks nice!

Are there more commits on the way? I think the current shape looks good, assuming the run_tests.sh has got passed.

LGTM, a minor comment


more informative error in ProvideSharedPlugin

What kind of change does this PR introduce?
refactor error message. related to https://github.com/webpack/webpack/issues/15864

Did you add tests for your changes?
no

Does this PR introduce a breaking change?
no

What needs to be documented once your changes are merged?
nothing

Thanks