Android 13 : Camera/Gallery doesn't open

Bug Report
Msg: cannot open camera:  20
Problem
Camera doesn't open, Gallery doesn't open, for Android 13 emulator
Pointing API 33 of Android in the build.gradle
What is expected to happen?
Camera should open, Gallery should open
What does actually happen?
Nothing, display a bug message : Msg: cannot open camera:  20
E/Capacitor/Browser: Error binding to custom tabs service
Command or Code
    const options: CameraOptions = {
      quality: 100,
      destinationType: this.camera.DestinationType.DATA_URL,
      sourceType: this.camera.PictureSourceType.SAVEDPHOTOALBUM,
      correctOrientation: true,
      allowEdit: false,
      targetHeight: 1280,
      targetWidth: 1280,
    };

    this.camera.getPicture(options).then()....

Environment, Platform, Device

Ionic, Android 13
Workarounds
Use API 32

In my case picking an image from Library works, but taking an image with whichever options gives the error Illegal Argument Exception

Confirmed on pure cordova app
Cordova android v11.0.0
cordova camera plugin v6.0.0
Build Tools / Target SDK 33 on Android 13 emulator
Error code 20 is produced to the JS console
Native logcat:
07-16 00:23:38.096   575   914 V UserSystemPackageInstaller: dumpPackageWhitelistProblems(): using mode ENFORCE|IMPLICIT_WHITELIST|IMPLICIT_WHITELIST_SYSTEM
07-16 00:23:38.113   575   914 I ActivityTaskManager: START u0 {act=android.content.pm.action.REQUEST_PERMISSIONS pkg=com.google.android.permissioncontroller cmp=com.google.android.permissioncontroller/com.android.permissioncontroller.permission.ui.GrantPermissionsActivity (has extras)} from uid 10157
07-16 00:23:38.120  6334  6334 D CordovaActivity: Paused the activity.
07-16 00:23:38.121   575   917 W ActivityTaskManager: Tried to set launchTime (0) < mLastActivityLaunchTime (352665)
07-16 00:23:38.125   575   594 D OomAdjuster: Not killing cached processes
07-16 00:23:38.142   575   917 D CoreBackPreview: Window{e737f07 u0 com.google.android.permissioncontroller/com.android.permissioncontroller.permission.ui.GrantPermissionsActivity}: Setting back callback OnBackInvokedCallbackInfo{mCallback=android.window.IOnBackInvokedCallback$Stub$Proxy@c13665d, mPriority=0}
07-16 00:23:38.150   402   446 D goldfish-address-space: claimShared: Ask to claim region [0x3f683d000 0x3f6e40000]
07-16 00:23:38.160  6334  6334 D CordovaActivity: Resumed the activity.
07-16 00:23:38.161  6334  6334 I chromium: [INFO:CONSOLE(1)] "20", source:  (1)
07-16 00:23:38.173   575   917 D CoreBackPreview: Window{e737f07 u0 com.google.android.permissioncontroller/com.android.permissioncontroller.permission.ui.GrantPermissionsActivity}: Setting back callback null
07-16 00:23:38.173   575  3087 W InputManager-JNI: Input channel object 'e737f07 com.google.android.permissioncontroller/com.android.permissioncontroller.permission.ui.GrantPermissionsActivity (client)' was disposed without first being removed with the input manager!
07-16 00:23:38.189  7107  7107 I GoogleInputMethodService: GoogleInputMethodService.onFinishInput():3204
07-16 00:23:38.190  7107  7107 I GoogleInputMethodService: GoogleInputMethodService.updateDeviceLockedStatus():2100 checkRepeatedly = false, unlocked = true
07-16 00:23:38.190  7107  7107 I GoogleInputMethodService: GoogleInputMethodService.onStartInput():1903 onStartInput(EditorInfo{inputType=0x0(NULL) imeOptions=0x12000000 privateImeOptions=null actionName=UNSPECIFIED actionLabel=null actionId=0 initialSelStart=-1 initialSelEnd=-1 initialCapsMode=0x0 hintText=null label=null packageName=io.cordova.hellocordovagh797 fieldId=100 fieldName=null extras=null}, false)
07-16 00:23:38.190  7107  7107 I GoogleInputMethodService: GoogleInputMethodService.updateDeviceLockedStatus():2100 checkRepeatedly = true, unlocked = true

At a glance, it appears something to do with handling permissions has changed in Android 13, as error code 20 is a Permission Error Code...
https://github.com/apache/cordova-plugin-camera/blob/4608f8ef8027a5c7130a3479c2d37a5e391c70e8/src/android/CameraLauncher.java#L100

Can we have an update on this bug please?
I think Android 13 is gonna be released soon

breautek

I tried adding the permissions as said in the Android doc https://developer.android.com/about/versions/13/behavior-changes-13
Like this :
    <!-- Required only if your app targets Android 13. -->
    <!-- Declare one or more the following permissions only if your app needs
    to access data that's protected by them. -->
    <uses-permission android:name="android.permission.READ_MEDIA_IMAGES" />
    <uses-permission android:name="android.permission.READ_MEDIA_AUDIO" />
    <uses-permission android:name="android.permission.READ_MEDIA_VIDEO" />

    <!-- Required to maintain app compatibility. -->
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"
                     android:maxSdkVersion="32" />
    <application ...>
        ...
    </application>
</manifest>```

But it didn't change. "Cannot open camera : 20" , "Cannot open gallery : 20"

After investigating, the error comes from this method on CameraLauncher.java file :
    public void onRequestPermissionResult(int requestCode, String[] permissions,
                                          int[] grantResults) {
        for (int r : grantResults) {
            System.out.println("PHOTO DEBUG - " + r);
         /*   if (r == PackageManager.PERMISSION_DENIED) {
                this.callbackContext.sendPluginResult(new PluginResult(PluginResult.Status.ERROR, PERMISSION_DENIED_ERROR));
                System.out.println("PHOTO DEBUG - " + r);
                return;
            }
*/

        }
        switch (requestCode) {
            case TAKE_PIC_SEC:
                takePicture(this.destType, this.encodingType);
                break;
            case SAVE_TO_ALBUM_SEC:
                this.getImage(this.srcType, this.destType);
                break;
        }
    }

If I comment this section about comparing value of r and PackageManager.PERMISSION_DENIED value, camera and gallery work good.
I'm not Android Expert but i try to get explanations

Is there any work to actually fix this issue? The solution above me doesn't seem that fitting since it can crash the application if the user declines to give access to the camera/gallery. Is there a reason why all the grantResults are -1 for android 13?

hello..
any update for this issue.

Hi all,
is there any information on this issue and if/when it's planned to be fixed?
Regards

This pull request seems a try to fix this issue : https://github.com/apache/cordova-plugin-camera/pull/814
I tried to install it but got some compilation error and it does not solve the issue yet.

I am hoping for this bug to be fixed too.  It is producing strange results where the gallery refuses to open on Android 33 Emulator, but works fine on my Pixel 7 with Android 13.

Is there a reason why all the grantResults are -1 for android 13?

Because Google changed permission requirements with Android 13
https://developer.android.com/about/versions/13/behavior-changes-13?hl=en

Before Android 13 the permission requested was READ_EXTERNAL_STORAGE
With Android 13 and higher this permissions was divided in three: Images, Video and Audio. For this plugin we just need images, thus with Android 13 and higher we will need to request READ_MEDIA_IMAGES
It should be noted this just happens for new installations

if your app was previously granted the READ_EXTERNAL_STORAGE permission, then any requested READ_MEDIA_* permissions are granted automatically when upgrading

@breautek I was digging into the repo and READ_EXTERNAL_STORAGE just popups in  /src/android/CameraLauncher.java. I guess we should have some sort of global switch that replaces READ_EXTERNAL_STORAGE by READ_MEDIA_IMAGES when faced with Android 13 or higher. This will be backwards compatible because, as said if READ_EXTERNAL_STORAGE was already granted in Android 12 or lower and user upgraded to Android 13, READ_MEDIA_IMAGES is granted automatically.

@Houdhey now I have no access to my test PC, but instead of commenting that portion of code, can you kindly just make another test? Can you try to replace all the READ_EXTERNAL_STORAGE entries by READ_MEDIA_IMAGES in /src/android/CameraLauncher.java and see if it works? Pleeease ;)

Hello @jfoclpf
Yes, there is a similar Pull Request doing this : https://github.com/apache/cordova-plugin-camera/pull/814/files
Now the problem is fixed, I can't reproduce the bug.
Thank you for your investigation @jfoclpf , and thanks to all investigators. I can close

Just to clarify, I should install plugin like this
cordova plugin add https://github.com/apache/cordova-plugin-camera.git#pull/814/master

Just temporarily obviously, till it is integrated and deployed into NPM repository

Actually, at the time I opened this bug,  I was using this plugin with Capacitor 2.0 (I don't know if they totally replaced Cordova).
So my installation process was : npm install@ionic-native/camera && npm install cordova-plugin-camera
Now, I don't work anymore on this project, so I created a new Ionic Project, and installed same versions that was bugging at the time.
With this : npm install<EMAIL_ADDRESS>&& npm install<EMAIL_ADDRESS>And with this new project, Camera is working. But it uses Capacitor 4
I couldn't update Capacitor in my project, in the time I opened this bug. So if you still have this bug, I recommend you to upgrade latest version of Capacitor.
If you need more help, we can see this in a private conversation

Hi ,
I tried to use the PR but i still can't build :/  :

This is my setup :

Hi , I tried to use the PR but i still can't build 🤔 I got this error :
This is my setup :

Show your full AndroidManifest.xml file

@felicienfrancois
I tried your fork and yes i can build now but i still get the error '20' on android 13 phone.

@keva91 read this comment
Try using PR #814 as you did before and try to guess which plugin is creating the conflict, the one which uses WRITE_EXTERNAL_STORAGE without setting android:maxSdkVersion
Can you show us your list of plugins?

@keva91 thank you for your feedback, I have not submitted my fork as a PR yet because it deserve tests and adjustment on the required permissions at runtime, depending on case.
I personaly use it with success, in DATA_URI mode.
Can you tell me what is your usage DATA_URI, FILE ? Taking photo or picking from library ?
Also, you can try thoose different versions :

READ_MEDIA_IMAGES Always requested (Same as this PR)
https://github.com/felicienfrancois/cordova-plugin-camera#f53d3cb2ef5add8f7ba37503c4a10fc8e1337600

READ_MEDIA_IMAGES Never requested (The one I use)
https://github.com/felicienfrancois/cordova-plugin-camera#e6e49b85b59263fc5782e835160d01a8932e3690

READ_MEDIA_IMAGES requested only when picking from library (Head, i.e. the one you tried)
https://github.com/felicienfrancois/cordova-plugin-camera#efcd9190d98130d815b442b96e0c69fe17637908

As far as I understand, the READ_MEDIA_IMAGES should not be requested as it may not be granted and it is not necessary, at least in my case. But maybe it is needed when using FILE mode

Also, you can try thoose different versions :

READ_MEDIA_IMAGES Always requested (Same as PR Android 13 support #814 )
https://github.com/felicienfrancois/cordova-plugin-camera#f53d3cb2ef5add8f7ba37503c4a10fc8e1337600
android 13: ❌
android 11: ✅
android 10: ✅
READ_MEDIA_IMAGES Never requested (The one I use)
https://github.com/felicienfrancois/cordova-plugin-camera#e6e49b85b59263fc5782e835160d01a8932e3690
android 13: ✅
android 11: ✅
android 10: ✅
READ_MEDIA_IMAGES requested only when picking from library (Head, i.e. the one you tried)
https://github.com/felicienfrancois/cordova-plugin-camera#efcd9190d98130d815b442b96e0c69fe17637908
android 13: ❌
android 11: ✅
android 10: ✅

@keva91 ok, thank you. So PR #814 has definitelly an issue with READ_MEDIA_IMAGES. According to android doc this permission is required when accessing images on external storage or created by other apps so there may still be cases where it is needed. I just made another change on my fork (adding READ_MEDIA_IMAGES to manifest), if you want to test it: https://github.com/felicienfrancois/cordova-plugin-camera#7d66143ed55d817676c10a460c66938519d4b85b

It works too. And it also prompt for permissions which is better i think :)

@felicienfrancois regarding your repo and user permissions, should't your add min and max sdk versions to avoid conflicts? That is:
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" android:maxSdkVersion="32" />
<uses-permission android:name="android.permission.READ_MEDIA_IMAGES" android:minSdkVersion="33" />

I see @felicienfrancois , I tested now, although correct my suggestions they are more likely to create conflicts with other plugins.

@jfoclpf Yes. Specifying a min or max sdk version creates conflicts with other plugins, even some officail plugins.
It seems that not specifying any min or max sdk version does not create any issue (I think it is ignored by versions not supporting this permission), but it still need to be validated.
As a result, I would recommend to remove minSdkVersion and maxSdkVersion from the PR

you're right @felicienfrancois just tested now and the build fails due to conflicts with other plugins. But READ_MEDIA_IMAGES is totally new, you can add min sdk here, since no other plugins use this permission. The conflict arises from WRITE_EXTERNAL_STORAGE

@felicienfrancois just tested now with these configs and works fine
```xml
  <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
  <uses-permission android:name="android.permission.READ_MEDIA_IMAGES" android:minSdkVersion="33" />
  <uses-permission android:name="android.permission.READ_MEDIA_VIDEO" android:minSdkVersion="33" />

@jkainth that PR is working OK, you can install that PR, but you can use also @felicienfrancois repo, it's working just fine, I'm using it in two of my projects
github:felicienfrancois/cordova-plugin-camera

@jfoclpf  thanks. I might have to.
I'm using OneSignal that forces SDK version to be 33 - and coz of that my cameras are borked.
Will try this weekend.

For what it's worth, using https://github.com/felicienfrancois/cordova-plugin-camera#e6e49b85b59263fc5782e835160d01a8932e3690 solved my error 20 problem on API 33 as well.

Hi everyone,
I Had similar error, and I fixed it by installing cordova-plugin-android-permissions plugin. Try it.

Hi @AndrWeisR ,
I had similar issue how can I fix that?
Can you provide exact solution /steps to fix this

I think it should be a bug in api 33. It could be override by removing for
loop in below code
permissions,
                                          int[] grantResults) {
        for (int r : grantResults) {
            System.out.println("PHOTO DEBUG - " + r);
         /*   if (r == PackageManager.PERMISSION_DENIED) {
                this.callbackContext.sendPluginResult(new
PluginResult(PluginResult.Status.ERROR, PERMISSION_DENIED_ERROR));
                System.out.println("PHOTO DEBUG - " + r);
                return;
            }
*/

        }
        switch (requestCode) {
            case TAKE_PIC_SEC:
                takePicture(this.destType, this.encodingType);
                break;
            case SAVE_TO_ALBUM_SEC:
                this.getImage(this.srcType, this.destType);
                break;
        }
    }```

On Wed, 19 Jul, 2023, 10:45 pm vaibhav65577<EMAIL_ADDRESS>wrote:

> Hi @AndrWeisR <https://github.com/AndrWeisR> ,
> I had similar issue how can I fix that?
>
> Can you provide exact solution /steps to fix this
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/cordova-plugin-camera/issues/797#issuecomment-1642461847>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AEYQQJQSHOAHDMHHYQYSLZTXRAI25ANCNFSM52E6T4NQ>
> .
> You are receiving this because you commented.Message ID:
><EMAIL_ADDRESS>>

1 ionic cordova platform rm android
2 ionic cordova plugin rm cordova-plugin-camera
3 ionic cordova platform add<EMAIL_ADDRESS>4 cordova plugin add https://github.com/apache/cordova-plugin-camera.git
5 In project structure (Project) change
gradle plugin version : 7.1.2
gradle version :  7.5
6 In project structure (SDK Location) of Gradle Settings
Gradle Jdk :  corretto-17

use these steps and now you can access the camera

https://github.com/felicienfrancois/cordova-plugin-camera#e6e49b85b59263fc5782e835160d01a8932e3690 is not working for me and giving me below error inAndroid Studio while building the app
Android build failed: Caused by: java.lang.RuntimeException: Unable to determine Android SDK directory.

I have used this olution and it's work for me
private void requestGalleryPermission() {
try {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) {
// Android 13 and above
if (ContextCompat.checkSelfPermission(this, Manifest.permission.READ_MEDIA_IMAGES) != PackageManager.PERMISSION_GRANTED ||
ContextCompat.checkSelfPermission(this, Manifest.permission.READ_MEDIA_VIDEO) != PackageManager.PERMISSION_GRANTED ||
ContextCompat.checkSelfPermission(this, Manifest.permission.READ_MEDIA_AUDIO) != PackageManager.PERMISSION_GRANTED) {
                ActivityCompat.requestPermissions(this,
                        new String[]{Manifest.permission.READ_MEDIA_IMAGES,
                                Manifest.permission.READ_MEDIA_VIDEO,
                                Manifest.permission.READ_MEDIA_AUDIO},
                        201);
            } else {
                startGalleryIntent();
            }
        } else {
            // Below Android 13
            if (ContextCompat.checkSelfPermission(this, Manifest.permission.WRITE_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED ||
                    ContextCompat.checkSelfPermission(this, Manifest.permission.READ_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) {

                ActivityCompat.requestPermissions(this,
                        new String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE,
                                Manifest.permission.READ_EXTERNAL_STORAGE},
                        201);
            } else {
                startGalleryIntent();
            }
        }
    } catch (Exception e) {
        Log.e("Exception", "in permission----->" + e);
    }
}

private void startGalleryIntent() {
    Intent intent = new Intent(Intent.ACTION_PICK,
            MediaStore.Images.Media.EXTERNAL_CONTENT_URI);
    startActivityForResult(intent, RESULT_LOAD_IMG);
}

I have used this solution
private void requestGalleryPermission() {
try {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) {
// Android 13 and above
if (ContextCompat.checkSelfPermission(this, Manifest.permission.READ_MEDIA_IMAGES) != PackageManager.PERMISSION_GRANTED ||
ContextCompat.checkSelfPermission(this, Manifest.permission.READ_MEDIA_VIDEO) != PackageManager.PERMISSION_GRANTED ||
ContextCompat.checkSelfPermission(this, Manifest.permission.READ_MEDIA_AUDIO) != PackageManager.PERMISSION_GRANTED) {
                ActivityCompat.requestPermissions(this,
                        new String[]{Manifest.permission.READ_MEDIA_IMAGES,
                                Manifest.permission.READ_MEDIA_VIDEO,
                                Manifest.permission.READ_MEDIA_AUDIO},
                        201);
            } else {
                startGalleryIntent();
            }
        } else {
            // Below Android 13
            if (ContextCompat.checkSelfPermission(this, Manifest.permission.WRITE_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED ||
                    ContextCompat.checkSelfPermission(this, Manifest.permission.READ_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) {

                ActivityCompat.requestPermissions(this,
                        new String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE,
                                Manifest.permission.READ_EXTERNAL_STORAGE},
                        201);
            } else {
                startGalleryIntent();
            }
        }
    } catch (Exception e) {
        Log.e("Exception", "in permission----->" + e);
    }
}

private void startGalleryIntent() {
    Intent intent = new Intent(Intent.ACTION_PICK,
            MediaStore.Images.Media.EXTERNAL_CONTENT_URI);
    startActivityForResult(intent, RESULT_LOAD_IMG);
}


ionic-angular

There are now a few folk on board for building nx-ionic-angular support.
I was about to create a new repo, but before doing so I wanted to check with you guys if you are open to add ionic-angular support within your repo. I feel there is lots of overlap and this way all ionic stuff would be in one location.
We could start building in a fork of your repo. After we get to a certain stage you guys could asses if you want to add it to the main repo. What do you think?

Hey there @ntziolis, thanks for the feedback. I plan to add an ionic-angular plugin to this workspace in the near future. Now that I have had a good amount of experience with the ionic-react plugin, I believe that I can get the Angular equivalent running pretty quickly. I expect to have a beta ready within a couple weeks, and when I do I can let you know. That being said, feel free to work on a plugin of your own if you'd like, I won't be offended at all.

That sounds perfect. I just added you to a private gitter channel. Not sure if you have the bandwidth to give some pointers but obviously having done the react piece your input would be invaluable

Obviously any other channel would be fine with me as well just name your poison :)

The best place to get this kind of help from me and other plugin authors would be the Nrwl community Slack server. I try to pop in and out of there when I have time, and there are a lot of other smart people that are more than willing to help out.

@devinshoemaker Have you taken a look at the issue @ntziolis linked? my team wants an Ionic-Angular monorepo project as well so I would like to contribute to this:
https://github.com/nrwl/nx/issues/2633

@luchillo17 sorry for the late reply. I have looked at that yes. I have been really busy lately but have a good idea of how to implement an Ionic Angular plugin. Once I get an initial release out, or even a beta, I'll be more than happy to take contributions. I'll be sure to comment on this issue once I get to that point.

High guys sry for the radio silence but the timeline one the project we wanted to use this for was moved up significantly and there was no time to spend on getting a nx plugin up and running before working on it sadly. I'm still available to contribute on any work necessary to make this happen though.

I've got the Angular plugin started, but sadly, similar story from me. This is a nights and weekends project for me and I've recently changed jobs and am moving so that time has been cut into. I'll definitely post on here when a beta is available.

Any chance to add me as a contributor, don't care at all if the repo is messy right now we all know it is.

I likely won't take contributions on the Angular plugin until I have at least a beta ready. But in general, you can fork this repo and create a PR if you'd like to make contributions.

Just wanted to give an update on this. Ionic Angular has been delayed due to some fundamental changes that I'm making to the Ionic React and Capacitor plugins that will require major releases. I plan to pick Ionic Angular back up once that is complete.

@devinshoemaker really exited to see this in action. We will wait for it :)

Initial Ionic Angular support has been released. Thanks for everyone's patience.
https://nxtend.dev/blog/ionic-angular-1.0.0


Don't merge: How to store a custom attribute, read/write

This is just a request for review of a use case. I want to store a custom edge attribute in the Graph, and I want to use that attribute in a PathDetailsBuilder. Am I doing it right?
What I particularly would like to know:

Do I need a whole FlagEncoder to store one attribute? I am asking because the idea behind FlagEncoder really seems to be specifically vehicle type or mode of transport. (How do I know this? Because a broken vehicle icon appears in the web client.) Is there a different public API for what I am trying to do?

Things I do not require feedback about:

that the FlagEncoder and the PathDetailsBuilder are hacked into GraphHopperManaged
that you are not seeing where I actually set the attribute -- just assume my setter on my custom FlagEncoder is used at some point during preprocessing

Sorry, I still think this is a misunderstanding. I am storing a property in the flags. Internally. That I can store whatever I want externally, in my own data structures, is clear.
The thing that I am trying to store internally is the direction bit I am inventing here. I already found out that I need that, and it is, of course, a direction-dependent property. So my use case is: Store an internal, direction dependent property. This is what I am trying to do here.
Where are the setBool, setLong methods with which I can easily set a property? Do you mean those in AbstractFlagEncoder?

That I can store whatever I want externally, in my own data structures, is clear.

If you have something externally then this does not depend on the direction so you would need some bit like you have implemented.

So my use case is: Store an internal, direction dependent property. This is what I am trying to do here.

Then the procedure via two variables should be used. See reverseSpeedEncoder in Bike2WeightFlagEncoder. (You do not need an extra bit)

so how is that easier than writing my own method, which I can give a descriptive name?

Uhm, this is a really good question :)
They will be useful if you implement a Weighting that is used by multiple encoders that are not in the same inheritance hierarchy, e.g. to fetch max speed or similar.


Organize Charter modifications

At the annual meeting, the voting members have the opportunity to  vote on modifications of the MapLibre Charter. We should give people the option to suggest changes to the Charter well ahead of time and then organize a vote on the changes.

Align with Strategy: https://maplibre.org/about/


websocket connection missing

Latest reactUI doesn't start websocket to /pushlistener

/pushlistener ist missing for you?
ws.current = new WebSocket((baseURL.substring(0, baseURL.indexOf("//")).includes("https") ? "wss://" : "ws://") + urlSubstr + "/pushlistener?clientId=" + getClientId()
Is in the code, but I'll check tomorrow

I see the log
connecting WebSocket
but there's not request in the network tab of Chrome and also no log on server-side?
Also no other websocket message.

The problem is the clientId. If the id contains a #, it won't work.
You should encode the URL parameter...
Anyway, server now sends another id without # in it.

now encoding url parameter in websocket url, is this closable?

As we talked about, I've added '#' in HttpSessionHandler.java and with encoding it works, without it didn't.


Observatory: Include size in display of known Set implementations

It would be helpful for known Set implementations to have better display, including the size summary
We have
_List (6) {  ⊞  }
_ImmutableList (3) {  ⊞  }
_InternalLinkedHashMap (143) {  ⊞  }

but
_CompactLinkedHashSet {  ⊞  }

For things like _List, _ImmutableList and _InternalLinkedHashMap, these are implemented in the vm and we are able to include the length without invoking any dart code.  To add the same for non-vm classes, we would need to do a bit of evaluation, which could have side-effects.  We generally try to avoid evaluating except when requested by the user.
I'm not closing the bug though because this would be nice information to see in Observatory.  I'm thinking it over.


Can use with material-ui?

How could I use react-data-components with a table like this.

That is currently not possible unless you modify all the components and change the classes. In the future I would like to make it easier to change, but I first need to get rid of the Twitter Bootstrap dependency.

Actually an easy way to implement this would be to have the ability to add custom classNames. I do not think this would conflict with having Twitter Bootstrap in the mean time.
I'm currently using react-toolbox ( based on material design as well ), which uses css-modules to manage styling, and i'm having a hard time customizing the look of my table because of the inability to customize css classes.  I am somehow able to do it, but in a very hacky way.
You have .container, .row, .table and these are very global and general. I see they are based of Bootstrap, but they make it very hard to use with anything other than Bootstrap.
Could we propose a Pull Request with the ability to add custom classes ?


Small fix to the --var-file example

The file names were not matching

Hi - in this particular case it's intentional. There's a difference between the --var-file and --vars-file. This example is using --var-file which is saying "get the entire contents of 1.txt and set the access_key_id variable with it". As opposed to --vars-file, which is parsing a YAML file for a hash and creating variables from those keys/values.

Makes sense. I changed it, so that the files are now consistently have the txt extension in the example.


Support latest version of cmak

The latest version supports new versions of kafka
https://github.com/yahoo/CMAK/pull/871

Hello

CMAK didnt release a new version with this feature yet.
The chart needs a docker image to run pod in k8s
This chart doesnt build cmak image but uses this docker https://github.com/hleb-albau/kafka-manager-docker
Currently chart doesnt allow to override default images with custom ones

So, until CMAK releases official new version I tend not to use the latest code from master branch, and also corresponding docke image repository must be addressed.
If you want to try to build docker image from the master and try it with this chart, it is required to make change to support CMAK docker image overide via values file, so PR is welcomed.

Some updates
I started working on some internal changes to the project that will result in support of the latest CMAK version as one of the consequences.

@eshepelyuk Hi, when do you think we can use the latest operator to support CMAK <IP_ADDRESS>, since we are using kafka 3.1.0.

@eshepelyuk Hi, when do you think we can use the latest operator to support CMAK <IP_ADDRESS>, since we are using kafka 3.1.0.

i plan to release it smth about next week.

@eshepelyuk Thanks for the update. 👍


Port brazilian honda HR-V 2023

Splitted from #27915
Discord user: kotarou9692
DongleId:<PHONE_NUMBER>16e718
Checklist

[x] added entry to CarInfo in selfdrive/car/*/values.py and ran selfdrive/car/docs.py to generate new docs
[x] test route added to routes.py
[x] route with openpilot:
147613502316e718|2024-01-07--09-11-33 (Gabriel 2024)
147613502316e718|2024-01-07--10-12-31 (Alex Takeda 2023)
[x] route with stock system:
147613502316e718|2024-01-07--09-20-48 (Gabriel 2024)

video with me, Alex Takeda (Kotarou) and Gabriel in this routes: https://youtu.be/Q11veROVwF4?t=27m

Why is the Brazilian one special and require its own port?

Brazilian not require ALT_BRAKE
Enviado do meu iPhone
Em 10 de jan. de 2024, à(s) 00:26, Adeeb Shihadeh @.***> escreveu:

Why is the Brazilian one special and require its own port?
—
Reply to this email directly, view it on GitHubhttps://github.com/commaai/openpilot/pull/30235#issuecomment-1884132144, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AP23P76G37W4JHMUHKLJM5TYNYC6NAVCNFSM6AAAAAA57NSJGWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBUGEZTEMJUGQ.
You are receiving this because you authored the thread.Message ID: @.***>

Can we defect that?

Can we defect that?

From #27915 with:

Any news regarding this port? @gaurovgiri


Very slow on find and replace compared with VS Code

Prerequisites

[ X] Put an X between the brackets on this line if you have done all of the following:

Reproduced the problem in Safe Mode: http://flight-manual.atom.io/hacking-atom/sections/debugging/#using-safe-mode
Followed all applicable steps in the debugging guide: http://flight-manual.atom.io/hacking-atom/sections/debugging/
Checked the FAQs on the message board for common solutions: https://discuss.atom.io/c/faq
Checked that your issue isn't already filed: https://github.com/issues?utf8=✓&q=is%3Aissue+user%3Aatom
Checked that there is not already an Atom package that provides the described functionality: https://atom.io/packages

Description
It is ver-very slow on find and replace on a file where the items to replace are pretty much.
For example, I need to replace ~343200 #012 string into \n (with regex) or even without regex, replace #012 => ..
It never finished (at least I didn't waited and killed the process ).
You can use the attached file. 012-1.7MB.txt.zip
I'm filing this issue as when I do the same in VS Code it takes just a couple of seconds.
Steps to Reproduce
In find and Replace window do the following:
Find => #012,
Replace with \n (with .*)
or
Replace with . (raw text)
Expected behavior: Complete in a couple of seconds
Actual behavior: very long time / killed the process
Reproduces how often: always on the attached file
Versions
You can get this information from copy and pasting the output of atom --version and apm --version from the command line. Also, please include the OS and what version of the OS you're running.
atom --version
Atom    : 1.22.1
Electron: 1.6.15
Chrome  : 56.0.2924.87
Node    : 7.4.0

apm --version
apm  1.18.8
npm  3.10.10
node 6.9.5 x64
python 2.7.10
git 2.15.0

Additional Information
foe comparison VS Code version:
/code --version
1.18.0
dcee2202709a4f223185514b9275aa4229841aa7

Hardware: MacBook Pro (13-inch, 2017)
Any additional information, configuration or data that might be necessary to reproduce the issue.
See the attached file to reproduce.

Thanks for taking the time to contribute!
We noticed that this is a duplicate of https://github.com/atom/find-and-replace/issues/653. You may want to subscribe there for updates.
Because we treat our issues list as the Atom team's backlog, we close duplicates to focus our work and not have to touch the same chunk of code for the same reason multiple times. This is also why we may mark something as duplicate that isn't an exact duplicate but is closely related.
For information on how to use GitHub's search feature to find out if something is a duplicate before filing, see the How Can I Contribute? section of the Atom CONTRIBUTING guide.


Update namespace definition

This is a Feature Request
Good first issue!
What would you like to be added
Revised definition of namespace.
Why is this needed
Suggested during Kubecon 21 talk that namespace definition is 5 years old. Page in question
Comments
Slides from Kubecon talk. See slides 5 and 6.
Related issue - https://github.com/kubernetes/website/issues/29307.

/good first issue

https://github.com/kubernetes/website/pull/30086 and https://github.com/kubernetes/website/pull/30086/files both aim to cover this.

/close
Fixed in #30088


Another Possible IgnoreCollectionOrder issue

Hi team,
First of all thank you for the great library, it helps a lot.
Could you please verify why the library finds two sets of objects different while i am expecting they are equal. Could it be a bug or misconfiguration on my side?
`
    public class TestContent
    {
        public string Title { get; set; }
        public IList<TestLocalisation> Localisations { get; set; }
    }
    public class TestLocalisation
    {
        public TestLanguage Language { get; set; }
        // here might be another List<> which should also ignore order
    }
    public class TestLanguage
    {
        public string Code { get; set; }
    }

    var expected = new List<TestContent>
    {
        new TestContent
        {
            Title = "1",
            Localisations = new List<TestLocalisation>
            {
                new TestLocalisation
                {
                    Language = new TestLanguage {Code = "en"}
                },
                new TestLocalisation
                {
                    Language = new TestLanguage {Code = "de"}
                }
            }
        }
    };

    var actual = new List<TestContent>
    {
        new TestContent
        {
            Title = "1",
            Localisations = new List<TestLocalisation>
            {
                new TestLocalisation
                {
                    Language = new TestLanguage {Code = "de"}
                },
                new TestLocalisation
                {
                    Language = new TestLanguage {Code = "en"}
                }
            }
        }
    };

    var config = new ComparisonConfig
    {
        CompareFields = false,
        ComparePrivateFields = false,
        CompareStaticFields = false,
        CompareStaticProperties = false,
        MaxStructDepth = 100,
        IgnoreCollectionOrder = true
    };
    var comparer = new CompareLogic(config);
    var comparisonResult = comparer.Compare(expected, actual);

`
The library returns the following:
Begin Differences (1 differences):
Types [String,String], Item Expected[Title:1].Localisations[Language:TestLanguage].Language.Code != Actual[Title:1].Localisations[Language:TestLanguage].Language.Code, Values (en,de)
End Differences (Maximum of 1 differences shown).
Tried configuring CollectionMatchingSpec but with no luck.
Thanks in advance

dgranov,
You are not going to be able to do a collection matching spec unless you have some sort of id in the TestLanguage.
Thanks,
Greg


Deploy UAT

Yeah. I didn't squash it into QA because it had commits from upstream in it, and now would not make sense to squash into a release branch


scrollEnabled not working in react-native-web project

hi @billoosijok , thanks for your great work.
the lib working very well in ios, but when i try to use it in react-native-web project.
i found it when open the interactive card will cause the scrollEnabled not working.
some code copy from your example#CardsInScrollView , and i hard code scrollEnabled={false}
<ScrollView
  contentContainerStyle={styles.scrollViewConentContainer}
  style={styles.scrollView}
  scrollEnabled={false}
>
   {this.state.cards}
 </ScrollView>

but:

@billoosijok yes.
i think the parent scrollview is override by the content scrollview.


Update to logstash-core-plugin-api

Ref: https://github.com/elastic/logstash/issues/5269

@ph are you sure we want to down grade the version from 2.0.3 to 1.0.0?

The specs fail because it is using the old Event API
I am creating a PR to fix that.
@ph - do you want to close this in favour of mine? I will set the gemspec plugin api to 2 as done in this PR but increase the version to 3.0.0. WDYT?

@guyboertje Yes closing this one.


Allow experimenting with the printResolution AppOption when printing with the built-in Firefox version

As have already been stated multiple times, simply increasing the printing resolution may have undesirable effects on both memory usage and general performance. Hence why PR #10854 did not add a preference, and only exposed AppOption by default in GENERIC builds for now.
However, considering how differently printing works in the built-in Firefox version (with mozPrintCallback) compared to the general default viewer, any testing done in the latter case might not be completely relevant to the first (and most important) case of the Firefox PDF Viewer.
Note that considering the implementation of AppOptions.get, this patch will be safe and should allow experimenting with printResolution in all builds of the default viewer[1]. By not, however, having printResolution appear in AppOptions for either the MOZCENTRAL or CHROMIUM build targets, there should be no indication of official support for now.
Furthermore, it shouldn't be a preference at this point in time (or even at all), since that makes it too easy for users to change it permanently[2] and possible "break" printing.

[1] By running PDFViewerApplicationOptions.get('printResolution', /* value here */); in the console after the viewer loads.
[2] I've seen Firefox bugs, filed in Bugzilla, where users modified e.g. preferences manually in about:config and then some time later (maybe months) wondered why something was suddenly broken. In those cases, trying to work out that a preference change was the culprit can take time/effort.

/botio-linux preview

Looks like a good idea; thanks!


Fix memory leak when copying strings

StringRef doesn't deallocate it's char pointer so that needs to be done after the external command has been run.
Also return the actual exit status of the external command.
rdar://55195607

Yeah, let's fix this for all commands in a different PR. I think the const char ** then shouldn't be converted to std::string at all but directly to StringRef.


Autopipeline support

Autopipline provides you a way for fetching multiple keys using a single command, batching happens transparently behind the scenes without bothering the caller.
Get´s are piled up until the next loop iteration. Once the next loop iteration is reached all get´s are transmitted using the
same Memcached operation.
The total number of get´s transmited within the same operation are limited, if more get´s are provided within the previous loop iteration they will be chunked on different operations.
Behind the scenes the connection pool provided by the client is still used, connections will be used if they are free, or new ones will be created if there is still room. Otherwise each batch will need to wait until a connection is released.
Autopipeline can boost up the throughput x2/x3.
How it is used
Example of how the Autopiline object is being used
client = await emcache.create_client([emcache.MemcachedHostAddress('localhost', 11211)])
autopipeline = client.autopipeline()
await autopipeline.get(b"key")

Is up to the client to decide how many Autopipeline instances are being created, but the recommendation is create one and use it during the whole life cycle of the application, as it was a generic Emcache Client.
Autopiline can be created for returning also the flags and the Cas value of the keys, by providing the return_flags and the return_cas parameters as True, by default they are set to False, during the creation of the object. Hence all calls to get will return Items with the flags and the Cas attributes populated.
What´s missing:

Documentation

Implements this feature https://github.com/pfreixes/emcache/issues/46

@vangheem thanks for the review.
Yes agree that autopipeline is not the best one, and it´s to coupled to the Redis world .... what about autobatching?
I was considering using the normal get and getspath, but my concern was on the restriction on having all get´s of a batch being semantically equal, so all of them would need to ask for return_flags or ask for the casvalue. The PR tried to address this by providing a factory which forces the user to choose how the keys are fetched behind the scenes, treating all of them equal later on.
But it comes with some cognitive burden and a more fragmented API, also raises some misalignments. For example,  using the traditional gets from the client - when cas is required  - but using the get when the autopipeline is used but coming from a Autopipeline object that was built using the cas=True argument. Weird.
Im about to change my mind again and go for you proposal, considering that the number of combinations - different kind of batch operations  - in terms of extra arguments is limited, I would say 4 at most. So each time that the user would execute a get or a gets the operation would be routed to one or the other depending the arguments provided. So the unique real change would be how the Emcache client would be constructed, which could provide an autobatching parameter for enabling this feature.
WDYT?

So each time that the user would execute a get or a gets the operation would be routed to one or the other depending the arguments provided

Yes, seems reasonable. Most of the time the user will be doing the same type of operations each time.

So the unique real change would be how the Emcache client would be constructed, which could provide an autobatching parameter for enabling this feature

I think that's a good idea. I like it.


Update the GitHub "build" action (workflow) to include code coverage report by using https://codecov.io

Add maven coverage reporter plugin
Update GH workflow to get codecov reports

Depends on #2


Adding the ability to set up Automated Builds

We built a module that depends on this lz4 module, but due to the utilities used by our consumers several do not easily have access to install windows-build-tools and build the package themselves locally. As a result, it makes more sense to publish pre-built binaries (as referred to in #67, I work with Mike who posted that issue).
In here I've added an appveyor.yml file which can be used to set up an appveyor job that pre-builds windows versions of the library, and set up the package.json to utilize node-pre-gyp to check and download assets from releases in github.
I've also updated the usage of the lz4 library to fix warnings of deprecated functions, and updated a couple of the dependencies. In my testing this works and the resulting tar.gz file is able to be downloaded and installed by node-pre-gyp. In order to use this, an appveyor.yml project would need to be created and connected to this repo, along with putting a github token (encrypted via appveyor similar to the way travis works) into the yml file in order to allow it to deploy and publish the binaries. Would these be things you'd be willing to do after merging this?
I've also condensed the compiled binary into a single lz4.node file since that was easier for specifying the artifacts and telling node-pre-gyp how to package things.

Hello,
I need to look at this as I am not familiar with appveyor.
I do agree though that this would be very useful.
This may take a bit of time on my side as this is a hobby project.
Thanks.

I'd also like to see support for prebuilt binaries in lz4. However, I don't think appveyor is needed either. I believe the standard practice for doing prebuilt binaries for Node packages is to use https://www.npmjs.com/package/prebuild-install (for downloading the binaries when a user installs)+ https://www.npmjs.com/package/prebuild (for creating the binaries and uploading them). Here is an example of how I added it another package: https://github.com/Venemo/node-lmdb/pull/148/files


adds ruby 2.4.0 support

Thanks for your pull request!
However, this duplicates https://github.com/ruby-grape/grape/pull/1543

@namusyaka … please 😄 … didn't see it, but this one didn't drop support for ruby < 2.4.0 😉

@LeFnord My description is bad. In fact, Fixnum cannot be dropped, it may be used in Ruby < 2.4.0.
Please take a look.

@namusyaka … yeap I know

closed in favour of https://github.com/ruby-grape/grape/pull/1543


Split Vietnam

I have thought a bit more about it. Regardless of whether the project use backzone or not, Northern Vietnam will still be in a different timezone than Southern Vietnam in timezone database. (Southern Vietnam is always Asia/Ho_Chi_Minh, while Northern Vietnam is either Asia/Hanoi in backzone or Asia/Bangkok in the standard dataset.)
Thus, as the timezone-bondary-buioder's objective is to create a geographical correspondence for the tz database, it seems appropriate to split Vietnam to reflect this difference in tz database.
As for exactly how to split it into different zone, I guess https://en.wikipedia.org/wiki/Vietnamese_Demilitarized_Zone would be a reasonable starting point.

Your comment in #65 is quite relevant:

The tricky thing here is Asia/Hanoi which have more data in the backzone tab and not just simply a link

I'm actually leaning towards using Asia/Bangkok to keep this library compatible with the current tz database. This would be the first area where a timezone identifier from another country will be used, but as you mention, it's probably the right thing to do.
As for geographic boundaries, it looks like the Quang Tri Province was the northernmost province of South Vietnam, so that could be the boundary.

Here's a link to the relation in OSM.


Write a Swift program to convert hours to weeks

Description
Write a Swift program to convert hours to weeks
1 week = 7 days = 168 hours
Input  : 840
Output : 5

How to contribute

Save the solution in ConvertHoursToWeeks.swift file
Add ConvertHoursToWeeks.swift file in convert-hours-to-weeks folder

!assign


Dialog Update

Expands upon the dialog system.
Changes:

DialogueBoxPsych (rewrite, new events)
TypedAlphabet (allow to get notified when the displayed text changes)
ReflectionFunctions (allow to call functions with no parameter)
DialogueEditorState

More details
You can create custom UIs/behaviors with these events:
function onDialogueStart()
    -- Called when a dialogue starts
end

function onCharacterMove(reason, character, startingPos, offsetPos, elapsed)
    -- Called whenever a dialogue character needs to move
    -- reason: The reason of the movement (enter or exit)
    -- character: Name of the character being moved
    -- startingPos: Starting position of the character (from DialogueCharacter.hx/DialogueCharacterFile.position)
    -- offsetPos: ??? (default to -600)
    -- returns: Should the character move according to the default system?
end

function onDialogueEvent(name, args)
    -- Called when a dialog event is triggered
end

function onDialogueConfirm()
    -- Called to see if the player can confirm the current dialog
    -- returns: Should the player be able to confirm the dialogue?
end

function onDialogueTextUpdate(text)
    -- Called when the dialogue text is updated
    -- text: Current text being displayed
end

wait, why you requesting me to review this, i dont even have any words for the-

Yeah my bad! Clicked something I shouldn't have ^^"


Undefined fileUri

Hello.
I just want to thank you for a great example you gave us and it helped me a lot on the project im currently busy with.
However, i seem to have an undefined fileUri when i have selected an image, but i do get the Uri part.
Regards.

Hi @Thabangtg,
Thank you for your interest in NativeScript.
The returned items from the selected images will be from Assets type, which do not have fileUri property. You could use getImage(), which will return imageSource and you could save it on the device with saveToFile(<file_path>, <file_type>). As a result you will be able to use the new created file path. For further help you could review the sample example from here.
Let me know, whether this helps.


Code archeology notes

Notes on reconstructing history
Because there's no record of this project's code repo, I thought it would be nevertheless be good to have some sort of history.
I downloaded the old sdists from https://pypi.org/project/termcolor/#history to reconstruct it.
This repo had started from an initial commit of the latest 1.1.0 sdist.
I followed https://stackoverflow.com/a/59548348/724176 to add new commits before the current repo's initial commit.
For each commit, I took the sdist contents, and made a single commit:
Version X.Y.Z

Co-authored-by: Konstantin Lepa<EMAIL_ADDRESS>
and committed like gc --date="2011-01-13T18:45:33.979856Z", using the upload_time_iso_8601 of the release from https://pypi.org/pypi/termcolor/json

Because this rewrote the Git history of this repo, if you have a clone, I suggest deleting and re-cloning. I'm doing this now to get a clean starting point to continue and won't be rewriting history once this is sorted :)

First pass, I had this locally:

But this on GitHub:

Seems there's also a difference between "author date" and "commit date":
https://github.com/orgs/community/discussions/22695#discussioncomment-3237743
So I re-filtered with their suggestion:
git filter-branch --env-filter 'export GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE"'
git push -f

And it now looks good locally:

And on GitHub:

...

Created tags from those commits like:
git tag -a 0.1   -m "Release 0.1"   9fa47eadc3ce76e3dc15115d420e9575baab6601
git tag -a 0.1.1 -m "Release 0.1.1" ba42abd77069a26224460e74baf9ba567f1facd8
git tag -a 0.1.2 -m "Release 0.1.2" 404c8ef7ba3ea89f08740a0c1a7e4beec988cd27
git tag -a 0.2   -m "Release 0.2"   2fcb6da9c402a6528af560d4c7b9193784f603e4
git tag -a 1.0.0 -m "Release 1.0.0" f152f6d40f8be53af513a4919ead8de915aaf79c
git tag -a 1.0.1 -m "Release 1.0.1" 85c8fa64c51dc5906231c23e5625643184e08f22
git tag -a 1.1.0 -m "Release 1.1.0" fce9d9a6fd97b659282b3e893500fe80cbeac6e2

Updated dates using https://stackoverflow.com/a/25939259/724176 :

```sh
git tag -l | while read -r tag ; do COMMIT_HASH=$(git rev-list -1 $tag) && GIT_COMMITTER_DATE="$(git show $COMMIT_HASH --format=%aD | head -1)" git tag -a -f $tag -m"$tag" $COMMIT_HASH ; done
git push --tags

And created GitHub releases from those tags.

Okay, all done here!
Ping @CozyDoomer, @gururajrkatti, @jdkloe, @jdufresne, @mpenning, @tmetzl:
As a one-off, history has been rewritten in this repo, you may want to delete/re-create your clones or branches.
Thanks!

Interesting read, good job!


Python database tools

The Python chapter contains a section "Database Interface" that doesn't seem helpful. As far as I know, what we use from Python to access databases is typically SQLAlchemy, as @bouweandela suggested. I have used this personally in the past, but it's been a long time and I don't feel confident enough on this topic to write more than just "SQLAlchemy good".
Does anyone with more recent experience want to write something about this? @HannoSpreeuw maybe? Needn't be more than a single sentence, but if you have good tips ("ALWAYS rtfm first" or "NEVER rtfm, it's horrible, try this instead" or...), that would be great as well.
Also, if anyone disagrees with the above assessment (that the currently listed database interface tools are essentially useless to our typical usecases), please correct me!

Btw, for the historians among you, it seems like the original contributor of this list is lost in the mists of time ;) https://github.com/NLeSC/guide/blame/52cf8b2fe97f571cff745f3d8a60d915deb5b4dd/software/language_specific_info.md

Sorry, I do have some experience with SQLAlchemy from PADRE, but that was mainly about getting it to work after a Python 2 --> 3 conversion. What you need is an opinion about whether SA is the preferred Object Relational Mapper. I truly don't know.

How about @suvayu?
I think @LourensVeen has the theoretical background on ORMs, but is not familiar with SA, or at least that's the last I heard... Perhaps still enough to make a sensible recommendation on this?

I have extensive but outdated experience with PostgreSQL, and I'm using sqlite3 from Python directly through its SQL-based API. I could probably do some research and write a reasonable recommendation, but I'm also swamped again, so at the earliest in 2024...

I'm guessing this is the section you are referring to?  My experience with both databases and SQLAlchemy is only at the level "I know enough to debug, and spot obvious mistakes".
That said, one (somewhat) low effort approach could be someone moderately experienced go through related awesome lists and select a few recommendations.
Another point would be, not to stop at just interfaces, but mention other related tools like db documentation, query optimisers/explainers, database modelling hints, and convenience utilities like DBCli.
I'll be back next week.  I can help as long as someone else also joins me.

@LourensVeen so what you are saying is that sqlite3 actually is a useful one to keep there?
@suvayu ok, this is an interesting direction. It's sounding more like a chapter on databases, though. Perhaps actually something like that is better than listing random libraries that nobody may ever use? I also remember from my brief time doing MySQL + SQLAlchemy that we ended up spending a lot of time just doing raw SQL and indeed interfacing directly with the MySQL CLI tool or e.g. through good old phpMyAdmin.
I would say then that the database interfaces list can simply be removed from this chapter. If people know the general way of working with databases and they need either an ORM or some direct interface between the DB they chose and the language they use, then Google will deliver it to them. Ok, maybe it makes sense to list popular ORMs in such a chapter, among which would be SQLAlchemy, but that would be the extent of the Python-/language-specific needs then.
Agree? Disagree? Things I missed?

@wrvhage What does the Data SIG think, is this something that makes sense?
Note btw that new Guide contribution guidelines will soon be online, so please don't write a database chapter yet ;)

sqlite is excellent, and definitely recommended if you have a use case that fits it. For lots of data or complex queries or if you need a separate server, then PostgreSQL is better, but it does have a separate server so it makes your application context more complex. And then of course there are column stores and NoSQL databases that may or may not be a better match than SQL. SQL often gets overlooked these days, which is a shame because there are definitely still lots of use cases where it's just the right solution.

My impression is for research use cases, these days duckdb is probably a good choice, since the data is usually static and it's more about analytics.

I came across this thread by chance, and thought I added my two cents from one of my previous research projects. There, we've been using sqlite extensively and over time built a database of almost 1TB  with many tables and relatively complex queries.
What we have been struggling is aggregation queries -- they are quite slow in sqlite for this size of db. I recently experimented with DuckDB, and I could connect to the sqlite existing database and perform some aggregation queries 10x faster than through sqlite.
In short, I think combining DuckDB and sqlite could be promising in many use-cases. I guess @suvayu  is right that the more static the data, the more useful is DuckDB. But if you are building a db from scratch or adding a lot of tables (which at least in the social sciences is not uncommon),  I think sqlite is better because it can persist row indices which I think DuckDB cannot. For analytics, it seems one can then still use DuckDB, but perhaps one could explore the combination of the two a bit more.

On Thu, 25 Jan, 2024, 11:10 f-hafner, @.***> wrote:

In short, I think combining DuckDB and sqlite could be promising in many
use-cases. I guess @suvayu https://github.com/suvayu is right that the
more static the data, the more useful is DuckDB. But if you are building a
db from scratch or adding a lot of tables (which at least in the social
sciences is not uncommon), I think sqlite is better because it can persist
row indices which I think DuckDB cannot. For analytics, it seems one can
then still use DuckDB, but perhaps one could explore the combination of the
two a bit more.

Interesting point about row indices. AFAIU, DuckDB doesn't support indices
because it doesn't need it. Since it's an OLAP db, the query execution
engine is designed to query any column without relying on indices like
traditional DBs (this is based on a Q&A with one of the devs at a PyData
talk). It depends a lot more on column types instead.
@f-hafner, if you are up for it, we could do a test with your example db. I
think instead of just connecting to the sqlite db from duckdb, we can
import the data and compare query performance.
Unfortunately all public comparisons are too synthetic to base general
recommendations for real world use.
--
Suvayu

I think I tried to replace the entire sqlite db with DuckDB in the past, but then abandoned the idea because I could not persist the row indices, but perhaps I should have tried a little more.
so yes, @suvayu , let's have a look together. I can't share the data publicly, but can give you access or prepare an extract that I can share.

With the datasets chapter merged and with the above addition (whether or not it is merged doesn't really matter, we will decide in the PR), this issue can be closed, thanks all for the discussion! For new additions, or if you want to revisit some particular points from this thread, feel free to open a new dedicated issue (this one has become a bit tangled).


[Bug]: I have the option to setup Biometrics even if I don't have a pin set

Describe the bug
I've tried Biometrics setup but I'm asked for a PIN code (even though I don't have any settings). This option should be invisible as long as no pin code is set, or I should be asked to set a pin code first.
Reproduce

Go to 'Setting'
Click on 'Security and Privacy'
See error

Screenshots / Recording

Operating system
Graphene OS and iOS
Bitkit version
v54
Log output
No response

There is "Change Pin Code" as well, even if I don't have a pin set

Ok... Need  a coffee. closing it


Getting A New ApplicationDbContext

Hi All,
DbContext (and hence the default ApplicationDbContext DbContext that comes with an ASP.NET 5 template web application) is not thread safe. There are times when one wants to use Parallel.ForEach() and within it's lambda new up a db context (using a using block) so that you can parallelize a set of row-by-row database updates.
I've been trying to figure this out. I even tried injecting IServiceProvider into my constructor, but calling GetService on it keeps giving me the same instance of ApplicationDbContext.
I need a way to create new instances of ApplicationDbContext. Any help would be greatly appreciated.
Thanks,
-Brian

Sounds like a question for EF?

I'm not sure that that's true. A plain old DbContext can definitely still be created using a using block. I thin this question should be fielded by whoever knows about the ApplicationDbContext that's a part of the ASP.NET web application template, mixed with someone who knows about DI in ASP.NET 5/Core...

@divega @HaoK this seems to be a question around this code: https://github.com/aspnet/Templates/blob/dev/src/Rules/StarterWeb/IndividualAuth/Startup.cs#L45-L46

It does sound like an EF question to me :smile:

I need a way to create new instances of ApplicationDbContext

The AddDbContext<TContext>() method adds TContext as a scoped service, so if you resolved it in the same request from DI you will always get the same instance, and that is by design.
Although I am not sure it is a good idea to parallelize updates like you are describing, we do recognize that there are valid scenario for having multiple instance of a DbContext to be used in the same request, and this has been discussed in the EF repro.
With RC1 bits and template this is a bit confusing and difficult, but for RC2 we have made a number of changes that will enable simply creating a new instance of the application's DbContext and passing the right DbContextOptions

Thanks for your response @divega . I guess I'm assuming that this is more an ASP.NET concern than an EF one because I was guessing that the implementation details of ApplicationDbContext (extending IdentityDbContext) and how it is all created by the template is not 'owned' by the EF team. Does the EF team own that? The way IdentityDbContext is implemented it hides the ability to create a new instance of ApplicationDbContext directly (at least that's what I think I'm seeing). So I was considering this a concern of the ASP.NET team because of that.
At the end of the day I just need help from the right team. Also, if you know for sure that my scenario is one that can be solved once we get to RC2, I'm cool with waiting for that.
Finally, I get your point about not wanting to willy nilly do things with parallelism and whatnot. But I'm confident that used sparingly and appropriately it's a tool in the toolbox that one should have when one needs. In the old world where we would "new up" db contexts via a using block, updates done to batches of rows via parallel.foreach could be done 10x to 20x faster than single threading.
Thanks for helping/listening.
-Brian Eriksen

I think what you see in the default template and ApplicationDbContext is mostly usage of patterns provided by EF, e.g. the AddDbContext<TContext>() method is defined in EF.
I actually hit the comment button too soon. Here is some more information:

The ability to create multiple instances of a `DbContext in the same request is discussed here: https://github.com/aspnet/EntityFramework/issues/4441.

The changes we made post RC1 which simplify the patterns to create new instances are discussed here:
https://github.com/aspnet/EntityFramework/issues/4668.

I am actually now thinking that https://github.com/aspnet/EntityFramework/issues/4441 can now be closed because you can now just use new and pass the appropriate dependencies on the constructor whenever you need a new instance.

I am going to go ahead and close this issue because I am confident that I have responded to your question, but feel free to reactivate or to start a new thread in the EF repo.


SP sweep solves different GPs than the same SP not using sweeps

I solved my simple flight profile code with a sweep of cruise altitudes ('sweep', np.linspace(20000,40000,4)). I then solved it with cruise altitude set to 40000 and got different answers. The cost during the sweep was 9.43e+04 and during the no sweep solve it was 1.026e+05. I'm not sure what's going on but it seems like there's a substitutions problem during the sweep?
I ran a diff on the constraints of the final GP solve for each of the two SPs and this was the output. The majority of this is expected since the altitude build up is different in the two problems, but the first diff seems wrong to me. It results from the constraint htoc <= dhClimb2 + alt10k
if htoc (varkey 'h_{toc}') is truly set to 40,000 the constraint should simplify to 30000 <= dhClimb2 which doesn't appear to happen in the code with the sweep.
This problem manifests for all of the solves in the sweep with the exception of the first solve where htoc is 20000 so the constrain in question should actually be 110000 <= dhClimb2`, which is what the constraint is in the last solve of the sweep. Hence my theory this could be a substitution problem.
45c45
<         3e+04 <= dh_\{climb2\}_CommercialAircraft\

---
>         1e+04 <= dh_\{climb2\}_CommercialAircraft\
60,61c60,61
< ho_Atmosphere, CommercialAircraft*A_\{fuse\}_CommercialAircraft*C_\{d_fuse\}_Commercho_Atmosphere, CommercialAircraft*C_\{L_\{Climb1\}\}_Climb1, CommercialAircraft_(0,)**2*K_CommercialAircraft*S_CommercialAircraft*VClimb1_Climb1, CommercialAircraftho_Atmosphere, CommercialAircraft*C_\{d_wc1\}_Climb1, CommercialAircraft_(0,)*S_CommercialAircraft*VClimb1_Climb1, CommercialAircraft_(0,)**2\
< ho_Atmosphere, CommercialAircraft*A_\{fuse\}_CommercialAircraft*C_\{d_fuse\}_Commercho_Atmosphere, CommercialAircraft*C_\{L_\{Climb1\}\}_Climb1, CommercialAircraft_(1,)**2*K_CommercialAircraft*S_CommercialAircraft*VClimb1_Climb1, CommercialAircraftho_Atmosphere, CommercialAircraft*C_\{d_wc1\}_Climb1, CommercialAircraft_(1,)*S_CommercialAircraft*VClimb1_Climb1, CommercialAircraft_(1,)**2\

---
> ho_Atmosphere, CommercialAircraft*C_\{L_\{Climb1\}\}_Climb1, CommercialAircraft_(0,)**2*K_CommercialAircraft*S_CommercialAircraft*VClimb1_Climb1, CommercialAircraft_(0,ho_Atmosphere, CommercialAircraft*C_\{d_wc1\}_Climb1, CommercialAircraft_(0,)*S_CommercialAircraft*VClimb1_Climb1, CommercialAircraft_(0,)**2\
> ho_Atmosphere, CommercialAircraft*C_\{L_\{Climb1\}\}_Climb1, CommercialAircraft_(1,)**2*K_CommercialAircraft*S_CommercialAircraft*VClimb1_Climb1, CommercialAircraft_(1,ho_Atmosphere, CommercialAircraft*C_\{d_wc1\}_Climb1, CommercialAircraft_(1,)*S_CommercialAircraft*VClimb1_Climb1, CommercialAircraft_(1,)**2\
103,104c103,104
< ho_Atmosphere, CommercialAircraft*A_\{fuse\}_CommercialAircraft*C_\{d_fuse\}_Commercho_Atmosphere, CommercialAircraft*C_\{L_\{Climb2\}\}_Climb2, CommercialAircraft_(0,)**2*K_CommercialAircraft*S_CommercialAircraft*VClimb2_Climb2, CommercialAircraftho_Atmosphere, CommercialAircraft*C_\{d_wc2\}_Climb2, CommercialAircraft_(0,)*S_CommercialAircraft*VClimb2_Climb2, CommercialAircraft_(0,)**2\
< ho_Atmosphere, CommercialAircraft*A_\{fuse\}_CommercialAircraft*C_\{d_fuse\}_Commercho_Atmosphere, CommercialAircraft*C_\{L_\{Climb2\}\}_Climb2, CommercialAircraft_(1,)**2*K_CommercialAircraft*S_CommercialAircraft*VClimb2_Climb2, CommercialAircraftho_Atmosphere, CommercialAircraft*C_\{d_wc2\}_Climb2, CommercialAircraft_(1,)*S_CommercialAircraft*VClimb2_Climb2, CommercialAircraft_(1,)**2\

---
> ho_Atmosphere, CommercialAircraft*C_\{L_\{Climb2\}\}_Climb2, CommercialAircraft_(0,)**2*K_CommercialAircraft*S_CommercialAircraft*VClimb2_Climb2, CommercialAircraft_(0,ho_Atmosphere, CommercialAircraft*C_\{d_wc2\}_Climb2, CommercialAircraft_(0,)*S_CommercialAircraft*VClimb2_Climb2, CommercialAircraft_(0,)**2\
> ho_Atmosphere, CommercialAircraft*C_\{L_\{Climb2\}\}_Climb2, CommercialAircraft_(1,)**2*K_CommercialAircraft*S_CommercialAircraft*VClimb2_Climb2, CommercialAircraft_(1,ho_Atmosphere, CommercialAircraft*C_\{d_wc2\}_Climb2, CommercialAircraft_(1,)*S_CommercialAircraft*VClimb2_Climb2, CommercialAircraft_(1,)**2\
140,141c140,141
< ho_Atmosphere, CommercialAircraft*A_\{fuse\}_CommercialAircraft*C_\{d_fuse\}_Commercho_Atmosphere, CommercialAircraft*C_\{d_wcr2\}_Cruise2, Cruise2, CommercialAircraft_(0,)*S_CommercialAircraft*VCruise2_Cruise2, Cruise2, CommercialAircraft_(0,)**ho_Atmosphere, CommercialAircraft**-1*K_CommercialAircraft*S_CommercialAircraft**-1*VCruise2_Cruise2, Cruise2, CommercialAircraft_(0,)**-2*W_\{avgCruise2\}_CommercialAircraft_(0,)**2\
< ho_Atmosphere, CommercialAircraft*A_\{fuse\}_CommercialAircraft*C_\{d_fuse\}_Commercho_Atmosphere, CommercialAircraft*C_\{d_wcr2\}_Cruise2, Cruise2, CommercialAircraft_(1,)*S_CommercialAircraft*VCruise2_Cruise2, Cruise2, CommercialAircraft_(1,)**ho_Atmosphere, CommercialAircraft**-1*K_CommercialAircraft*S_CommercialAircraft**-1*VCruise2_Cruise2, Cruise2, CommercialAircraft_(1,)**-2*W_\{avgCruise2\}_CommercialAircraft_(1,)**2\

---
> ho_Atmosphere, CommercialAircraft*C_\{d_wcr2\}_Cruise2, Cruise2, CommercialAircraft_(0,)*S_CommercialAircraft*VCruise2_Cruise2, Cruise2, CommercialAircraft_(0,)**2 + ho_Atmosphere, CommercialAircraft**-1*K_CommercialAircraft*S_CommercialAircraft**-1*VCruise2_Cruise2, Cruise2, CommercialAircraft_(0,)**-2*W_\{avgCruise2\}_CommercialAircraft_(0,)**2\
> ho_Atmosphere, CommercialAircraft*C_\{d_wcr2\}_Cruise2, Cruise2, CommercialAircraft_(1,)*S_CommercialAircraft*VCruise2_Cruise2, Cruise2, CommercialAircraft_(1,)**2 + ho_Atmosphere, CommercialAircraft**-1*K_CommercialAircraft*S_CommercialAircraft**-1*VCruise2_Cruise2, Cruise2, CommercialAircraft_(1,)**-2*W_\{avgCruise2\}_CommercialAircraft_(1,)**2\
179c179
<         T_\{sl\}_CommercialAircraft = 1.29*L_\{atm\}_CommercialAircraft**0.17*T_\{atm\}_Atmosphere, CommercialAircraft**0.83*h_Atmosphere, CommercialAircraft**0.17\

---
>         T_\{sl\}_CommercialAircraft = 1.3*L_\{atm\}_CommercialAircraft**0.24*T_\{atm\}_Atmosphere, CommercialAircraft**0.76*h_Atmosphere, CommercialAircraft**0.24\

the code that reproduces this is Commercial_Flight_Profile_Simple.py at  https://github.com/hoburg/gpkit-models/pull/73/commits/c2ea2e44fa16b70fac72534465b6d803540fe576 It's currently set to do the sweep.

@mayork, could you confirm that this still happens with the latest release?

Could not reproduce


Fix parsing of the 2nd field of a pid stat field

The 2nd parameter on a /proc//stat file is the filename of the executable. This may potentially have spaces in the filename. So the algorithm was changed to search for the ending parentheses.

This is related to https://github.com/RuntimeTools/appmetrics/issues/258

Anyone going to have a chance to review this by chance?

@beans sorry for the delay, most of the team has been out on vacation so just getting back on top of al the issues.  The change looks fine but we need you to sign a  CLA that we are in the process of setting up for this git repo

@baens Can you fill our the CLA linked from here please https://github.com/RuntimeTools/omr-agentcore/blob/master/CONTRIBUTING.md

@tobespc Vacation, how horrible :) No worries, just wanted to ping and make sure it didn't get lost anywhere.
So, once I fill out where should I send it? I'm guessing I need to fax it somewhere?

@baens can you scan it as a pdf and send it to<EMAIL_ADDRESS>please

JUST found it, sorry about that

no worries :-)

@tobespc Finally sent it, let me know if you need anything else from me.

Thanks @baens , got the CLA now, looks fine.

LGTM

Awesome thanks @tobespc any idea when a release will be cut with this in it?


Fixed #33766 -- Resolve FilteredRelation.condition at referencing time.

A lot of the complexity that relates to FilteredRelation.condition resolving came from the fact that it was referencing the relation it was filtering against and not the filtered relation itself.
For example, given
qs.annotate(
    adventure_books=FilteredRelation('books, condition=Q(books__category="adventure"))
)

Translating the condition to be Q(adventure_books__category="adventure")  once the query has knowledge about adventure_books allows for the removal of the reuse_with_filtered_relation convoluted logic that fooled join resolving into resolving books to adventure_books.

The full diff of the SQL change for the filtered_relation tests
--- test_exclude_relation_with_join (filtered_relation.tests.FilteredRelationTests):postgresql:01ae9d4ca9
+++ test_exclude_relation_with_join (filtered_relation.tests.FilteredRelationTests):postgresql:d7b1aba90a
@@ -4,5 +4,6 @@
                 "filtered_relation_author"."object_id"
 FROM "filtered_relation_author"
 INNER JOIN "filtered_relation_book" book_alice ON ("filtered_relation_author"."id" = book_alice."author_id"
-                                                   AND (NOT (UPPER(book_alice."title"::text) LIKE UPPER(%s))))
+                                                   AND (NOT (UPPER(book_alice."title"::text) LIKE UPPER(%s)
+                                                             AND book_alice."title"::text IS NOT NULL)))
 WHERE book_alice."id" IS NOT NULL

The test_exclude_relation_with_join seems more adequate to apply negation against handle null fields but it doesn't have an impact here (since Book.title.null is False) though so there might be something at play in terms of field resolving but we should add a test to make sure filtered conditions do perform adequate null handling.
--- test_multiple_times (filtered_relation.tests.FilteredRelationTests):postgresql:01ae9d4ca9
+++ test_multiple_times (filtered_relation.tests.FilteredRelationTests):postgresql:d7b1aba90a
@@ -5,7 +5,5 @@
 FROM "filtered_relation_author"
 INNER JOIN "filtered_relation_book" book_title_alice ON ("filtered_relation_author"."id" = book_title_alice."author_id"
                                                          AND (UPPER(book_title_alice."title"::text) LIKE UPPER(%s)))
-INNER JOIN "filtered_relation_book" T3 ON ("filtered_relation_author"."id" = T3."author_id"
-                                           AND (UPPER(T3."title"::text) LIKE UPPER(%s)))
 WHERE (book_title_alice."id" IS NOT NULL
-       AND T3."id" IS NOT NULL)
+       AND book_title_alice."id" IS NOT NULL)

This demonstrate that this patch changes the semantic of FilteredRelation so it's reuse between filter call. That was something hinted by Anssi as a likely desirable property (https://github.com/django/django/pull/7560#issuecomment-280312114) to opt-out of the filter().filter() behaviour when dealing with multi-valued relationships and have it be reused. Likely not something we want to change here though.
--- test_nested_m2m_filtered (filtered_relation.tests.FilteredRelationTests):postgresql:01ae9d4ca9
+++ test_nested_m2m_filtered (filtered_relation.tests.FilteredRelationTests):postgresql:d7b1aba90a
@@ -5,5 +5,7 @@
 LEFT OUTER JOIN "filtered_relation_author_favorite_books" ON ("filtered_relation_author"."id" = "filtered_relation_author_favorite_books"."author_id")
 LEFT OUTER JOIN "filtered_relation_book" T4 ON ("filtered_relation_author_favorite_books"."book_id" = T4."id"
                                                 AND (UPPER(T4."title"::text) LIKE UPPER(%s)))
+LEFT OUTER JOIN "filtered_relation_book" T5 ON ("filtered_relation_author_favorite_books"."book_id" = T5."id"
+                                                AND (UPPER(T5."title"::text) LIKE UPPER(%s)))
 ORDER BY "filtered_relation_book"."title" ASC,
-         T4."title" ASC
+         T5."title" ASC

This one is a bug, the filtered relation alias should be reuse.
--- test_with_m2m (filtered_relation.tests.FilteredRelationTests):postgresql:01ae9d4ca9
+++ test_with_m2m (filtered_relation.tests.FilteredRelationTests):postgresql:d7b1aba90a
@@ -5,5 +5,5 @@
 FROM "filtered_relation_author"
 INNER JOIN "filtered_relation_author_favorite_books" ON ("filtered_relation_author"."id" = "filtered_relation_author_favorite_books"."author_id")
 INNER JOIN "filtered_relation_book" favorite_books_written_by_jane ON ("filtered_relation_author_favorite_books"."book_id" = favorite_books_written_by_jane."id"
-                                                                       AND ("filtered_relation_author_favorite_books"."book_id" IN (%s)))
+                                                                       AND (favorite_books_written_by_jane."id" IN (%s)))
 WHERE favorite_books_written_by_jane."id" IS NOT NULL

This one is equivalent as "filtered_relation_author_favorite_books"."book_id" = favorite_books_written_by_jane."id" per the JOIN condition.

PR should be ready for review. The SQL diff for the queries and filtered_relations tests is
--- test_exclude_relation_with_join (filtered_relation.tests.FilteredRelationTests):postgresql:fb535e0a90
+++ test_exclude_relation_with_join (filtered_relation.tests.FilteredRelationTests):postgresql:ec481b0258
@@ -4,5 +4,6 @@
                 "filtered_relation_author"."object_id"
 FROM "filtered_relation_author"
 INNER JOIN "filtered_relation_book" book_alice ON ("filtered_relation_author"."id" = book_alice."author_id"
-                                                   AND (NOT (UPPER(book_alice."title"::text) LIKE UPPER(%s))))
+                                                   AND (NOT (UPPER(book_alice."title"::text) LIKE UPPER(%s)
+                                                             AND book_alice."title"::text IS NOT NULL)))
 WHERE book_alice."id" IS NOT NULL

The reason for the addition of this superflous AND book_alice."title"::text IS NOT NULL statement is that since filtered relation conditions are now resolved on first references instead of at the last minute the filter(book_alice__isnull=False) is not applied yet (it's the book_alice resferences that triggers the condition recolsing) so the join is still LEFT OUTER when it reaches the logic in charge of determining if an IS NOT NULL should be added.
The query remains the same so I don't think it's a huge deal if this gets added as it will only happen under rare circumstances (negated, demoted via IS NOT NULL filtering).
--- test_with_m2m (filtered_relation.tests.FilteredRelationTests):postgresql:fb535e0a90
+++ test_with_m2m (filtered_relation.tests.FilteredRelationTests):postgresql:ec481b0258
@@ -5,5 +5,5 @@
 FROM "filtered_relation_author"
 INNER JOIN "filtered_relation_author_favorite_books" ON ("filtered_relation_author"."id" = "filtered_relation_author_favorite_books"."author_id")
 INNER JOIN "filtered_relation_book" favorite_books_written_by_jane ON ("filtered_relation_author_favorite_books"."book_id" = favorite_books_written_by_jane."id"
-                                                                       AND ("filtered_relation_author_favorite_books"."book_id" IN (%s)))
+                                                                       AND (favorite_books_written_by_jane."id" IN (%s)))
 WHERE favorite_books_written_by_jane."id" IS NOT NULL

This one is equivalent as "filtered_relation_author_favorite_books"."book_id" = favorite_books_written_by_jane."id" per the JOIN condition.

@charettes Thanks :heart:  ticket-33929 and ticket-34229 have been closed as duplicates of this issue, so we should add tests for them.

@felixxm I pushed some tests for both tickets but the MTI case is really complex to solve (hence the failing test)
Any objections to merging the first to commits and re-opening ticket-34229 to focus on the MTI case and using the third commit as a regression test?

@charettes Looks great at first glance, thanks :star: I hope you don't work so late because of me :wink: :bed:

@felixxm I pushed some tests for both tickets but the MTI case is really complex to solve (hence the failing test)
Any objections to merging the first to commits and re-opening ticket-34229 to focus on the MTI case and using the third commit as a regression test?

No :+1: Sounds good.

@felixxm ticket-33929 is covered by this test. Aggregation has nothing to do with it, it was a variant of ticket-33766 it's just that the rhs was spawning JOIN against another FilteredRelation
https://github.com/django/django/blob/965917cd2616eec479722b575b6666c910363df3/tests/filtered_relation/tests.py#L811-L827
I pushed another commit that also removes the unnecessary build_filtered_relation_q method which was a copy-pasta of _add_q with disabled join type updates.
To summarize, first commit solves the issue for ticket-33766 and ticket-33929, second and third perform cleanups, fourth is a demonstration of the variant that warrants re-opening ticket-34229. LMK if that doesn't make sense.

@charettes Marvelous :clap: :star2: Thanks :1st_place_medal:

I added test for FilteredRelation with self reference, see comment.

I added test for FilteredRelation with self reference

Thanks, I missed that one!


Admin access and database usage

Hi o1298098! May I also please have Admin access? email<EMAIL_ADDRESS>Also, my understanding is you are using MySql instead of Firestore. Any particular reason why you switched?
Thanks.

@omeedterainsights , already set。
Firestore is great, but I use a free plan, it is easy to exceed the limit 😭

Thanks and great job, best movie app I've seen!


linter does not pick up console.logs

Somewhere along the way we must have disabled that check, because I just checked in a console.log with no complaints!

I just tested this out and it seems to be working the same as always (console.log disallowed, console.warn and console.error are OK).
I added a console.log statement and then ran npm run eslint:
$ npm run eslint

><EMAIL_ADDRESS>eslint /Users/chris/Dev/influxdb/ui
> eslint '{src,cypress}**/*.{ts,tsx}'

/Users/chris/Dev/influxdb/ui/src/Setup.tsx
  47:5  error  Unexpected console statement  no-console

✖ 1 problem (1 error, 0 warnings)

Actually now I'm seeing this again. Will investigate.


Add IES support in the inspector

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

Snapshot stored with reference name:
refs/pull/15953/merge
Test environment:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15953/merge/index.html
To test a playground add it to the URL, for example:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15953/merge/index.html#WGZLGJ#4600
Links to test babylon tools with this snapshot:
https://playground.babylonjs.com/?snapshot=refs/pull/15953/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/15953/merge
https://gui.babylonjs.com/?snapshot=refs/pull/15953/merge
https://nme.babylonjs.com/?snapshot=refs/pull/15953/merge
To test the snapshot in the playground with a playground ID add it after the snapshot query string:
https://playground.babylonjs.com/?snapshot=refs/pull/15953/merge#BCU1XR#0

Visualization tests for WebGPU (Experimental)
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15953/merge/testResults/webgpuplaywright/index.html

WebGL2 visualization test reporter:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15953/merge/testResults/webgl2playwright/index.html

Visualization tests for WebGL 1 have failed. If some tests failed because the snapshots do not match, the report can be found at
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15953/merge/testResults/webgl1/index.html
If tests were successful afterwards, this report might not be available anymore.

Waiting for the other one to get merged


[DOCS-1606] Build/test action adjustments

Attempting to adjust our build/test github action once again. That is, only this check:

Changes
Instead of building on push, it should build on changes to PRs:

opened  (does not apply to draft PRs)
marked ready_for_review
synchronized : commits were pushed to the branch. This means that a push will still trigger a build, but only if you have an open PR.

Any PRs marked as “draft” will not be built. The build kicks off once you mark it “ready for review”.

Testing
Tested this in a bunch of situations:

No PR = no build
Open draft PR = no build
Push to draft PR = no build
Create PR from a branch with multiple commits: build/test runs on the latest commit only
Push a bunch of commits to an open PR, not in draft mode: build/test runs on the latest commit only

Am I missing anything?
Previews
N/A - though if you click on the green checkmark next to the last commit in this PR, you'll notice a skipped build. This was skipped automatically when the PR was in draft mode.

Happy to approve after we discuss, but the one thing I see as worth thinking about before we take this route completely is considering the case where we're reorganizing content/TOC/CSS/<anything_else_we_want_to_look_at_in_draft> and want a netlify preview for other folks to take a look at a draft/WIP. I guess we could manually kick off a build for that case? And when the whole team has netlify access that won't be the potential issue that it is atm ...

Happy to approve after we discuss, but the one thing I see as worth thinking about before we take this route completely is considering the case where we're reorganizing content/TOC/CSS/<anything_else_we_want_to_look_at_in_draft> and want a netlify preview for other folks to take a look at a draft/WIP. I guess we could manually kick off a build for that case? And when the whole team has netlify access that won't be the potential issue that it is atm ...

Replied on slack, but in the interest of having a paper trail: the change in this PR is specific to our build/test GH action, which includes a linter and link checker. This PR doesn't affect Netlify at all; Netlify will continue to build on drafts and WIPs, it just won’t build branches without PRs.


Sparse matrix integer support

The current Cupy sparse matrix initialization does not support integer matrices as in the case of Scipy. Was integer support left out intentionally from the sparse matrix routines?

@anaruse Am I right that cuSPARSE used to support only float, double, float complex, and double complex, but now with the new Generic API it also supports char/int8_t and int/int32_t?
https://docs.nvidia.com/cuda/cusparse/index.html#cusparse-generic-enum-data-types

Yes, cuSPARSE generic API also supports the integer data types. However, note that the combinations of supported data types vary from function to function.
cusparseSpMV(): https://docs.nvidia.com/cuda/cusparse/index.html#cusparse-generic-function-spmv

It sounds like it would be the first step would be to allow support for additionally types during matrix construction. Would it also be good to include a type check within each function to warn about unexpected behavior if the matrices are different formats?

@dloney I wonder if you have done some tests using cuSPARSE's API directly for its int8/int32 supports and compare the performance with, say, float32? The limited support in cuSPARSE seems nontrivial and so worrying to me, because we'd need to set up a dtype guard to every single function. I wanna know if it's worth the effort.
Also, what would be the use cases you have in mind for integer sparse matrices? I understand having better compatibility with SciPy is always a good thing, but not if the underlying library is lacking some critical pieces...I imagine if we don't set up the guard right, some users would come and complain "why is there a type conversion happening at this line?" And we'd have countless branches to be covered in our tests.
Just my two cents 😅 The PFN team may have already looked into this, so you can definitely wait for their feedbacks!


Yoda should go away after first account created

The partial authorization Yoda has seems confusing maybe it should go away after the first admin account is created?

Moved to https://github.com/Zonama/emuyoda/projects/1


Recursive Objects Support

Does API Blueprint (and Apiary.io as well) support self-referenced structures?
Currently I need to describe tree-alike structure and I'm doing it like this:
    ## Category (object)
    + categoryId: `321` (number) - category identifier
    + categoryName: `Super Category` (string) - category title
    + childCategory (optional, array[Category]) - array of child categories

And I get the notice saying: Base type 'Category' circularly referencing itself.

Hey, The API Blueprint does not support self-references in MSON. At least for now.

Hey! Thank you for the answer.

+1

We are exploring the possibility of allowing circular references – the use of incomplete name type in value definition. Will keep you updated.

This is now working using the latest drafter prerelease 2.3.0-pre.1


Update full set

ref: #25
Source mapping was updated. Need to run updated script.

https://github.com/markdown-it/markdown-it-emoji/commit/f1bc7217e9eb4d954c145faf316718e263778e34


Missing info on TreatDiscoveredPartnersAsUnverified

This page is missing details on the "TreatDiscoveredPartnersAsUnverified" property on the Get-CsTenantFederationConfiguration information.
Does Teams ignore this? Was this only for Skype for Business?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 59900927-5e5f-77a4-04be-e05a91330fca
Version Independent ID: 1e4d6b87-f57c-fa45-1630-3c32b821d93c
Content: Manage external access (federation) - Microsoft Teams
Content Source: Teams/manage-external-access.md
Service: msteams
GitHub Login: @MikePlumleyMSFT
Microsoft Alias: mikeplum

hi @lindspea , I think we need msft to confirm the intended behavior and then add the response to this article. Thanks!

@danmyhre @lindspea
Here's what I heard back from a few members of the team:

From Set-CsTenantFederationConfiguration,
-TreatDiscoveredPartnersAsUnverified
When set to True, messages sent from discovered partners are considered unverified. That means that those messages will be delivered only if they were sent from a person who is on the recipient's Contacts list. The default value is False ($False).
I would not expect Teams itself to care about this, but SFBO of course does. And federated interop between onprem and Teams goes thru SFBO, at least for now. So for any Teams org that is doing federation with pure onprem SfB orgs or onprem users in hybrid orgs, this setting would be honored specifically in those cases, if the Teams org sets it.  In those scenarios it would provide the behavior above. Of course, for federation between Teams to Teams, as well as when we complete federation via Teams Hybrid Edge without any dependency on GFS, then I assume this setting would no longer be honored, unless Teams went out of its way to honor it.  If I recall, you own federation, so I’m hoping you can clarify whether Teams honors it.  Btw, if Teams does not and is not planning to honor this, I would probably not recommend talking up this setting otherwise you might have a perceived take back once GFS-less federation lands.

I think Teams ignores this option even though the communication goes via SfBO infrastructure.  The enforcement of this is on the pool and it is using the contact list and I don’t think it is applied for Teams-only users or won’t be applied in the very near future as we make them pool-less.

Ah, so if it’s enforced on pool, rather than GFS edge, this would go away earlier. Currently about 40% of all users already bypass user pools for routing so they would not honor this. Eventually, all users will bypass user pools.

Thanks!

@lindspea - do you think we can close this issue now? I don't expect more feedback from the team.....
Thanks!

@tiburd Hi, I think the documentation still needs updating. If this is a SFB only setting, that definitely needs to be called out that it does nothing for Teams which is where we are all headed.


[Languages]An image manipulation guide using OpenCV python library

Proposed title of article
An image manipulation guide using OpenCV python library
Proposed article introduction
OpenCV is a python library that is used in the manipulation of images. Images can be manipulated using various techniques and methods in order to get different displays of one original image. For example, an image can be manipulated using several functions that are provided in the python library mentioned above and displayed as either bigger, smaller, tall and many more other styles as compared to the original image.
Therefore, all this is going to be catered for in this article step by step beginning with the installation of the required modules to the display of an output.
Key takeaways

How to perform dilation
How to perform a log transformation
How to perform a linear transformation

Article quality
Content quality in the article will be high and well understood
References
N/A

@neheMaish There seems to be many articles covering this topic how will it be unique?

@WanjaMIKE , what I am going to major on is that there are some functionalities  that are not available in other modules like wand but are present in OpenCV

@WanjaMIKE  still , when we take it as a comparison with other libraries , it will be a very long article in this case

@neheMaish
Not sure I'm following your comment.
This article will be unique because it will focus on "other modules like wand"??

Closing due to inactivity


Design changes for explore rates and form explainer

Design changes for explore rates
Changes

Modifed code to fix issues related to GHE/OAH/Buying-a-House/issues/19

Testing

Enable the OAH_JOURNEY flag.
Visit http://localhost:8000/owning-a-home/explore-rates/#about

Screenshots
Desktop:

Mobile:

Checklist

[ ] PR has an informative and human-readable title
[ ] Changes are limited to a single goal (no scope creep)
[ ] Code can be automatically merged (no conflicts)
[ ] Code follows the standards laid out in the development playbook
[ ] Passes all existing automated tests
[ ] Any change in functionality is tested
[ ] New functions are documented (with a description, list of inputs, and expected output)
[ ] Placeholder code is flagged / future todos are captured in comments
[ ] Visually tested in supported browsers and devices (see checklist below :point_down:)
[ ] Project documentation has been updated
[ ] Reviewers requested with the Reviewers tool :arrow_right:

Testing checklist
Browsers

[ ] Chrome
[ ] Firefox
[ ] Safari
[ ] Internet Explorer 8, 9, 10, and 11
[ ] Edge
[ ] iOS Safari
[ ] Chrome for Android

Accessibility

[ ] Keyboard friendly
[ ] Screen reader friendly

Other

[ ] Is useable without CSS
[ ] Is useable without JS
[ ] Flexible from small to large screens
[ ] No linting errors or warnings
[ ] JavaScript tests are passing

@anselmbradford, give this another look when you get a chance. I've added changes for the form-explainer.

LGTM 👍


automatically cast values to Array instances when setting the appropriate attribute's value

If you are looking for a more succinct change to mine, to solve this, the other option could be to use Descriptors which enforce the assignment for you, which would alleviate the need to have to check during every attribute set and be isolated to just those attributes which are effected.

Rejected PR in favor of PR #12


slam tool inclusion

Current status: We agreed to keep grid conversion and metadata conversion as a separate postprocessor as it will be used by a number of different applications such as the RES.
Work with Anke and Dasha to understand implications for fetch-data task to load LFRic based UMified data

The SLAM tool looks to become part of the RNS, so data should already be processed by the time it gets to us. It should just be a case of selecting the correct filename, which currently include the suffix _struct.


Grammarly support for Tina Rich Text field

When using Grammarly, it appears that I can fix spelling issues but the text does not change.
https://www.loom.com/share/dbf95c92e7224ecd85d0acde48891c04
I thought this worked in the past, but perhaps it was only supported by other Tina fields.

Having the same issue, is there any update for this?

Having the same issue, is there any update for this?

Nothing as far as I'm aware.