Feature: provide interface to the detailed faces information in database

The database contains much detailed info about the people/faces in the photo

I would love an export option for this information as well. Perhaps they could be added as EXIF keywords in the photos themselves?

@AaronVanGeffen the names of the people in the images are already included in the export for both JSON and XMP (stored in XMP:Subject and IPTC:PersonInImage).  I am also working on an ExifTool interface to directly write this and other metadata to the image upon export.  This issue (though written very vaguely!) is referring to the detailed information such as the actual face regions and characteristics (e.g. left eye closed, right eye closed, has smile, etc.) that Photos stores about the faces. (In Photos 5, much of this information is in ZDETECTEDFACE table) It would be useful to have access to this info, for example, for experimenting with face detection or machine learning.

Need to add height, width, orientation to PhotoInfo.  (Should add size while I'm at it)

See this note on orientation

Photos 4
CREATE TABLE RKFace (modelId integer primary key autoincrement, uuid varchar, isInTrash integer, personId integer,
hasBeenSynced integer, adjustmentUuid varchar, imageModelId integer, sourceWidth integer, sourceHeight integer, centerX
decimal, centerY decimal, size decimal, leftEyeX decimal, leftEyeY decimal, rightEyeX decimal, rightEyeY decimal, mouthX
decimal, mouthY decimal, hidden integer, manual integer, hasSmile integer, blurScore decimal, isLeftEyeClosed integer,
isRightEyeClosed integer, nameSource integer, poseRoll decimal, poseYaw decimal, posePitch decimal, faceAlgorithmVersion
integer, expressionConfidence decimal, expressionType1 integer, expressionType2 integer, expressionType3 integer,
expressionScore1 decimal, expressionScore2 decimal, expressionScore3 decimal, qualityMeasure integer,
clusterSequenceNumber integer, syncPropertyModifiedDate timestamp, faceGroupId integer,
confirmedFaceCropGenerationState integer, faceType integer, trainingType integer, cloudNameSource integer, personUuid
varchar);

Photos 5
CREATE TABLE ZDETECTEDFACE (
  Z_PK INTEGER PRIMARY KEY, Z_ENT INTEGER,
  Z_OPT INTEGER, ZAGETYPE INTEGER, ZASSETVISIBLE INTEGER,
  ZBALDTYPE INTEGER, ZCLOUDLOCALSTATE INTEGER,
  ZCLOUDNAMESOURCE INTEGER, ZCLUSTERSEQUENCENUMBER INTEGER,
  ZCONFIRMEDFACECROPGENERATIONSTATE INTEGER,
  ZEYEMAKEUPTYPE INTEGER, ZEYESSTATE INTEGER,
  ZFACEALGORITHMVERSION INTEGER, ZFACIALHAIRTYPE INTEGER,
  ZGENDERTYPE INTEGER, ZGLASSESTYPE INTEGER,
  ZHAIRCOLORTYPE INTEGER, ZHASSMILE INTEGER,
  ZHIDDEN INTEGER, ZISINTRASH INTEGER,
  ZISLEFTEYECLOSED INTEGER, ZISRIGHTEYECLOSED INTEGER,
  ZLIPMAKEUPTYPE INTEGER, ZMANUAL INTEGER,
  ZNAMESOURCE INTEGER, ZQUALITYMEASURE INTEGER,
  ZSMILETYPE INTEGER, ZSOURCEHEIGHT INTEGER,
  ZSOURCEWIDTH INTEGER, ZTRAININGTYPE INTEGER,
  ZASSET INTEGER, Z34_ASSET INTEGER,
  ZFACECROP INTEGER, ZFACEGROUP INTEGER,
  ZFACEGROUPBEINGKEYFACE INTEGER,
  ZFACEPRINT INTEGER, ZPERSON INTEGER,
  ZPERSONBEINGKEYFACE INTEGER, ZADJUSTMENTVERSION TIMESTAMP,
  ZBLURSCORE FLOAT, ZCENTERX FLOAT,
  ZCENTERY FLOAT, ZLEFTEYEX FLOAT, ZLEFTEYEY FLOAT,
  ZMOUTHX FLOAT, ZMOUTHY FLOAT, ZPOSEYAW FLOAT,
  ZQUALITY FLOAT, ZRIGHTEYEX FLOAT,
  ZRIGHTEYEY FLOAT, ZROLL FLOAT, ZSIZE FLOAT,
  ZYAW FLOAT, ZGROUPINGIDENTIFIER VARCHAR,
  ZMASTERIDENTIFIER VARCHAR, ZUUID VARCHAR
);

Working with this query on Photos 5
SELECT
ZGENERICASSET.ZUUID,
ZDETECTEDFACE.ZUUID,
ZDETECTEDFACE.ZPERSON,
ZPERSON.ZFULLNAME,
ZDETECTEDFACE.ZAGETYPE,
ZDETECTEDFACE.ZBALDTYPE,
ZDETECTEDFACE.ZEYEMAKEUPTYPE,
ZDETECTEDFACE.ZEYESSTATE,
ZDETECTEDFACE.ZFACIALHAIRTYPE,
ZDETECTEDFACE.ZGENDERTYPE,
ZDETECTEDFACE.ZGLASSESTYPE,
ZDETECTEDFACE.ZHAIRCOLORTYPE,
ZDETECTEDFACE.ZHASSMILE,
ZDETECTEDFACE.ZHIDDEN,
ZDETECTEDFACE.ZISINTRASH,
ZDETECTEDFACE.ZISLEFTEYECLOSED,
ZDETECTEDFACE.ZISRIGHTEYECLOSED,
ZDETECTEDFACE.ZLIPMAKEUPTYPE,
ZDETECTEDFACE.ZMANUAL,
ZDETECTEDFACE.ZQUALITYMEASURE,
ZDETECTEDFACE.ZSMILETYPE,
ZDETECTEDFACE.ZSOURCEHEIGHT,
ZDETECTEDFACE.ZSOURCEWIDTH,
ZDETECTEDFACE.ZBLURSCORE,
ZDETECTEDFACE.ZCENTERX,
ZDETECTEDFACE.ZCENTERY,
ZDETECTEDFACE.ZLEFTEYEX,
ZDETECTEDFACE.ZLEFTEYEY,
ZDETECTEDFACE.ZMOUTHX,
ZDETECTEDFACE.ZMOUTHY,
ZDETECTEDFACE.ZPOSEYAW,
ZDETECTEDFACE.ZQUALITY,
ZDETECTEDFACE.ZRIGHTEYEX,
ZDETECTEDFACE.ZRIGHTEYEY,
ZDETECTEDFACE.ZROLL,
ZDETECTEDFACE.ZSIZE,
ZDETECTEDFACE.ZYAW,
ZDETECTEDFACE.ZMASTERIDENTIFIER
FROM ZDETECTEDFACE
JOIN ZGENERICASSET ON ZGENERICASSET.Z_PK = ZDETECTEDFACE.ZASSET
JOIN ZPERSON ON ZPERSON.Z_PK = ZDETECTEDFACE.ZPERSON
ORDER BY ZGENERICASSET.ZUUID

ZAGETYPE:
0 =  ?? (have seen children and adults; perhaps "unknown" or "not yet processed")?
1 = infant
2 = child
3 = older child
4 = ??
5 = adult
ZBALDTYPE:
0 = ??
1 = not seen in database
2 = not much hair
3 = has hair

For Photos 4:
SELECT
RKFace.modelId,
RKFace.uuid,
RKVersion.uuid,
RKPerson.name,
RKFace.isInTrash,
RKFace.personId,
RKFace.imageModelId,
RKFace.sourceWidth,
RKFace.sourceHeight,
RKFace.centerX,
RKFace.centerY,
RKFace.size,
RKFace.leftEyeX,
RKFace.leftEyeY,
RKFace.rightEyeX,
RKFace.rightEyeY,
RKFace.mouthX,
RKFace.mouthY,
RKFace.hidden,
RKFace.manual,
RKFace.hasSmile,
RKFace.isLeftEyeClosed,
RKFace.isRightEyeClosed,
RKFace.poseRoll,
RKFace.poseYaw,
RKFace.posePitch,
RKFace.faceType,
RKFace.personUuid
FROM
RKFace
JOIN RKPerson on RKPerson.modelId = RKFace.personId
JOIN RKVersion on RKVersion.modelId = RKFace.imageModelId

See also phace

The "drawing" code that handles this for Photos 4. And the orientation adjustments that I validated against my personal photos lib at the time.

@neilpa perfect--thanks!  I did a lot of work on this over the weekend and have added a FaceInfo object with all the relevant details (not yet committed to GitHub).  I was able the bounding box correct for all exif cases but hadn't yet figured out the transforms for the other points. (I ended up with upside down faces for example on photos that had been rotated).

Thanks to @neilpa I've got the bounding boxes and face points working for all EXIF orientations....except for pictures rotated inside Photos.  Photos rotated using Image | Rotate don't change the value for ZORIENTATION and I've not yet been able to decipher how Photos is storing the orientation for these.

Photos rotated using Image | Rotate don't change the value for ZORIENTATION

I was going to guess that this would end up in ZADJUSTMENT or ZUNMANAGEDADJUSTMENT but that doesn't appear to be the case from looking at the records I have for my photos lib.
I've got a few different hacks which can partially "diff" two different snapshots of the photos DB. It's come in handy for reverse engineering stuff but I mostly use it to detect and backup newly imported assets. I'll see if I can use this trick to figure out how that rotation is represented.

Also, if you haven't already, you should look at the ACHANGE, ATRANSACTION, and ATRANSACTIONSTRING tables. Those have helped me decipher how things fit together. Especially when trying to trace the meanings of numeric identifiers across tables. (In that case I usually just create a SQL dump and grep the output text file).

Thanks. I've been using sqldiff a lot to compare the database between changes.  It looks like the Photos adjustments, including rotation, are in a .plist file in the same file as the edited images (resources/renders/X/ where X is first character of UUID) but the adjustment data is stored in a binary format I've not yet tried to decipher.   All adjustments produce an entry in ZINTERNALRESOURCE and there's a ZDATASTOREKEYDATA column that I suspect may map to data in the .plist file which has a DataStore field.

Thanks for the pointers, especially sqldiff. Can't believe I haven't seen that before.

I spent some time poking at this but haven't deciphired the format of the embedded blobs in the :adjustmentData field of the .plist files you mention. I did a little hex dump slicing, dicing, and diffing but nothing obvious jumped out. Will likely need a more methodical approach that I didn't have time for at the moment.
Also, In my limited testing though it doesn't look like ZDATASTOREKEYDATA can be used to detect/differentiate those rotations. It's always the same small binary value for all three rotations (90, 180, 270). I was hoping to maybe find some other clues with sqldiff for a non-plist option but no luck so far.

The .plist file in the renders folder is a serialized PHAdjustmentData object.  Unfortunately, the specification for the adjustmentData property is not specified -- it's free form data that any application that edits photos via PhotoKit can set.  So in this case, we need to look at the Photos specific format which Apple doesn't document.  This also means though that if user edited photo in some other app or plugin that used PhotoKit the rotation data might be stored in a different format (not sure how Photos would then know how to apply face regions...)
I've spent a lot of time with a hex editor trying to identify how the rotation data is stored by Photos and so far have come up empty handed.  The data field appears to be a series of binary records of variable length delimited by 0x0A09 but I've not yet found any common bytes across these for different sequences of edits and rotations that would identify the rotation.
For now, I think I'll push the initial FaceInfo implementation with a caveat that it'll be wrong for images rotated in Photos.

I think I found it!  Was looking in the wrong place.  The rotation information for images edited in Photos doesn't appear to be stored in the database but the rotation of the face itself is stored in ZDETECTEDFACE.ZROLL and ZDETECTEDFACE.ZYAW.  These appear to be angles in radians so I should be able to rotate the face points accordingly.  It appears that ZROLL is the angle in radians the face is rotated about an axis perpendicular to the screen where clockwise is negative.

I've spent a lot of time with a hex editor trying to identify how the rotation data is stored by Photos and so far have come up empty handed. The data field appears to be a series of binary records of variable length delimited by 0x0A09 but I've not yet found any common bytes across these for different sequences of edits and rotations that would identify the rotation.

Same for me. I spent a couple hours with Hex Fiend yesterday evening looking at various diffs and have yet to make any real progress. Surprising since it's only ~150 bytes of data.

I think I found it!

Awesome that you figured this out for photos with detected faces at least.

v0.31.0 has initial support for face regions. See docs.  Note: The bounding box for the face region appears to be correct for all possible rotations and flips of the image but for the left_eye, right_eye, and mouth properties which return coordinates (in the image coordinate frame) for the eyes and mouth are not always correct if the face is tilted off normal.  The angles to describe the necessary rotation are stored in roll, pitch, and yaw but I've not yet figured out how to apply the transformation (except for roll which is updated when an image is itself rotated).  See issue #196.

Now need to add face regions to XMP sidecars.  See iphoto2xmp:
https://github.com/jensb/iphoto2xmp/blob/2e47eacb45f385f9447fa674ef0bbd5a362c14d8/iphoto2xmp_template.xmp.erb#L141-L160

I think the XMP sidecar is now properly encoding the face regions...however, I've not been able to find a way to test this.  I've tried both Digikam and Lightroom, both of which I'm unfamiliar with, and can't figure out how to get them to import face regions from an XMP file despite hours of googling.  I'm going to put this issue on hold until I can find an app or some code that will show me the face regions in an exported image so I can verify they're being encoded properly.

I've not been able to find a way to test this. I've tried both Digikam and Lightroom, both of which I'm unfamiliar with, and can't figure out how to get them to import face regions from an XMP file despite hours of googling

I hit this exact same issue when working on this in phace. I could not find an app that actually used the encoded face regions. IIRC the docs for DigiKam claimed to but in all my experimenting I couldn't find an example of it working. Best I did was find some sample images with the XMP face regions already encoded and made sure I could round-trip the data based on my understanding of the format.

@neilpa I've figured out how to make DigiKam at least write the XMP files with embedded face regions.  My plan is to mark face regions in DigiKam then export the XMP files for a bunch of photos in every possible EXIF orientation then write a program that reads the XMP files and draws the face regions with Pillow.  Assuming DigiKam is properly encoding the face regions and I can visually verify that I can interpret the encodings correctly, that will be give me a way to then verify the regions I'm encoding are correct.  But it does lead to the question of what the value is in face regions if apps don't actually read this information.  I like the idea of exporting it to future proof the export but it is frustrating the state of this so bad given how much effort it takes to properly tag a library with thousands of photos!  In researching this I learned that even though there's a Metadata Working Group standard for face regions, Microsoft developed their own which is completely different (but best as I can tell, encodes no additional information).  DigiKam does encode the Microsoft format as well so perhaps I can add that to osxphotos too.

Hi,
for example pigallery2 use the metadata of the image files to show the persons/faces
so it will be cool to keep the faces, store them in the metadata and a presentation tool like pigallery2 can use them instead of refined all faces again ..
from pigallery2 website ... Faces (persons) support reads Adobe's XMP Face region metadata. (It is defined by the Metadata Working Group (MWG).)
https://github.com/bpatrik/pigallery2
best regards Dominique

Thanks @d0m1n1qu3 I'll take a look at pigallery2.  Currently, the face information is written to the XMP sidecar if you use osxphotos export /path/to/export --sidecar xmp option but I've not been able to properly test this as I can't find an app that actually uses this information.  If pigallery2 will read the sidecars, that would help confirm I'm exporting the data correctly.

Would love to See xmp faceregion support. Supported by Digikam, Lightroom and others like Photostation on a Synology NAS. Tell me, if I could help.
I exported the data with the PersonInImage Tag but I see no way to convert it to Faceregion via Exiftool.

Would love to See xmp faceregion support. Supported by Digikam, Lightroom and others like Photostation on a Synology NAS. Tell me, if I could help.
I exported the data with the PersonInImage Tag but I see no way to convert it to Faceregion via Exiftool.

osxphotos currently exports the personInImage info (name of person) but not the face region. It does have access to the face regions info but I've not found a way to validate the XMP nor found any program that will actually read this info. Lightroom writes it but I couldn't figure out how to make Lightroom read it. Happy for some help here: 1) an example of a valid XMP with Face Regions so I can copy the format, 2) ability to test the creature against some app that will actually read it from XMP.

osxphotos currently exports the personInImage info (name of person) but not the face region. It does have access to the face regions info but I've not found a way to validate the XMP nor found any program that will actually read this info. Lightroom writes it but I couldn't figure out how to make Lightroom read it. Happy for some help here: 1) an example of a valid XMP with Face Regions so I can copy the format, 2) ability to test the creature against some app that will actually read it from XMP.

Sorry for the delay. So here is a true xmp, generated from Digikam. The first is normal orientation with People named left to right (LTR1, LTR 2, etc) and the second Data is the same picture and naming only rotated left. Had to change the type from amp to txt. Does this help? do you need anything else? Will be happy to test the new creature :-)
digikam_test_normal.txt
digikam_test_rotate.txt

Sorry for the delay. So here is a true xmp, generated from Digikam. The first is normal orientation with People named left to right (LTR1, LTR 2, etc) and the second Data is the same picture and naming only rotated left. Had to change the type from amp to txt. Does this help? do you need anything else? Will be happy to test the new creature :-)
digikam_test_normal.txt
digikam_test_rotate.txt

That's helpful -- thanks! I'll be working on XMP this weekend -- #349, #350 -- so will take a look at this as well.

That's helpful -- thanks! I'll be working on XMP this weekend -- #349, #350 -- so will take a look at this as well.

Than thank you very much in advance!

Than thank you very much in advance!

The DigiKam XMP files contain face regions in two formats:
The Metadata Working Group Regions (mwg-rs), see after page 51 here.
Microsoft MPRI format ("Microsoft Photo Region Info"?)

People Rectangles
People's names however, are only part of the people-tagging feature. In addition to storing people's names in the metadata, the schema also supports region information that identifies the specific area (a rectangle) the person is shown in the image.
The rectangle information is represented by four comma-delimited decimal values, such as "0.25, 0.25, 0.25, 0.25". The first two values specify the top-left coordinate; the final two specify the height and width of the rectangle. The dimensions of the image for the purposes of defining people rectangles are normalized to 1, which means that in the "0.25, 0.25, 0.25, 0.25" example, the rectangle starts 1/4 of the distance from the top and 1/4 of the distance from the left of the image. Both the height and width of the rectangle are 1/4 of the size of their respective image dimensions.

The DigiKam XMP files contain face regions in two formats:
The Metadata Working Group Regions (mwg-rs), see after page 51 here.
Microsoft MPRI format ("Microsoft Photo Region Info"?)

People Rectangles
People's names however, are only part of the people-tagging feature. In addition to storing people's names in the metadata, the schema also supports region information that identifies the specific area (a rectangle) the person is shown in the image.
The rectangle information is represented by four comma-delimited decimal values, such as "0.25, 0.25, 0.25, 0.25". The first two values specify the top-left coordinate; the final two specify the height and width of the rectangle. The dimensions of the image for the purposes of defining people rectangles are normalized to 1, which means that in the "0.25, 0.25, 0.25, 0.25" example, the rectangle starts 1/4 of the distance from the top and 1/4 of the distance from the left of the image. Both the height and width of the rectangle are 1/4 of the size of their respective image dimensions.

Yes I think for broad compatability. But I think mwg is the best way

Yes I think for broad compatability. But I think mwg is the best way

@martinhrpi do you know how to make Digikam actually read the face tags from XMP? I've tried and not been able to make Digikam display the tagged faces in an XMP file. I worked on this issue several months ago and gave up because I could not find a way to test that the output is correct.  E.g. I need a way to write an XMP file then have some program read the XMP file and display the tagged regions to see if they're correct.   Without that, I'm shooting in the dark.

@martinhrpi do you know how to make Digikam actually read the face tags from XMP? I've tried and not been able to make Digikam display the tagged faces in an XMP file. I worked on this issue several months ago and gave up because I could not find a way to test that the output is correct.  E.g. I need a way to write an XMP file then have some program read the XMP file and display the tagged regions to see if they're correct.   Without that, I'm shooting in the dark.

exiftool region tags:
https://exiftool.org/TagNames/MWG.html#Regions
exiftool -xmp-mwg-rs:RegionAppliedToDimensionsH=4000 -xmp-mwg-rs:RegionAppliedToDimensionsUnit="pixel" -xmp-mwg-rs:RegionAppliedToDimensionsW=6000 -xmp-mwg-rs:RegionAreaX=0.319270833 -xmp-mwg-rs:RegionAreaY=0.21015625 -xmp-mwg-rs:RegionAreaW=0.165104167 -xmp-mwg-rs:RegionAreaH=0.30390625 -xmp-mwg-rs:RegionName=John -xmp-mwg-rs:RegionRotation=0 -xmp-mwg-rs:RegionType="Face" myfile.xmp

exiftool region tags:
https://exiftool.org/TagNames/MWG.html#Regions
exiftool -xmp-mwg-rs:RegionAppliedToDimensionsH=4000 -xmp-mwg-rs:RegionAppliedToDimensionsUnit="pixel" -xmp-mwg-rs:RegionAppliedToDimensionsW=6000 -xmp-mwg-rs:RegionAreaX=0.319270833 -xmp-mwg-rs:RegionAreaY=0.21015625 -xmp-mwg-rs:RegionAreaW=0.165104167 -xmp-mwg-rs:RegionAreaH=0.30390625 -xmp-mwg-rs:RegionName=John -xmp-mwg-rs:RegionRotation=0 -xmp-mwg-rs:RegionType="Face" myfile.xmp

So Digikam reads the mwg in xmp I provided to you. I copied the image and XMP, deleted the original and re-imported it and it worked.
Did you tell Digikam to read/write xmp in the settings?

So Digikam reads the mwg in xmp I provided to you. I copied the image and XMP, deleted the original and re-imported it and it worked.
Did you tell Digikam to read/write xmp in the settings?

Btw I use the Beta. So Digikam 7.2 Beta 2.
and concerning exiftool: there is no way to convert your exported Data to mwg via exiftool as far as I found.

Btw I use the Beta. So Digikam 7.2 Beta 2.
and concerning exiftool: there is no way to convert your exported Data to mwg via exiftool as far as I found.

Did you tell Digikam to read/write xmp in the settings?
I did.  But I can't figure out how to make Digikam actually show the face regions.  (I don't use Digikam so not familiar with it -- just need something for testing)

Did you tell Digikam to read/write xmp in the settings?
I did.  But I can't figure out how to make Digikam actually show the face regions.  (I don't use Digikam so not familiar with it -- just need something for testing)

So let me make something for you :)

So let me make something for you :)

So here is a Test-Picture from Google (Creative Commons). I imported the picture to Digikam, named the man "Test Man". Then I copied the picture away from Digitkam and delete it in Digikam. No data left in Digikam.
Then, with the help of exiftool
exiftool -overwrite_original -all=
I wiped the data from the picture file (so only the data from sidecar xmp) are left.
Then I re-Imported the picture and voila: after reading from the Sidecar: there was the face again with the same rectangle (I think) and the correct name.

test.txt

So here is a Test-Picture from Google (Creative Commons). I imported the picture to Digikam, named the man "Test Man". Then I copied the picture away from Digitkam and delete it in Digikam. No data left in Digikam.
Then, with the help of exiftool
exiftool -overwrite_original -all=
I wiped the data from the picture file (so only the data from sidecar xmp) are left.
Then I re-Imported the picture and voila: after reading from the Sidecar: there was the face again with the same rectangle (I think) and the correct name.

test.txt

Thanks!

Thanks!

After wasting many hours in following loop: 1) modify XMP, 2) load it in digikam, 3) curse, 4) delete digikam database 5) repeat, I've figured out that Digikam is reading the Microsoft MPRI region info and not the standard mwg region info.
Given this knowledge, I think I can write this info to the XMP. I would still like to include mwg regions and I think osxphotos is correctly writing these but I have no way to test.  Lack of ability to test this feature is why I abandoned it months ago.  I wish a simple app existed that would read an XMP, validate it, and then draw/label face regions on the jpeg which I could use for testing.  I've searched Google and Github to no avail.  I'd write it except of course, it would only validate that I'm interpreting the spec correctly, not that I actually got it right.  While the standard case is fairly easy, handling all the different rotations / orientations for an image is tricky and that's what I need to test.

After wasting many hours in following loop: 1) modify XMP, 2) load it in digikam, 3) curse, 4) delete digikam database 5) repeat, I've figured out that Digikam is reading the Microsoft MPRI region info and not the standard mwg region info.
Given this knowledge, I think I can write this info to the XMP. I would still like to include mwg regions and I think osxphotos is correctly writing these but I have no way to test.  Lack of ability to test this feature is why I abandoned it months ago.  I wish a simple app existed that would read an XMP, validate it, and then draw/label face regions on the jpeg which I could use for testing.  I've searched Google and Github to no avail.  I'd write it except of course, it would only validate that I'm interpreting the spec correctly, not that I actually got it right.  While the standard case is fairly easy, handling all the different rotations / orientations for an image is tricky and that's what I need to test.

I would love to use what you have built so far. As a Workaround, since Digikam seems to use microsoft but also writes mwg I could use osxphotos to write the face and get it out of Photos, and then use Digikam to make it "more available" aka transform it to the other Face-Standards.

I would love to use what you have built so far. As a Workaround, since Digikam seems to use microsoft but also writes mwg I could use osxphotos to write the face and get it out of Photos, and then use Digikam to make it "more available" aka transform it to the other Face-Standards.

So XnView can read mwg. But not in the sidecar it seems, only in the Image-File. Tested with this file (delete the Microsoft Tags first).

So XnView can read mwg. But not in the sidecar it seems, only in the Image-File. Tested with this file (delete the Microsoft Tags first).

Thanks -- I had previous found & tried xnview but as you noted, I couldn't get it to read the sidecar.   I'll post a beta release as soon as I can get digikam to read the xmp generated by osxphotos.  Currently, exiftool says the XMP is valid but I cannot get digikam to read any XMP file it did not actually generate.

Thanks -- I had previous found & tried xnview but as you noted, I couldn't get it to read the sidecar.   I'll post a beta release as soon as I can get digikam to read the xmp generated by osxphotos.  Currently, exiftool says the XMP is valid but I cannot get digikam to read any XMP file it did not actually generate.

So thank you very very much for the beta - happy to test it. As for sidecar-support on XnView: in the current Version it supports Sidecar files as I found out minutes ago. Testet it with the emptied image file and amp sidecar where I changed a view things via a text-editor. It worked so I think you could test the data with XnView!

So thank you very very much for the beta - happy to test it. As for sidecar-support on XnView: in the current Version it supports Sidecar files as I found out minutes ago. Testet it with the emptied image file and amp sidecar where I changed a view things via a text-editor. It worked so I think you could test the data with XnView!

edit: But it seems like XnView doesn't draw the face if the data is in the sidecar only.
This is the main issue.  I know the sidecar is valid because exiftool accepts it (and I consider exiftool the gold standard) but because I need to apply coordinate transformations for the face region I really need to see what some other app thinks the bounding box is.  I will try to apply the XMP, via exiftool, to the image file then see if XnView will show it.

edit: But it seems like XnView doesn't draw the face if the data is in the sidecar only.
This is the main issue.  I know the sidecar is valid because exiftool accepts it (and I consider exiftool the gold standard) but because I need to apply coordinate transformations for the face region I really need to see what some other app thinks the bounding box is.  I will try to apply the XMP, via exiftool, to the image file then see if XnView will show it.

I will try to apply the XMP, via exiftool, to the image file then see if XnView will show it.

I tried that with Test Man and it worked.

I will try to apply the XMP, via exiftool, to the image file then see if XnView will show it.

I tried that with Test Man and it worked.

I think mwg regions are correct now.  First photo is face region in Photos (stored by Photos as circle) and second is in XnView (XMP stores region as rectangle).  Still can't get Digikam to read the XMP though.  Now I'll work on confirming the MPRI regions are correct as that seems to be what Digikam uses.  These use a different coordinate system.

I think mwg regions are correct now.  First photo is face region in Photos (stored by Photos as circle) and second is in XnView (XMP stores region as rectangle).  Still can't get Digikam to read the XMP though.  Now I'll work on confirming the MPRI regions are correct as that seems to be what Digikam uses.  These use a different coordinate system.

I think I now have the Microsoft Photo Regions working.  Identical results for both MWG regions and MPRI regions.  (Photo is creative commons from Flickr--one of the photos in the osxphotos test library).  First is MPRI region and second is MWG-RS region.

I think I now have the Microsoft Photo Regions working.  Identical results for both MWG regions and MPRI regions.  (Photo is creative commons from Flickr--one of the photos in the osxphotos test library).  First is MPRI region and second is MWG-RS region.

If both MWG-RS and MPRI face regions are in the XMP data, both XnView and Digikam

If both MWG-RS and MPRI face regions are in the XMP data, both XnView and Digikam

@martinhrpi  I've added beta support for face regions in XMP in v0.39.21.  Note, to use this you'll need to use the undocumented "--beta" flag when exporting.  e.g.
osxphotos export /path/to/export --sidecar xmp --beta --sidecar-drop-ext
I'd be happy for any feedback you have on whether this is working.  There may be issues with different photo orientations though the ones I've tested do work.  I still need to figure out why Digikam won't read the XMP sidecar.

@martinhrpi  I've added beta support for face regions in XMP in v0.39.21.  Note, to use this you'll need to use the undocumented "--beta" flag when exporting.  e.g.
osxphotos export /path/to/export --sidecar xmp --beta --sidecar-drop-ext
I'd be happy for any feedback you have on whether this is working.  There may be issues with different photo orientations though the ones I've tested do work.  I still need to figure out why Digikam won't read the XMP sidecar.

One known issue: the face region info in Photos is stored relative to the most recent version of the photo (e.g. if photo has adjustments/edits, face region applies to the edited photo, not the original).  osxphotos currently exports the same XMP data for both edited and original version so if you resized or cropped the photo, face regions for the original will be incorrect.  I still need to think about how to handle this as it will complicate the XMP sidecar logic to deal with this properly.  Happy for any thoughts on how to deal with this case.

One known issue: the face region info in Photos is stored relative to the most recent version of the photo (e.g. if photo has adjustments/edits, face region applies to the edited photo, not the original).  osxphotos currently exports the same XMP data for both edited and original version so if you resized or cropped the photo, face regions for the original will be incorrect.  I still need to think about how to handle this as it will complicate the XMP sidecar logic to deal with this properly.  Happy for any thoughts on how to deal with this case.

Uff!  Testes ist with a test-database of about 60 photos, works perfect! I can‘t thank you enough!
Will test it now with a 16.000 picture data-base and will think about the edited ones, too. I scanned a lot of old pictures (yet another photos database) and edited them - so I have quite a good testing ground.
thank you so much so far - you saved me months of work!

Uff!  Testes ist with a test-database of about 60 photos, works perfect! I can‘t thank you enough!
Will test it now with a 16.000 picture data-base and will think about the edited ones, too. I scanned a lot of old pictures (yet another photos database) and edited them - so I have quite a good testing ground.
thank you so much so far - you saved me months of work!

Oh wow so Digikam even uses the not yet named faces for their "unknown" category. So you get the ones you already named in MacOS Photos AND you get the ones that you didn't name in Photos. That is even more convenient.
To give you some perspective of your impact (at least on me):

Mac User for over 15 years, first with iPhoto now with Photos.
I also have a Synology NAS with their "Synology Moments" App (which could detect Faces but stores them in their own database) and Photostation (which is bad in detection but reads xmp faces).
So I moved my photos from MacOS to the NAS and named about 40.000 pictures again in "Moments".
Now Synology introduced a new App as part of their new OS as a beta (https://community.synology.com/enu/forum/20/post/139108) which is the badest part of both old apps (Moments/Photostation), since their is no migration of their database and no more Tag-Reading....
So I had enough and switched to Digikam for naming pictures and "archiving" them in the picture by a well-known open standard (even though it seems with faces their are a 1000 standards).
So with your help I can now export a rest of about 16.000 pictures with ALL of the data from MacOS in the matter of minutes and not months or weeks, not counting about 3.000 scanned Dias, negatives and so on.

Thank you again for that great help - and I will look into the edit thing :-)

Oh wow so Digikam even uses the not yet named faces for their "unknown" category. So you get the ones you already named in MacOS Photos AND you get the ones that you didn't name in Photos. That is even more convenient.
To give you some perspective of your impact (at least on me):

Mac User for over 15 years, first with iPhoto now with Photos.
I also have a Synology NAS with their "Synology Moments" App (which could detect Faces but stores them in their own database) and Photostation (which is bad in detection but reads xmp faces).
So I moved my photos from MacOS to the NAS and named about 40.000 pictures again in "Moments".
Now Synology introduced a new App as part of their new OS as a beta (https://community.synology.com/enu/forum/20/post/139108) which is the badest part of both old apps (Moments/Photostation), since their is no migration of their database and no more Tag-Reading....
So I had enough and switched to Digikam for naming pictures and "archiving" them in the picture by a well-known open standard (even though it seems with faces their are a 1000 standards).
So with your help I can now export a rest of about 16.000 pictures with ALL of the data from MacOS in the matter of minutes and not months or weeks, not counting about 3.000 scanned Dias, negatives and so on.

Thank you again for that great help - and I will look into the edit thing :-)

Success!  New version v0.39.22 generates XMP sidecars that Digikam will read! Must still be used with --beta to enable this.  As soon as I get a little more testing I'll enable this by default.

Success!  New version v0.39.22 generates XMP sidecars that Digikam will read! Must still be used with --beta to enable this.  As soon as I get a little more testing I'll enable this by default.

And I have found (I think) that it‘s not a only problem with pictures edited by OS Photo but also with selfies or pictures rotated outside of photos by exif info. So I don‘t think there is a fix - but this small percentage of wrongly drawn faces is nothing against the big advantages!

And I have found (I think) that it‘s not a only problem with pictures edited by OS Photo but also with selfies or pictures rotated outside of photos by exif info. So I don‘t think there is a fix - but this small percentage of wrongly drawn faces is nothing against the big advantages!

@martinhrpi if you have one of the "incorrect" images you're willing to share, send me the image and XMP file to<EMAIL_ADDRESS>and I'll see if I can figure it out.  I think I have an idea how to fix this (though there are some photos for which I cannot easily get the correct orientation due to how Photos stores this data).

@martinhrpi if you have one of the "incorrect" images you're willing to share, send me the image and XMP file to<EMAIL_ADDRESS>and I'll see if I can figure it out.  I think I have an idea how to fix this (though there are some photos for which I cannot easily get the correct orientation due to how Photos stores this data).

I think I've fixed the incorrect face regions for most photos -- tested with several different exif orientations.  See release v0.39.23.  Photos rotated within Apple Photos app may still be incorrect as these photos do not change the orientation data in the database.  I think I can extract the correct orientation using the native PhotoKit interface but this will take more work.

I think I've fixed the incorrect face regions for most photos -- tested with several different exif orientations.  See release v0.39.23.  Photos rotated within Apple Photos app may still be incorrect as these photos do not change the orientation data in the database.  I think I can extract the correct orientation using the native PhotoKit interface but this will take more work.

Ahhh saw your comment to late. I have just now collected 35 MB of problematic pictures. Would just Need to Upload those.
I will test the new Version and give feedback.

Ahhh saw your comment to late. I have just now collected 35 MB of problematic pictures. Would just Need to Upload those.
I will test the new Version and give feedback.

YES! I can confirm the images that were incorrect because the camera set the rotation or iOS switched it because of selfie are now correct!

YES! I can confirm the images that were incorrect because the camera set the rotation or iOS switched it because of selfie are now correct!

Great! Thanks much for the testing and feedback.  I'll promote this new XMP format to "non-beta" in the next release.  Now I'll need to work on --no-person (#284) option because some people want to export XMP without details on people.

Great! Thanks much for the testing and feedback.  I'll promote this new XMP format to "non-beta" in the next release.  Now I'll need to work on --no-person (#284) option because some people want to export XMP without details on people.

Thank you for your great work! I will now export 24.000 pictures! Hooray :-)

Thank you for your great work! I will now export 24.000 pictures! Hooray :-)

One more update --  I found the face regions weren't correct for a couple of EXIF orientations.  I've tested this with photos of every orientation I had in my library (only EXIF 1, 3, 6, 8) and they were all correct with this latest fix.

One more update --  I found the face regions weren't correct for a couple of EXIF orientations.  I've tested this with photos of every orientation I had in my library (only EXIF 1, 3, 6, 8) and they were all correct with this latest fix.

Here's a quick script to get the "best" 3 photos of each possible orientation that also contain faces and print out the UUIDs in a form useable by --uuid-from-file for testing:
import osxphotos

photos = osxphotos.PhotosDB().photos()
for orientation in range(1,9):
    # valid EXIF orientations are 1-8
    face_photos = sorted([p for p in photos if p.orientation == orientation and p.face_info], key=lambda p: p.score.overall, reverse=True)
    if face_photos:
        num_photos = 3 if len(face_photos) >= 3 else len(face_photos)
        for x in range(num_photos):
            photo = face_photos[x]
            print(f"#{photo.orientation}")
            print(photo.uuid)

Here's a quick script to get the "best" 3 photos of each possible orientation that also contain faces and print out the UUIDs in a form useable by --uuid-from-file for testing:
import osxphotos

photos = osxphotos.PhotosDB().photos()
for orientation in range(1,9):
    # valid EXIF orientations are 1-8
    face_photos = sorted([p for p in photos if p.orientation == orientation and p.face_info], key=lambda p: p.score.overall, reverse=True)
    if face_photos:
        num_photos = 3 if len(face_photos) >= 3 else len(face_photos)
        for x in range(num_photos):
            photo = face_photos[x]
            print(f"#{photo.orientation}")
            print(photo.uuid)

Fixed face regions for mirrored exif orientations in v0.39.25.  All exif orientations have now been tested.

Fixed face regions for mirrored exif orientations in v0.39.25.  All exif orientations have now been tested.

v0.40.0 now includes face regions by default in the XMP file (no need for --beta flag). All exif orientations seem to be working.  Some photos rotated in Photos.app may still be incorrect.

v0.40.0 now includes face regions by default in the XMP file (no need for --beta flag). All exif orientations seem to be working.  Some photos rotated in Photos.app may still be incorrect.

@RhetTbull this is really impressive. Thanks for creating this functionality.
Is there a way to get this functionality working on the iphone -- I'm not familiar enough (yet) with differences between how the photos library on macosx exists vs photos on mobile.

@wasauce osxphotos itself is a command line tool written in python and will not run on the iPhone. You can however connect an iPhone to  a Mac and copy the Photos database to the Mac (using something like iMazing) then use osxphotos to read the database and output information about your photos. This is not very practical.
What specifically are you trying to do on the iPhone?

@RhetTbull - my hope is to be apply to apply some of the same approach as what you have done here in an iOS app. Obviously not in python. Goal is to be able to in an app, with permissions granted, show the photos that have names/people identified -- and then help users create profiles with those photos for those people.
Are you aware of any projects that has made this work in swift / on an iOS device?
Thank you again!

@wasauce I don't believe this will be possible on iOS due to sandboxing. On iOS, the only way to access Photos and associated metadata is via the PhotoKit API and Apple does not provide a way to access information about persons in the photos via the public API.  OSXPhotos works by directly reading the Photos.sqlite database in the Photos library.  The database exists on iOS but it is not possible (AFAIK) to read the database from an iOS app.
There is a Mac app, Face Explorer that uses the same techniques as OSXPhotos to read face data from the Photos library but again, this requires full access to the underlying database.
It would be great if Apple provided better APIs for working with Photos but so far they've not shown any interest in doing so.

@RhetTbull - Thanks very much for your detailed reply. I greatly appreciate it.
Yes, I read the PhotoKit documentation and via everything I see, I don't see Apple providing a way to access information about people in the photos.
Thank you for the link to Face Explorer -- interesting.
What you have built here is quite impressive. Thank you again for your replies!


Can I use latest deb-s3 version to create ppa for Ubuntu 20.04 (focal)

Can I use latest deb-s3 version to create ppa for Ubuntu 20.04 (focal)

I'm creating repos for focal with this version, so yes ;)

@morph027 thank you, when we'll expect that version please?

@morph027 thank you, when we'll expect that version please?

@arbabnazar he's saying you can create PPAs already 😅

@arbabnazar he's saying you can create PPAs already 😅

Jip. I guess you refer to the PR https://github.com/krobertson/deb-s3/pull/164 which has been merged into this with https://github.com/deb-s3/deb-s3/commit/b305fefbaa089ef66d5d2b7a2bb7e9965c8bf82f.
I still can't release on rubygems, as the original package belongs to @krobertson, so you need to install manually like written in the README

Jip. I guess you refer to the PR https://github.com/krobertson/deb-s3/pull/164 which has been merged into this with https://github.com/deb-s3/deb-s3/commit/b305fefbaa089ef66d5d2b7a2bb7e9965c8bf82f.
I still can't release on rubygems, as the original package belongs to @krobertson, so you need to install manually like written in the README


test: skip 2 ams tests under Python 2.7 to unblock CI

@BrianBlum, I will log an issue to track the investigation, but right now I  need to do it to unblock other PRs which are impacted by these 2 test failures.
I am not able to reproduce it in my dev box with 2.7, and git history has not shown any related change. Same tests are passing at #9317, so more likely a flaky test?

[x] The PR has modified HISTORY.rst describing any customer-facing, functional changes. Note that this does not include changes only to help content. (see Modifying change log).

[x] I adhere to the Command Guidelines.

We are deprecating the support on Python 2.7. Hence we are won't fix this.


WIP: Use portable zip download for imagemagick on windows

for #316

apparently there's no libwand there - that's not so useful


Write Pact Tests for Complainant/Witness/Accused on CaseDetails page

Business Context:
We badly need end-to-end tests to die, but we can't feel secure in eliminating them until we have tests that achieve similar coverage. To that end we are writing pact tests that will allow us to establish a contract between the client and the server and run integration tests against said contract.
As a developer who hates the flakiness of e2e tests
I want to be able to replace e2e tests with pact tests
So that the pipeline can pass consistently
In Scope

Add/Edit/Remove Complainant/Witness/Accused
Should be able to add deeper details for civilians (address, phone, etc.) and see them when expanding the row
Should be able to anonymize both officer and civilian

Out of Scope

Open Questions

Acceptance Criteria

Scenario:
Given
When
Then

Scenario:
Given
When
Then

Mockups
Checklist (should be reviewed before grooming as the BA or Pair are writing the card—NOT reviewed for the first time at grooming)

Discuss the technical implementation of this card.
Discuss how this feature would be included in or affect the Audit Log.
In respect to security, what could go wrong? Brainstorm threats using STRIDE model, especially if this card introduces a new technical architecture or design pattern. (TIMEBOX: 7 mins) https://docs.google.com/document/d/18AFl1d3SRieI4d_8R3GyEazqQOycLNDHGN_vOYIHuiE/edit#heading=h.aio9zb5j63wl
Ensure any open questions have been resolved with stakeholders.
Determine if documentation should be referenced or created.
Ensure to include all necessary mockups to this card.
Discuss what success/failure user feedback messages would be present, if any occur (front end/back end).
Discuss why this feature would not be implemented behind a toggle / what would happen if this card was partially done while a release branch was being cut.
If this card is part of a sequence of cards, discuss if this card should be blocked until the testing of previous  cards in the sequence have been finished.
Have we added edge cases as ACs that we're considered from a QA perspective
Does this card overlap with any entities outlined in API Housekeeping?

Notes:

should add to case-details.pact.test.js


Pika/web freeze on installing when project has polka package

Hi team!
I setup fresh new sapper-svelte package from their document. then install pika/web. when run npx @pika/web it seem setup correctly untill start install package, at polka it say like this:
Trungs-MacBook-Pro:admin trungrueta$ npm i -D @pika/web
npm WARN<EMAIL_ADDRESS>No repository field.
npm WARN<EMAIL_ADDRESS>No license field.

+<EMAIL_ADDRESS>added 44 packages from 64 contributors and audited 3160 packages in 9.187s
found 0 vulnerabilities

Trungs-MacBook-Pro:admin trungrueta$ npx @pika/web
⠙ @pika/web installing... compression, polka, sirvError: Could not load http (imported by /Volumes/Storage/buildweb/tapforce/projects/submit/v2/admin/node_modules/polka/index.mjs): ENOENT: no such file or directory, open 'http'
⠹ @pika/web installing... compression, polka, sirv

this is info about sapper-svelte project: https://sapper.svelte.dev/docs
sapper can fast create project follow :
npx degit "sveltejs/sapper-template#rollup" my-app
Thankyou!
PS: not only polka package, i found that when run pika.web and inside package dependencies contains packages with css, it will failed and when pika failed, it will freeze at install stage forever.
This is example when i install bulma (package UI only)
yarn run v1.15.2
warning package.json: No license field
$ pika-web
⠋ @pika/web installing... Error: Unexpected character '@' (Note that you need plugins to import files that are not JavaScript)
    at error (/Volumes/Storage/buildweb/tapforce/projects/submit/v2/admin/node_modules/rollup/dist/rollup.js:9435:30)
    at Module.error (/Volumes/Storage/buildweb/tapforce/projects/submit/v2/admin/node_modules/rollup/dist/rollup.js:13407:9)
    at tryParse (/Volumes/Storage/buildweb/tapforce/projects/submit/v2/admin/node_modules/rollup/dist/rollup.js:13320:16)
    at Module.setSource (/Volumes/Storage/buildweb/tapforce/projects/submit/v2/admin/node_modules/rollup/dist/rollup.js:13630:33)
    at /Volumes/Storage/buildweb/tapforce/projects/submit/v2/admin/node_modules/rollup/dist/rollup.js:16460:20
    at async Promise.all (index 0)
    at async Promise.all (index 0)
    at async Promise.all (index 0)
    at async install (/Volumes/Storage/buildweb/tapforce/projects/submit/v2/admin/node_modules/@pika/web/dist-node/index.js:339:25)
    at async cli (/Volumes/Storage/buildweb/tapforce/projects/submit/v2/admin/node_modules/@pika/web/dist-node/index.js:380:18)
⠴ @pika/web installing... bulma```

For the JS side, you may need to specify exactly which modules you need in your frontend. Polka is a server-side tool (thus the requirement on the Node core http module). Since http isn't a userland module, it won't be found in node_modules and pika won't know what to do.

i see, but how about Bulma . Bulma is package with stylesheet-only (sass | scss | css ), it used in client side only and if i must exclude all packages with not contains style in code that will be hard.

See #76.

Yup, that's something we're trying to tackle now in #76. In the meantime, you could look into moving out CSS from your node_modules/ directory yourself as a part of your install.
# Example:
"prepare": "npx @pika/web && cp node_modules/bulma/bulma.min.css web_modules/bulma.css"

Tooting my own horn a bit... Up until pika I've been using  vendor-copy. It's less capable than pika is for JS (it only copies stuff, which is why I'm migrating my stuff away from it to pika), but might be what you're looking for for CSS.

Nice! Thanks for sharing. Closing with follow up in #76


Interface usage refactoring

Common naming for interfaces IClassName. So changed all names to this standart. Also changed Ctx to Context.

@fletcherist  Исправлено


MAINT: stats.mode: fix issues with nan_policy='propagate'

Reference issue
closes gh-9815
What does this implement/fix?
Two issues were observed in gh-9815:

mode with nan_policy='propagate' did not always produce the expected result when the input contained NaNs
mode with nan_policy='propagate' encountered an error when the input was an object array containing types that could not be compared

This fixes these issues by replacing mode's fall-back algorithm.
Now:
import numpy as np
from scipy import stats
stats.mode([np.nan, np.nan, np.nan, 1., 1., 2., 3., 3., 3., np.nan])  # ModeResult(mode=nan, count=4)
stats.mode(np.array([10, True, 'hello', 10], dtype='object'))  # ModeResult(mode=10, count=2)

Additional information
There are multiple binary representations of NaN. This PR does not attempt to treat all of them as the same value. However, it is careful to preserve the NaN representations of the user input so that they are compared as intended. This is important, because NumPy may alter the representation of NaNs in a sequence when converting it to a numerical array.

from collections import Counter
import numpy as np

a = [2, np.nan, 1, np.nan]
Counter(a)  # Counter({2: 1, nan: 2, 1: 1})
Counter(np.asarray(a, dtype=object))  #  Counter({2: 1, nan: 2, 1: 1})
Counter(np.asarray(a, dtype=float))  # Counter({2.0: 1, nan: 1, 1.0: 1, nan: 1})

It might be possible to treat different NaNs as the same, but we would probably have to change them to a sentinel value first (e.g. a[np.isnan(a)] = 'a_sentinel_value_not_already_in_data'). For object arrays, we'd have to loop through element-by-element (not just slice by slice) because np.isnan doesn't work on object arrays.  Do we want to do that? There was not consensus in gh-9815 on whether that was important or not, just that it was important to document. Either way, we'll explain it in a Notes section.

NumPy may alter the representation of NaNs in a sequence when converting it to a numerical array.

Is it documented anywhere?

It might be possible to treat different NaNs as the same, but we would probably have to change them to a sentinel value first

I think treating all nans the same way would lead to a more intuitive behavior (at least, from the perspective of the user). For example, if two nans were produced in a different manner, they might have different IEEE 784 representations and lead to unintuitive results:
>>> import numpy as np
>>> from scipy.stats import mode
>>> import struct
>>> def binary(num):
...     return ''.join('{:0>8b}'.format(c) for c in struct.pack('!f', num))
>>> zero = np.float64(0)
>>> my_nan = (zero / zero)
<stdin>:1: RuntimeWarning: invalid value encountered in double_scalars
>>> binary(my_nan)
'11111111110000000000000000000000'
>>> binary(my_nan) == binary(np.nan)
False
>>> a = [2, np.nan, 1, my_nan]
>>> x = np.asarray(a, dtype=object)
>>> y = np.asarray(a, dtype=float)
>>> binary(x[1]), binary(x[3])
('01111111110000000000000000000000', '11111111110000000000000000000000')
>>> binary(y[1]), binary(y[3])  # NumPy didn't change the binary representation when converting from object -> float
('01111111110000000000000000000000', '11111111110000000000000000000000')
>>> binary(np.nan)
'01111111110000000000000000000000'
>>> mode(x)
ModeResult(mode=2, count=1)  # oops, mode should be `nan` intuitively

For object arrays, we'd have to loop through element-by-element (not just slice by slice) because np.isnan doesn't work on object arrays. Do we want to do that?

Should we treat nan specially for object arrays? I think if the user passes an object array, each element should be treated as an object (even all types of nans) and calculations must be done accordingly. To me, nan_policy makes sense only for numeric array since nans can be problematic for numeric computations.

Are you saying we should use the binary representation to catch all kind of nan? It seems quite difficult to guarantee anything here no?

I just mean replacing the nan values with a common nan value (or a sentinel value as @mdhaber suggested). Something like a[np.isnan(a)] = np.nan. This will convert all the nan values to a common representation.

Are you saying we should use the binary representation to catch all kind of nan? It seems quite difficult to guarantee anything here no?

@tupui I just mean replacing the nan values with a common nan value (or a sentinel value as @mdhaber suggested). Something like a[np.isnan(a)] = np.nan. This will convert all the nan values to a common representation.

This should convert all the nan values to a common representation.

I tried that. I don't think it did.

I tried that. I don't think it did. This is part of the problem. NumPy's behavior regarding NaNs is weird.

Why not keep using np.unique since it treats different types of nans as equivalent and gives expected output:
>>> import numpy as np
>>> zero = np.float64(0)
>>> my_nan = zero / zero
<stdin>:1: RuntimeWarning: invalid value encountered in double_scalars
>>> a = [1, np.nan, 2, my_nan]  # different types of nans
>>> np.unique(x, return_counts=True)
(array([ 1.,  2., nan]), array([1, 1, 2]))

So here's what I think I'm hearing:
For numerical arrays, treat all NaNs as the same.
For object arrays, treat NaNs as objects - different representation NaNs are different.
Is that right?

Yes, that'd simplify things and lead to intuitive behavior.

Why not keep using np.unique since it treats different types of nans as equivalent and gives expected output:

I think it is either version or machine dependent. That was the original plan (see mdhaber/scipy#69). I switched to my Mac, ran tests before submitting the PR, and got failures. That's why I added if contains_nan: <convert to object array>. But on this machine, it's working fine when leaving them as floats.
I'll try leaving them as floats and seeing if tests pass on CI.

Yeah, it seems that np.unique is not consistent across platforms/versions.
Here's the problem, stated concisely:

When an input sequence is converted to a numerical array, NaNs in the input with identical representations can turn into NaNs with different representations
On some platforms/versions, np.unique treats NaNs with different representations as distinct.

My fix was to convert the input to an object array.

When the input sequence is converted to an object array, the NaNs maintain their original representation. This is the heart of the fix: NaNs that start with identical representations stay identical.
Even though Counter treats different NaN representations as distinct, all the NaNs in the input of the tests have the same representation, so they are considered to be the same.

If we really want this to treat all NaNs as identical on all platforms/NumPy versions, I think we have to use a sentinel value.

If we really want mode to treat all NaNs as identical on all platforms/NumPy versions, I think we have to replace NaNs with a sentinel value. Do we want to go that route?

You noted that for object arrays, we need to go element by element which sounds pretty bad for large arrays. Do you think we can replace all nan values with a sentinel and then run np.unique? This way we can just avoid coercing the inputs to object arrays.

For numerical input, that's the plan. For general object arrays passed as inputs, no. mode has supported object arrays in the past, so we're stuck with them unless we want to deprecate that use.

@tirthasheshpatel @tupui I looked into this a bit and there was indeed a change in np.unique behavior between 1.20 and 1.21 that explains the test failures in the current PR.
I'd suggest we:

Keep the logic as-is. It sounds like the new behavior (the one we agree with) will continue to be the default.
Document the difference in behavior depending on NumPy version
Check NumPy version in the test and xfail for old NumPy
Merge this
Discuss deprecation of non-numerical array support separately

How does that sound?

Sounds good to me, as you showed, it look like NumPy discussed this in length so it would make sense to follow their conclusions and rely on their default.

How about you, @tirthasheshpatel? If you agree, I think this one is ready.

Thanks for digging into NumPy's changes and issues, @mdhaber!

How does that sound?

Sounds good to me too!

you're welcome to open an issue about deprecating non-numerical array support for mode.

Done: #15551


eclipse plugin doesn't support multi module sample project

Current Behavior
In my project I use a layout like described in: https://docs.gradle.org/current/samples/sample_java_modules_multi_project.html
I am trying to create an eclipse project for this but it fails.
No .classpath file is created. Only an 'empty' .project
Expected Behavior
Run 'gradlew eclipse' and start editing/developing the sourcecode in eclipse
Context (optional)
No response
Self-contained Reproducer Project
see https://docs.gradle.org/current/samples/sample_java_modules_multi_project.html
Gradle version
Gradle 8.11
Build scan URL (optional)
No response
Your Environment (optional)
No response

Your issue appears to be a question about how to use Gradle. They should be asked in a different channel, so we will close it. Your question might be answered in Gradle documentation or on our community Slack or forums. Find links to these resources on help.gradle.org.
If you think our analysis is wrong, please provide us with more detailed information explaining why.

You should use buildship to import the project into eclipse.
You should only apply the eclipse plugin if you have customization to the eclipse import.

Applying that plugin to the root project is most likely not what you want since this is not a java project, it should be applied to the project needing the customization or through the convention plugin of the sample.


Fix typo and delete unnecessary spaces

typo: Github -> GitHub
delete unnecessary spaces: As Markdown syntax, there was a large amount of half-width spaces that did not affect the display, so I deleted it

@changeworld : Thanks for your contribution! The author, @georgewallace, has been notified to review your proposed change.

#sign-off Thanks for the contribution, it looks good. Thanks!


Show User Avatars in email or email list

Similar to what Sparrow or Gmail does

Makes it much simpler to scroll through the Inbox. Sources can be Gravatar, Facebook and Twitter. The image of the sender is shown.

:+1:

:+1:

:+1:

@smacking has suggested a design:
Current Design: http://cl.ly/0m2K2Y2l0S3l
My Suggestion: http://cl.ly/0t2f090P121y
Would be great to find a good source of avatar icons for this UI. We've tried using Gravatar in the past, which works alright but often returns pictures people consider "older" than their latest Facebook / Google profile image. Anyone know of other sources we could use?

@bengotow That looks amazing!
I haven't really worked with the Facebook API yet, but it should be possible to get the pictures of Facebook friends and even "strangers" with http://stackoverflow.com/a/10360054/445598.
The first time you launch Sparrow it will ask you to auth with Facebook for that very reason.

@KrauseFx Airmail also uses on top of that the Favicons from Websites and checks if there is a Gravatar user registered with the same email.

Linked in and twitter would be good sources if doable via the API. Checking address / contract book, then linkedin then twitter then gravatar then a fall back default N1 / initials based icon might be a nice cascade option.

I currently use Airmail on mac and this is one feature that I really find important. It's not a matter of having eye candy; it simply helps you parse messages faster. As @foxgeere just said, cascading avatar curation is really nice and with the publicly accessible APIs available, shouldn't be too hard to implement.

+1 It helps to parse the incoming messages much faster.

Would love this too. As it is now, where N1 shows the name for every participant in a thread, it makes it very difficult to quickly go through my inbox list. I kinda get the felling that I need to read each and every message whereas having a profile picture makes it much faster to pick individual messages from the list.

Single most useful feature for any email client. Please, yes!

:+1: Really looking forward to this in a future release! As @KrauseFx and @herrhaase mentioned, Facebook integration followed by Gravatar and favicon lookups would probably do the trick.

Clearbit offers this for company logos. Could be useful as a fallback if a person avatar can't be found, e.g. https://logo.clearbit.com/facebook.com

+++1
This feature is really a must have (among others I guess).
Any status update?

+1!
This is the only thing keeping me from switching. It makes parsing my inbox flood so much easier.

+1

+1

I'd like them round like face2face / Mail Butler

I've built a quick and dirty plugin that seems to have been working fine from me on a basic level for the past 2-3mo. Here it is. https://github.com/unity/n1-avatars - I'd welcome help to polish and release properly!
There's a few things that I'd love to see in the API to help with caching lookups. I'll post here a bit later

👍

Airmail Style Please:


Small ES6 tweaks to Xen.

GitHub hid my comment in 'outdated' under xen-state.js, in any case, I think it's worth merging the bits that remain here.


Apiv2 Last Mile: Well-defined entrypoint for jupyter

Add an entrypoint usable from Jupyter (or interactive python shell) for running a protocol. There should be one for both protocol apis 1 and 2.
This should probably take the form of something that returns an instantiated protocol context that can have methods called on it. It should connect to the hardware controller using #3545 .

Proposal
Since we are forgoing access to the hardware controller via a jsonrpc client/server for now, we can simply use a normal entrypoint input.

All tools requiring entrypoints (deck calibration/commandline, simulation, jupyter) will utilize a shared protocol context function.
For interactive protocols (i.e. python shell and jupyter notebook) there will be a protocol context for both simulation and connecting to the actual hardware.
For non-interactive protocols (i.e running a protocol from the commandline) we will simply utilize a fully connected protocol context.
When it's time for backcompat, calls to instrument/labware will always be the simulation context until someone runs robot.connect() and then it will be a regular ProtocolContext.

Now that locking is removed as of #3962, we don't have to worry about strictly managing protocol contexts like before.
Jupyter Notebook Future
If we decide that we want to build out the features of jupyter notebook, we could try to implement https://github.com/ipython-contrib/jupyter_contrib_nbextensions.
This way users wouldn't need to explicitly specify a simulating context and a protocol context. They could instead switch between the two with some type of GUI shown in the README above.
Notes
Since interactive protocols will not be utilizing a run method, we'll need to make sure we are very explicit about the differences in the docs.


Capture video reported as failed

Thanks for such a simple (and powerful) tool!
I noticed when trying to use --capture-video-as-attachment with a Vimeo video that it is reported as not working: [09:18:31] WARN STEP [8/10]: Out-of-browser capture of video as attachment (if any) - failed.  I tried it by telling scoop where yt-dlp was installed, which worked independently. Is it perhaps hitting a timeout of some kind? I'm using v0.6.5.
$ scoop --yt-dlp-path /Users/edsummers/.pyenv/shims/yt-dlp --capture-video-as-attachment true https://vimeo.com/453850086
[09:17:55] INFO Capture-specific temporary folder /Users/edsummers/.nodenv/versions/18.11.0/lib/node_modules/@harvard-lil/scoop/tmp/31KcXh/ created.
[09:17:55] INFO TCP-Proxy-Server started {"address":"::1","family":"IPv6","port":9000}
[09:17:55] INFO User Agent used for capture: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.6167.57 Safari/537.36
[09:17:56] INFO Scoop 0.6.5 was initialized with the following options:
[09:17:56] INFO {
  logLevel: 'info',
  screenshot: true,
  pdfSnapshot: false,
  domSnapshot: false,
  captureVideoAsAttachment: true,
  captureCertificatesAsAttachment: true,
  provenanceSummary: true,
  attachmentsBypassLimits: true,
  captureTimeout: 60000,
  loadTimeout: 20000,
  networkIdleTimeout: 20000,
  behaviorsTimeout: 20000,
  captureVideoAsAttachmentTimeout: 30000,
  captureCertificatesAsAttachmentTimeout: 10000,
  captureWindowX: 1600,
  captureWindowY: 900,
  maxCaptureSize: 209715200,
  autoScroll: true,
  autoPlayMedia: true,
  grabSecondaryResources: true,
  runSiteSpecificBehaviors: true,
  headless: true,
  userAgentSuffix: '',
  blocklist: [
    '/https?://localhost/', '<IP_ADDRESS>/8',
    '<IP_ADDRESS>/8',           '<IP_ADDRESS>/10',
    '<IP_ADDRESS>/8',          '<IP_ADDRESS>/16',
    '<IP_ADDRESS>/12',        '<IP_ADDRESS>/29',
    '<IP_ADDRESS>/24',         '<IP_ADDRESS>/24',
    '<IP_ADDRESS>/16',       '<IP_ADDRESS>/15',
    '<IP_ADDRESS>/24',      '<IP_ADDRESS>/24',
    '<IP_ADDRESS>/4',          '<IP_ADDRESS>/4',
    '<IP_ADDRESS>/32',   '::/128',
    '::1/128',              '::ffff:0:0/96',
    '100::/64',             '64:ff9b::/96',
    '2001::/32',            '2001:10::/28',
    '2001:db8::/32',        '2002::/16',
    'fc00::/7',             'fe80::/10',
    'ff00::/8'
  ],
  intercepter: 'ScoopProxy',
  proxyHost: 'localhost',
  proxyPort: 9000,
  proxyVerbose: false,
  publicIpResolverEndpoint: 'https://icanhazip.com',
  ytDlpPath: '/Users/edsummers/.pyenv/shims/yt-dlp',
  cripPath: '/Users/edsummers/.nodenv/versions/18.11.0/lib/node_modules/@harvard-lil/scoop/executables/crip'
}
[09:17:56] INFO 🍨 Starting capture of https://vimeo.com/453850086.
[09:17:56] INFO STEP [1/10]: Out-of-browser detection and capture of non-web resource
(node:7913) ExperimentalWarning: The Fetch API is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
[09:17:56] INFO Requested URL is a web page
[09:17:56] INFO STEP [2/10]: Wait for initial page load
[09:17:58] INFO STEP [3/10]: Capture page info
[09:17:58] INFO STEP [4/10]: Browser scripts
[09:17:59] INFO STEP [5/10]: Wait for network idle
[09:18:00] INFO STEP [6/10]: Scroll-up
[09:18:00] INFO STEP [7/10]: Screenshot
[09:18:00] INFO STEP [8/10]: Out-of-browser capture of video as attachment (if any)
[09:18:31] WARN STEP [8/10]: Out-of-browser capture of video as attachment (if any) - failed
[09:18:31] INFO STEP [9/10]: Capturing certificates info
[09:18:35] INFO STEP [10/10]: Provenance summary
[09:18:35] INFO Closing browser and intercepter
[09:18:35] INFO TCP-Proxy-Server closed
[09:18:35] INFO Clearing capture-specific temporary folder /Users/edsummers/.nodenv/versions/18.11.0/lib/node_modules/@harvard-lil/scoop/tmp/31KcXh/
[09:18:35] INFO Exporting capture to WACZ
[09:18:36] INFO 1 WARC(s) to process
[09:18:36] INFO Initializing output stream at<EMAIL_ADDRESS>[09:18:36] INFO Initializing indexer
[09:18:36] INFO Indexing WARCS
[09:18:36] INFO Harvesting sorted indexes from trees
[09:18:36] INFO Writing CDX to WACZ
[09:18:36] INFO Writing pages.jsonl to WACZ
[09:18:36] INFO Writing WARCs to WACZ
[09:18:36] INFO Writing datapackage.json to WACZ
[09:18:36] INFO Writing datapackage-digest.json to WACZ
[09:18:36] INFO Finalizing WACZ
[09:18:36] INFO WACZ was finalized
[09:18:36] INFO /Users/edsummers/archive.wacz saved to disk.

Here is the output of yt-dlp which seems to work, but as you can see it runs for a while...
$ /Users/edsummers/.pyenv/shims/yt-dlp https://vimeo.com/453850086
[vimeo] Extracting URL: https://vimeo.com/453850086
[vimeo] 453850086: Downloading webpage
[vimeo] 453850086: Downloading JSON metadata
[vimeo] 453850086: Downloading JSON metadata
[vimeo] 453850086: Downloading jwt token
[vimeo] 453850086: Downloading JSON metadata
[vimeo] 453850086: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 453850086: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 453850086: Downloading fastly_skyfire m3u8 information
[vimeo] 453850086: Downloading fastly_skyfire m3u8 information
[vimeo] 453850086: Downloading google_mediacdn m3u8 information
[vimeo] 453850086: Downloading google_mediacdn m3u8 information
[vimeo] 453850086: Downloading akfire_interconnect_quic MPD information
[vimeo] 453850086: Downloading akfire_interconnect_quic MPD information
[vimeo] 453850086: Downloading fastly_skyfire MPD information
[vimeo] 453850086: Downloading fastly_skyfire MPD information
[vimeo] 453850086: Downloading google_mediacdn MPD information
[vimeo] 453850086: Downloading google_mediacdn MPD information
[info] 453850086: Downloading 1 format(s): hls-google_mediacdn-1602
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 565
[download] Destination: GRASSROOTS RISING by Robert C. Winn [453850086].mp4
[download]  64.2% of ~ 661.98MiB at    1.12MiB/s ETA 04:17 (frag 364/565)

Hey there @edsu ! Thank you very much for the detailed report!

I can confirm that you likely ran into a timeout issue! I'd suggest adjusting settings accordingly:
scoop "https:// url of page containing long video" --capture-video-as-attachment-timeout=1800000

By default, Scoop ignores capture-level time and size constraints for attachments (--attachments-bypass-limits defaults to true). This allows you to fine-tune timeouts for attachments, independently of how the main "web" capture process goes.
You likely won't need to adjust anything else: that said, I'd recommend setting --max-capture-size and --capture-certificates-as-attachment-timeout to higher limits so you see less warnings.
scoop "https:// url of page containing long video" --capture-video-as-attachment-timeout=1800000 --max-capture-size=1000000000 --capture-certificates-as-attachment-timeout=180000

I tried it by telling scoop where yt-dlp was installed, which worked independently

Unless you want to use a specific executable for yt-dlp that is not the one Scoop pulls on install, passing --yt-dlp shouldn't be necessary. If Scoop cannot find yt-dlp, it will yell at you with a message along those lines:
ENOENT: no such file or directory, stat '/path/to/scoop/executables/yt-dlp'

Would you mind letting me know if you did see that message originally?

Finally, if you encounter further issues, I'd suggest passing --log-level=trace so we have more info about what went wrong 😄 .
Cheers 👋

Yes, it was the timeout, thanks!


How can I write from_json for a complex datatype that is not default constructible?

Hi. This is more of a question rather than a issue.
I have a requirement where I need to create objects from serialized json (serialized using this library only) but those types are not default constructible. I went through the section regarding get() for non-default constructible/non-copyable types but wasn't sure as to how am I supposed to use it for a complex datatype.
Can you please help?

 What did you try?
Basically you have to open up the namespace nlohmann in your own code, as shown in the section you linked:
namespace mylib
{
struct complex_type
{
  complex_type() = delete;
};
}

namespace nlohmann
{
template <>
struct adl_serializer<mylib::complex_type>
{
  // ... proceed as shown in the section
};
}

Thanks a lot. I was able to get it to work.


import error no module tkinter

When I run python buy_and_hold_strategy.py   I ran into the problem
ImportError: No module named '_tkinter'
and could not run the backtest
please help

Is the module installed?
Did you install  with/without sudo x....?
http://tkinter.unpythonic.net/wiki/How_to_install_Tkinter

Are you running this on Windows, Mac or Linux?


controlTransfer Linux always returns LIBUSB_ERROR_IO where MIDI 2.0 USB is in use

Bug Description:
I am working with MIDI 2.0 USB Devices. MIDI 2.0 gets Group Terminal Blocks using Control Transfer.
When attempting this on Linux I receive a LIBUSB_ERROR_IO message. In OSX this same code works well.

Steps to Reproduce:

Run an update Linux Kernal and ALSA with MIDI 2.0 Support and connect a MIDI 2.0 USB Device. You can create a Linux Gadget to achieve this.
Find the device and open
do the following :

dev.controlTransfer(0x81, 0x06, 0x2601, interfaceId, 0x5,
                (error, data) => {
                    if(error){
                       //  This will always error with LIBUSB_ERROR_IO on Linux where driver is claimed by MIDI 2.0 Driver
                    }
});

I believe this is/was an issue in libUSB that has been resolved(?)
Issue does not occur in OSX. Does not occur in Linux where I detach the driver and reattach manually.
Additional Information

Operating System: RPI running kernel 6.2
Runtime (node -v or electron -v): electron 19.1.9
Node USB Version (npm ls usb): 2.12.1

As you are on Linux, I assume your udev rules are fine?

Yes I believe so:
groupadd usbusers
usermod -a -G usbusers $USER
echo 'SUBSYSTEM=="usb", MODE="0666", GROUP="usbusers"' | tee /etc/udev/rules.d/99-usbusers.rules
udevadm control --reload
udevadm trigger

On other systems where I dettch from the kernel the controlTransfer works.

Any more detail on this? e.g. a PR or fixed issue we can check has made it into the release being used by node-usb.

Sorry I did read a bunch of tickets for libUSB to find what I thought had been fixed in 1.0.27 however it seems that they are windows related.

I don't know a lot about this, but I wonder if the kernel is taking control of the device as it has a driver installed?

IIRC, detaching the kernel driver is successful, but even then, you still get LIBUSB_ERROR_IO on transfers?

IIUC, detaching the kernel driver is successful, but even then, you still get LIBUSB_ERROR_IO on transfers?
I can dettach from the kernel successfully. If I then do a ControlTransfer it is successful.

If I do not dettach from from the Kernel on OSX, and then attempt a Control Transfer it is successfull.
If I do not dettach from from the Kernel on Linux, and then attempt a Control Transfer it is not successfull. :)
I hope I am clear?

hmm, so your point is that control transfers should work even if you don't detach the kernel driver? I'm not sure about that, it looks reasonable to me that the kernel driver reserves the device for itself.
in any case, I doubt this issue has anything to do with node-usb, at most it would be relevant to libusb


Steemia Privacy Policy

Privacy Policy document details how a company stores and make use of user/customer information; its safety and disclosure. It helps to instill user confidence about the platform they are visiting/using

Hello,
I've read your changes and it looks fine. Thanks for your contribution, we really appreciate your effort in make the platform better.
Merged!


3.0 Breaking Changes

I would like to discuss whether for 2.1 we should
a) Disable a new instance per test by default (so all the tests in a suite are a single instance of that class, which is the case with junit, spek, scalatest, specs2 etc). #163 #174
b) Disable project scan by default. #226 #225 #219
c) Upgrade to jUnit 5.1 #156  #187
d) Change generators to return an Iterable rather than a single instance #212
e) Your requirement here...

First, I'm a proponent of semantic versioning and would raise the version to 3.0, if we would introduce breaking changes.
a) I'm still thinking that a new instance per test is a good thing to isolate tests. Isolated tests are helpful on parallel execution, too. The problem mentioned in #163 would not be solved, if we would switch the default, since it would only mask the root cause.
To #174: I commented directly on the issue. I think onInstacePerTest=false wouldn't solve the problem.
b) Could probably be solved enforcing a certain class name for config.
c) ok
d) ok

@helmbold Let's go semver then, and bump to 3.0.0.
This means we can also be a little bit more adventurous in anything we like. One idea I was thinking, was that we could replace all specs with a "do what you want spec", where you have a preconfigured list of keywords - should, test, given, etc, and you can just lay them out like you want. The nesting is up to you, only the final String.invoke method is where the test is executed.
We should also add future based matchers. These will simply do the blocking for you in the test runner, so you can do things like, myAsyncFun("foo") shouldBe "bar" and myAsyncFun returns Future<String> in this case.
I'm also going to look at junit 5 see what new features it would allow.

a) What is the reason behind proposition to disable a new instance per test by default? If kotlintest wants to be easy to understand for JUnit users, it should create new instance for every test. This is how it works in JUnit 5 and this how it worked before - http://junit.org/junit5/docs/current/user-guide/#writing-tests-test-instance-lifecycle. Also this is not the cause of #174 as I've already commented on that issue.
c) Sure, but do you know how would that influence Android developers? Will this require any additional configuration?
d) Sounds good. Possibly it might not require to change Gen interface to not break too many things?

@michalbrz I didn't realize junit was one instance per test. Maybe I've been doing Scala too long. @helmbold did you know this ?

@sksamuel Yes, I was aware of JUnit using one instance per test by default. Test parametrization in JUnit 4 even only works that way (or at least, parallel test execution would be impossible otherwise).

3.0 keeps one instance per test.
Project scan now disabled.


Added Avaritia Ingots.

Included:
Crystal Matrix Ingot + Animation
Infinity Ingot
Neutronium Ingot

pog

Thanks🙇♂


Select 2 - Problem displaying Reslults of AJAX Request

Hello Everyone,
I opened this issue because i have a little mind brooking problem.
I wrote a Script that make a Search over a LDAP Server to get the Users Full name and User ID.
Always when i start the AJAX request, i get the results but they are not displaying.

For Example here is my JS Code
$(".js-data-example-ajax").select2({
     ajax: {
     url: "<?php echo $url ?>/bin/ajax/ldap_search.php",
     dataType: 'json',
     delay: 250,
     data: function (params) {
          return {
              q: params.term // search term
          };
      },
      processResults: function (data) {
          console.log(data);
          return {
                results: data
          };
       },
      cache: true
      },
     minimumInputLength: 4
});

And this is my JSON Respnse
["Mike Crawford (NGIE113)", "Mike Preuss (NGIB956)", "Mike Weiller (NEN7992)", "Mike Theissen (AA10101)", "LDA1231 (lda1231)", "LDA1232 (lda1232)", "Mike Manant-CONT (ZENE413)", "Mike Breyer (NEND861)", "Mike Peiffer (AA05391)", "Mike Scholtes-CONT (ZA26077)", "Mike Peters (AA43681)", "Mike Mossong (AA28172)", "Mike Padilla (AA76273)", "Mike Denis-CONT (ZA34482)", "Mike Klein (AA82171)", "Mike Mathieu (AA91973)"]
But Somehow when i type nothing displays but when i check the console log the data is there.

[x] Tried to add success and results instant of  processResults but without success
[x] Tried to add formatResult: formatResult, formatSelection: formatSelection, initSelection: initSelection but without success
[x] Tried to add formatResult: ,formatSelection: but the problem is still there

I hope someone can help me because it's really frustrating
Thank you in advance

ajax is totally broken in 4+ - I couldn't get it to work either.

Hi dear community,
I found a resolution for this problem, the problem was like every problem between the keyboard and the chair ;p,
I haven't defined the function for templateResult and templateSelection. This Fiddle helped a lot http://jsfiddle.net/59p19q7z/3/

ajax is totally broken in 4+

@ciprianmicutaru this is simply not true.  You just need to transform your data appropriately (see https://select2.org/data-sources/formats), or use templating (https://select2.org/dropdown and https://select2.org/selections).
Glad you got this figured out.


DO NOT MERGE: Testing build solutions

@sbello I was just looking at some stake branches in MP and so this.. Shall we close this and delete it (check diff)? Its something you did a couple of years ago.

Closed now, I dont think this is, or should be, relevant any more.


Whether api-v1 will go offline in the future is no longer supported

Hello, at present, the official version of api-v2 has been released, we found that v2 is quite different from v1, for our development team, the amount of code we need to modify will be larger. I want to know Whether api-v1 will go offline in the future is no longer supported?

Good question. This was briefly mentioned in the v2 release announcement, but I can explain more here.
I will keep supporting v1. It will continue to get bug fixes, compatibility updates, and security patches. However, v1 probably won't get new features. You can use v1 and v2 together and slowly move to v2.
I will support v1 for a reasonable amount of time unless one of these happen:

The number of v1 downloads stays very low for a long time. This would likely mean people are not using it much anymore.
A future pfSense update makes v1 incompatible and fixing it would require big changes.

I will also open polls in the future to see how many people are still using v1 before deciding to stop supporting it.
Hope this helps.


When reading from a large file ( > 300MB ). The rtspatt will tack up all the memory (4GB)

Probably it's because rtspatt decode entire file to memory before do anything? It there a way to workaround this?

Hi @ingjieye, I guess the only solution is either to allocate more memory or reduce the size of the file by compressing it (you can achieve this easily using ffmpeg).
Unfortunately, I'm afraid there's nothing else that can be done to fix this issue, since that's how gstreamer works, which is what is under the hood of RTSPATT :/


Dependencies from *.csproj file prevent successful build

Describe the issue or request
IMPORTANT NOTE: the linked branch has updated cadl that does not work with the emitter. If you checkout this branch make sure to use the cadl from this  commit
USEFUL NOTE: You can find the fully generated folder here
Attempting to generate an SDK for OpenAI Inference (link to branch) generates a project with a csproj file that doesn't seem to be correctly setup for the repository.
The dependency in question is Azure.Core doesn't seem to be correctly imported.  Microsoft.Azure.AutoRest.CSharp
The generated file looks like so:
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
    <Nullable>annotations</Nullable>
  </PropertyGroup>

  <PropertyGroup>
    <LangVersion>9.0</LangVersion>
    <IncludeGeneratorSharedCode>true</IncludeGeneratorSharedCode>
    <RestoreAdditionalProjectSources>https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-net/nuget/v3/index.json</RestoreAdditionalProjectSources>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Azure.AutoRest.CSharp" Version="3.0.0-beta.20221201.1" PrivateAssets="All" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Azure.Core" Version="1.25.0" />
  </ItemGroup>

</Project>

Describe your ideas for solutions
The target is to generate an SDK for this repository. The main dependency we need to use is Azure.Core but the generated file does not seem to match other SDKs using Azure.Core. Other SDKs setup seems different

duplicate of #2885

We already support to create compliable project file in scaffolding folder creation. close it.


🐛 [OFFICIAL] Generated Issue: TypeError

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
The API has returned an error:
TypeError
Status code: 0
Message: Cannot assign string to property App\Dto\SearchCommand::$page of type Spatie\LaravelData\Optional|int
Trace: /var/www/jikan-rest/vendor/spatie/laravel-data/src/Resolvers/DataFromArrayResolver.php on line 49
Expected Behavior
The API should have returned a successful response with data.
Steps To Reproduce
Http Request: `GET /v4/anime?sfw=&unapproved=&page=&q=&sfw=&genres=&genres_exclude=&letter=&producers=&start_date=2001-01-01&end_date=2010-01-01
Environment
No response
Anything else?
No response

Probably we need to upgrade LaravelData package. The following PR might be a solution: https://github.com/spatie/laravel-data/pull/602
Otherwise some global transformer would be required, because all the endpoints are affected by this.
I mean if you set an empty param, you get an error message instead of treating it empty/non-existent.
@nadla20xx For now I can only recommend not passing empty params to the API.

Found the source of the issue, it wasn't the laravel data package version. :tada: