download the official pkg in case you need those

Results obtained from curl https://www.sony.net/Products/DigitalPaperSystem/Software/DP/info.xml
As of the day this is created:
For DPT-RP1:
URI="https://www.sony.net/Products/DigitalPaperSystem/Software/DP/FwUpdater.pkg"
Version="<IP_ADDRESS>00"

For DPT-CP1:
URI="https://www.sony.net/Products/DigitalPaperSystem/Software/DP/FwUpdater_<IP_ADDRESS>61.pkg"
Version="<IP_ADDRESS>61"

Modded official PKGs please refer to https://github.com/HappyZ/dpt-tools/wiki/PKGs-I-Made/.

Thank you very much! But there was a bit more problem ...
I downloaded the official firmware, entered the diagnosis mode, ran 'official_pkg_unpacker_pkg.sh' to unpack it, copied the 'system.img' into mmcblk0p16.  When I ran md5sum, I got 'a35d6d6697b637184aeb41b4f356ff95', instead of '5081ec24954714373229a7c76154509a
' you mentioned.
I repeated the whole process and got the same md5 value. So, what's going on ?

@Zack-Q you shall compare the system.img with the system.img.md5 inside your downloaded official pkg.

Yeah, they don't match. But boot.img and boot.img.md5 match. It's weird. I download system.img from 'https://github.com/jbschooley/DPTstuff/blob/master/README.md', which matches with the unpacked one. I'm trying to download them again.

I give up. Try the unpacked system.img even it doesn't match with its system.img.md5.

@Zack-Q  ahha, I see the confusion parts. The system.img.md5 is the md5 of the image after extraction (since system.img is a sparse image):
md5sum -b /dev/mmcblk0p9 | cut -d " " -f1

The official system.img for DPT-RP1 should have a35d6d6697b637184aeb41b4f356ff95.

Thanks for your patience. I succeeded in flashing the official system.img. But after rebooting, it prompts Unfortunately, Setting has stopped. So I fw the official FwUpdater.pkg again, but the problem is still unsolved. And dpt doesn't responds when press the home button.

Solved, adb removes the /data/system/{ExtMgr.db,ExtMgr.db-journal}, everything back to normal. Appreciate your help again, really!

I see. You need to get adb logcat so I know what is going on. Without it, I can only guess it is due to some caching issues. Try in adb shell, and step 5 in Launcher app under 0x4 section.

BTW here's some already unpacked boot.img and system.img with md5 sums if you prefer:
https://github.com/jbschooley/DPTstuff/tree/master/FW <IP_ADDRESS>00 Stock

@HappyZ Here is How I damage my system:
My device is DPT-CP1, version <IP_ADDRESS>61
I try to install kindle apk to /system/app, and I push apk's native libs to /sytem/lib. Then I can not boot CP1, it reboot repeatly!
Anyway, Thanks @HappyZ , I will follow your advice to try to recover the system partition.

This is weird. I would be curious in the logcat before rebooting.
What happens if you just use adb to install the Kindle app?
On Fri, Dec 14, 2018, 07:47 hitmoon<EMAIL_ADDRESS>wrote:

@HappyZ https://github.com/HappyZ Here is How I damage my system:
My device is DPT-CP1, version <IP_ADDRESS>61
I try to install kindle apk to /system/app, and I push apk's native libs
to /sytem/lib. Then I can not boot CP1, it reboot repeatly!
Anyway, Thanks @HappyZ https://github.com/HappyZ , I will follow your
advice to try to recover the system partition.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/HappyZ/dpt-tools/issues/31#issuecomment-447330151,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEdSKgGT-Ulm1c6udcF-51Vy7ypcLgihks5u46vggaJpZM4ZQR6F
.

@HappyZ
use adb to install the kindle app is success, but the app is sit on /data/ by default and I can not launch it from the Launcher.

Really Really thank you guys @HappyZ @Zack-Q and @jbschooley, My CP1 is back to life again!

Even if you remove the app it still reboots?
On Fri, Dec 14, 2018, 08:35 hitmoon<EMAIL_ADDRESS>wrote:

@HappyZ https://github.com/HappyZ
use adb to install the kindle app is success, but the app is sit on /data/
by default and I can not launch it from the Launcher.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/HappyZ/dpt-tools/issues/31#issuecomment-447343149,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEdSKqH4kYx2I0AU-WOR4kprVacpOrWrks5u47cegaJpZM4ZQR6F
.

Even if you remove the app it still reboots?
…
On Fri, Dec 14, 2018, 08:35 hitmoon @.*** wrote: @HappyZ https://github.com/HappyZ use adb to install the kindle app is success, but the app is sit on /data/ by default and I can not launch it from the Launcher. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AEdSKqH4kYx2I0AU-WOR4kprVacpOrWrks5u47cegaJpZM4ZQR6F .

Yes, I think I pushed some libs which corrupt the system's original ones, and cause it misfunction to boot normally.
By the way, I have some knowledge about linux and android system, If needed, I am ready to help.

Or, equivalently, you can use the dpt-tools.py in diagnosis mode.
Just use restore-systemimg and follow the instructions.

When it asks Is it a sparse image?, if you get it from official pkg, then answer yes.
If you use system.img from taobao pkg, then answer no.
Otherwise, ping me.

urgent!
I use system.img from official pkg, but answer no!
(Then restore the previously backed up files bootimg.bak and update script in this same diagnosis interface)
Now I can't enter the desktop, and can't enter the Big Square, it's only three small long square on the up-left corner.

well.. you probably overwrote your system partition with the sparse image which is incorrect..
you should still be able to get into diagnosis mode: before the third bar appear, press the HOME button once. It should directly go to diagnosis mode - if you could get into diagnosis before.

I can't use python3 dpt-tools -ip xxx connect dpt, how do I control >>> diagnosis through my computer ?

Just do python3 dpt-tools.py --diagnosis directly
On Tue, Feb 19, 2019 at 10:34 PM yefeiyu<EMAIL_ADDRESS>wrote:

I can't use python3 dpt-tools -ip xxx connect dpt, how do I control >>>
diagnosis through my computer ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/HappyZ/dpt-tools/issues/31#issuecomment-465419458,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEdSKl5ZksZ0yhQuxOeUlczvp3n8hHtPks5vPNBJgaJpZM4ZQR6F
.

--
BR | Yanzi Zhu
https://happyz.me

Can't get into diagnosis mode whatever I press the HOME button.:-(

Looking forward to your guidance.

Can't get into diagnosis mode whatever I press the HOME button.:-(
There are only three gray rectangular squares in the upper-left corner. I connected Mac to use diagnosis, the result shows:
(And it did not find the COM port when I connected the PC.)

if you have the bar appearing, then you should be able to get into diagnosis mode..
you need to find COMx in your windows and input that when it asks for serial port. /dev/tty.usbmodem01 isn't correct..

I tried 3 computers, Mac, Win10, XP. and I had select "Show hidden devices" in Win10 and XP, but the port lists remains unchanged no matter how I plugged into the USB line or not.
I used an MacBook before, when I use --diagnosis the >>>Enter the serial port I just press Enter to select the default port.
Today I tried /dev/tty.usbmodem01 and other port numbers such as 02, 03, 1, 2, 3, they all got same error message.
>>> Enter the serial port [/dev/tty.usbmodem01]: /dev/tty.usbmodem1
[error] Cannot open serial port /dev/tty.usbmodem1 due to [Errno 2] could not open port /dev/tty.usbmodem1: [Errno 2] No such file or directory: '/dev/tty.usbmodem1'

In Mac, are you able to see something similar to it under folder /dev/?
On Wed, Feb 20, 2019, 19:30 yefeiyu<EMAIL_ADDRESS>wrote:

I tried 3 computers, Mac, Win10, XP. and I had select "Show hidden
devices" in Win10 and XP, but the port lists remains unchanged no matter
how I plugged into the USB line or not.
I used an MacBook before, when I use --diagnosis the >>>Enter the serial
port I just press Enter to select the default port.
Today I tried /dev/tty.usbmodem01 and other port numbers such as 02, 03,
1, 2, 3, they all got same error message.

Enter the serial port [/dev/tty.usbmodem01]: /dev/tty.usbmodem1
[error] Cannot open serial port /dev/tty.usbmodem1 due to [Errno 2] could not open port /dev/tty.usbmodem1: [Errno 2] No such file or directory: '/dev/tty.usbmodem1'

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/HappyZ/dpt-tools/issues/31#issuecomment-465824719,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEdSKsTGs56aCzf1U5rSDCF6XjbmjyIXks5vPfargaJpZM4ZQR6F
.

I recently encountered same problem, my Mac can't find the usb connection of my dpt-rp1. I'm not quite sure how I solved this, but you can try this: disconnet dpt-rp1 completely(pull out usb cable from your computer AND dpt-rp1), AFTER dpt-rp1 enter diagnostic stat, then connet.

On the mac I have not seen any changes in inserts and uninserted via lsusb.

Looks like yours stuck at the button check - which fails in potentially two ways:

You press the HOME key too late, try to keep pressing HOME during the boot up (into diagnosis mode)

When you got the image, you said no. So the script uses dd to dump into the img in system partition. Although not very likely, a plausible explanation is the erroneous dump either overwrites the diagnosis initialize script, or maybe affect the tmpfs causing the script failed to recognize the pressed key. If this actually happens, then there is no way of rescuing it. And hopefully you are still within warranty.

I am in China, there is no warranty, it is difficult to hand it to the United States.
Normal boot can see the storage disk, can you upload the brush file to here, and then boot automatically use the files flash dpt?

I am in China, there is no warranty, it is difficult to hand it to the United States.
Normal boot can see the storage disk, can you upload the brush file to here, and then boot automatically use the files flash dpt?

oh! you have the storage disk? great, then download the official PKG I modded at https://github.com/HappyZ/dpt-tools/wiki/PKGs-I-Made. Then just put the PKG file in the storage and rename it to FwUpdater.pkg. Then reboot the device, and it will flash it itself - if the dd does not hurt other things.

I had another FwUpdater.pkg in the storage disk lots of days. But it seems to have never worked. :)
Let's me try.

I had another FwUpdater.pkg in the storage disk lots of days. But it seems to have never worked. :)
Let's me try.

What is that FwUpdater.pkg? If it's the official one, then it will not pass the version check (as your system is the latest). You have to use the one I modded. However, I am not sure why you cannot press the key and then go to diagnosis mode. This sounds suspicious.

I have tried countless times and many ways to push Home key:-(
Now both of these files have been tried, they are not reflected, they are all quietly staying there.

And the storage disk's name is shown as Untitled on Mac, and U盘 on PC,
Which have other's 4 files on Mac:
.fseventsd
.Spotlight-V100
.trashes
System Volume Information

5 files on PC, the above 4 and:
 ._FwUpdater.pkg

If you press HOME key, does it show a solid block top left corner, or it's still three bars?

still 3 bars.

I remember that from the day before yesterday, when I can normally enter diagnosis mode, it changes, at first display 3 bars -> disappear -> and then a rectangular square appears.

I remember that from the day before yesterday, when I can normally enter diagnosis mode, it changes, at first display 3 bars -> disappear -> and then a rectangular square appears.

The normal behavior when you press HOME correctly is: 3 bars -> top left block + center block both appear.
I do not see any possibility the script pauses before HOME button check, only possible after. To confirm, you first turn off the device.
Then you hold HOME key, and pressed POWER key. You released POWER, then yellow light flashes, and you keep holding HOME. Is that correct?

Yes.I did this. Moreover, I have tried all kinds of ways.

Use the OTG line to enter diagnostic mode, is the same method or result entered as the script? Do I have to buy an OTG line to try?

Use the OTG line to enter diagnostic mode, is the same method or result entered as the script? Do I have to buy an OTG line to try?

It won't be helpful as it goes through the same script. Yours apparently stuck at something else, which we don't know what it is yet.

$ adb push LICENSE /sbin
adb: error: failed to copy 'LICENSE' to '/sbin/LICENSE': remote Permission denied
LICENSE: 0 files pushed. 0.5 MB/s (1074 bytes in 0.002s)
$ adb push LICENSE /system/sbin
adb: error: failed to copy 'LICENSE' to '/system/sbin': remote Read-only file system
LICENSE: 0 files pushed. 1.0 MB/s (1074 bytes in 0.001s)
$ adb push LICENSE /system/bin
adb: error: failed to copy 'LICENSE' to '/system/bin': remote Read-only file system
LICENSE: 0 files pushed. 1.0 MB/s (1074 bytes in 0.001s)
$ adb push LICENSE /etc
adb: error: failed to copy 'LICENSE' to '/etc': remote No such file or directory
LICENSE: 0 files pushed. 0.9 MB/s (1074 bytes in 0.001s)

$ adb ls /
000041ed 000003e8 5c6f476b .
000041ed 000003e8 5c6f476b ..
000041c0 00001000 00000000 teesst
000041ff 00000028 5c6f476b tmp
000041fd 00001000 5aa38506 NVM
0000a1ff 00000018 5c6f476a sdcard
000041e9 00000078 5c6f476a storage
00004140 00000028 5c6f476a config
000041f8 00001000 5aa38518 cache
000041ed 00000000 5c6f476a acct
0000a1ff 0000000e 5c6f476a vendor
0000a1ff 00000011 5c6f476a d
0000a1ff 0000000b 5c6f476a etc
000041fd 000000a0 5c6f476a mnt
000081a4 000241f6 5bf14a4c sepolicy
000041f9 000001b8 5c6f476b data
000081e8 0000293e 5bf14a4c init_bsp.pxa1908.rc
000081e8 00000322 5c146b2e init.usb.rc
000081e8 00004b55 5bf14a4c init.pxa1908.rc
000081a4 000002ca 5c141477 adb_keys
000081e8 00004b4b 5bf14a4c init_bsp.rc
000041ed 0000003c 00000000 lib
000081a4 000011df 5bf14a4c ueventd.pxa1908.rc
000081e8 000009b3 5bf14a4c init.tel.rc
000081a4 0000263b 5bf14a4c service_contexts
0000416d 00000000 5c6f476a sys
000041ed 00000028 00000000 system
000081e8 0000012d 5bf14a4c init.zygote32.rc
000081a4 00000ad3 5bf14a4c property_contexts
000081a4 00001170 5bf14a4c ueventd.rc
000081e8 00000205 5bf14a4c init.pxa1908.security.rc
000081e8 000003a4 5bf14a4c init.pxa1908.tel.rc
000041e8 000000b4 5c146bfd sbin
000081a4 00000210 5bf14a4c seapp_contexts
000081a4 0000003d 5bf14a4c selinux_version
000081a4 0000016e 5c14692f default.prop
000081a0 00000c11 5bf14a4c fstab.pxa1908
0000a1ff 0000000d 00000000 charger
000081e8 00000b6b 5c146de3 init.pxa1908.usb.rc
000081e8 00000787 5bf14a4c init.trace.rc
000081e8 000002b4 5bf14a4c init_bsp.pxa1908.tel.rc
000081e8 000003b0 5bf14a4c init.environ.rc
000081e8 00005b11 5bf14a4c init.rc
000041ed 0000003c 00000000 res
000081e8 00005831 5bf14a4c init.pxa1908.sensor.rc
000081e8 00038940 5bf14a4c init
000081a4 00004676 5bfe255f file_contexts
0000416d 00000000 00000000 proc
000041c0 00000028 5acc85e9 root
000041ed 00000c1c 5c6f476b dev

$ adb ls /
adb: error: failed to get feature set: no devices/emulators found

It's lost the connection 5min later without my touch.
And it's seems that the dpt hasn't root permission, so I put the FwUpdate.pkg into storage disk can't work.
Can we use all of the files deviceid.dat   privatekey.dat publickey.dat  workspace.dat which I copied from Digital Paper App (now the App can't connect the dpt properly) on MacOs last time 1month before?
And I have tried all above methods many times, but didn't pay attention to the length of time.
I try again now.

Unfortunately, the adbd I included is a secured version so it has no permission to do any changes. Also since you do not have sh it cannot remount / anyway.. I thought maybe /etc/ or /sbin/ is writeable, but it was incorrect without a proper remount.
Applying FwUpdate.pkg is a totally different process. It should be independent to the Android system. I had a discussion with shankerzhiwu and we could not figure out the root cause where the update script got stuck, based on logical analysis.
You cannot use those files since your system partition is wiped out.. And what exactly did you do after you did restore-systemimg with the wrong command?
The timing is for flashing PKG to go through. It may take a while (minutes) to unpack PKG etc. (for the modded official PKG).

(The following 'termsare written in memory and are not accurate. The sequence of operations is correct.) There have 5 commands in the--diagnosislist, after I didrestore-systemimgwith the wrong command, I executed the following commands in turn:  1, Restore system image (at here I selectnofor official img file). 2, Restore my boot.img.bak file on the storage disk which back up at one month ago. 3, Click theSupper Su, whick told me haven't someting about supper su, failure. 4, Click the update script, succeeded. Reboot, then I can't enter the system of --diagnosis`.

(The following terms are written in memory and are not accurate. The sequence of operations is correct.)
There have 5 commands in the --diagnosis list, after I did restore-systemimg with the wrong command, I executed the following commands in turn:
1, Restore system image (at here I select no for official img file).
2, Restore my boot.img.bak file on the storage disk which back up at one month ago.
3, Click the Supper Su, whick told me haven't someting about supper su, failure.
4, Click the update script, succeeded.
Reboot, then I can't enter the system of --diagnosis.

Although it may not be super helpful in your case, I can only guess in your 4th step the update script was only patched half way through and maybe the rest did not go through?
To confirm if that is actually the case: put the PKG file in the mass storage. Then try to boot into diagnosis mode. Previously you said it just stuck at the three bars, correct? Now before the third bar appear, press the POWER button once (not HOME button). It will delete the PKG ideally, if the script is patched correctly. So now if your press RESET button to turn off the device, and then start to boot into normal system, the mass storage should have nothing in it.
Otherwise, I really got no clues why it happens like this. Sorry about that.

I tried many times, it didn't delete the PKG file.
This issue ends here? I am consider return it to factory, and sorry for the trouble caused to you and shankerzhiwu.

My dpt is on the way back to repair, it is full of blood!
At the begining I asked one of Taobao cracker, who required the same price as crack dpt.
5days before, the second Taobao cracker only need a small amount of money to do this, although his crack is also very expensive. (From the perspective of selfness help, all of us should give HappyZ our respect! Of cause, charge the fee, work hard, and be equally precious)
The first cracker said his method is different with ours, now I guess the second cracker is different too.
At last, May we modify the sentence of  Is it a sparse image? to Is it a sparse (offical) image? or use a popular word:-)

@yefeiyu thanks, and im glad you got your device recovered. I would be very curious how they recover it other than opening it up and de-soldering your emmc.
That's a good advice. I'll change that accordingly.

That's all.
For all of us who are grateful, you should put a donation coffee link. This is a common practice, not for what you get , but for our gratitude.

@yefeiyu much appreciated :)

Dear HappyZ. Thanks you very much for your tool. I cracked it successfully, however I encounter a problem.
I modified ./system/app/AppLauncher/AppLauncher.apk to check if it can change width or numbers of icon.  Moreover I forgot to backup it. Now it shows "AppLauncher Stopped" when I click home button.
I already used system resetting and --diagnosis mode to recovery but this situation still happens.
After I ran  adb logcat | grep "launcher" it shows:
> E/AndroidRuntime( 4902): 	at com.sony.apps.applauncher.presentation.presenter.impl.LaunchMenuPresenter.initialize(LaunchMenuPresenter.java:117)
> E/AndroidRuntime( 4902): 	at com.sony.apps.applauncher.presentation.presenter.impl.LaunchMenuPresenter.<init>(LaunchMenuPresenter.java:37)
> E/AndroidRuntime( 4902): 	at com.sony.apps.applauncher.presentation.service.impl.LaunchViewService.initialize(LaunchViewService.java:122)
> E/AndroidRuntime( 4902): 	at com.sony.apps.applauncher.presentation.service.impl.LaunchViewService.onCreate(LaunchViewService.java:46)
> W/ActivityManager( 2143): Process com.sony.apps.applauncher has crashed too many times: killing!
> W/ActivityManager( 2143): Killing 4902:com.sony.apps.applauncher/u0a14 (adj 0): crash
> W/ActivityManager( 2143): Spurious death for ProcessRecord{38a0a1d 4902:com.sony.apps.applauncher/u0a14}, curProc for 4902: null
> W/BroadcastQueue( 2143): Unable to launch app com.sony.apps.applauncher/10014 for broadcast Intent { act=com.sony.infras.DPExtensions.MenuKeyHandler.MENU_KEY_PUSHED flg=0x10 }: process is bad
> W/BroadcastQueue( 2143): Unable to launch app com.sony.apps.applauncher/10014 for broadcast Intent { act=com.sony.infras.DPExtensions.MenuKeyHandler.MENU_KEY_PUSHED flg=0x10 }: process is bad

May you help me? Or give me a copy of original AppLauncher.apk to have a try? Thank you very much.

@kzmain can you start a new issue instead of commenting on the old one?
here is the original AppLauncher apk and its odex:
AppLauncher.tar.gz

closed as new firmware came out #92

Solved, adb removes the /data/system/{ExtMgr.db,ExtMgr.db-journal}, everything back to normal. Appreciate your help again, really!

I wonder how you managed to get to adb after flashing the official system.img?

Hi HappyZ, I got storage disk in Windows 10 and cannot connect to Digital Paper App.  I tried to copy the modded Official pkg into the storage disk, and restart, but nothing happenend.  Could you please help? Thanks a lot


Add MIGRAINE by BOYWITHUKE

Song Name
MIGRAINE
Artist Name
BOYWITHUKE
Source
Youtube
Youtube Link
tIxLU8WUK1Y
Direct File Link
No response
Song ID
1
Extra
Created from Auto Nong.
Created for level: 96931245

already added


DCMAW-8229: fetch public key and get content encryption key

DCMAW-XXXXX
What changed

Why did it change

Checklists

[ ] There is a ticket raised for this PR that is present in the branch name
[ ] No PII data logged. See guidance here
[ ] Demo to a BA, TA, and the team.
[ ] Update README with any new instructions or tasks

Need to add lambda permission to use kms:decrypt in this PR


feat: Adding AdaBoost Classification

Added AdaBoost Classification with FlAML framework.
Testing
Choosing dataset and Classification ML workflow
Select data:

Here we select column 2 "Label" for Y set, column [3,12] as X set.

Then choose mode2: Classification. Here we can see "AdaBoost" in label 11:

In feature engineering part we get the data.

AdaBoost in non-autoML workflow
Select parameters for AdaBoost. Here we expose 3 hyperparameter in the model: N estimator for how many decision trees we are going to use, Learning rate for the model, max_depth for the depth of the decision trees used in AdaBoost.

Training and collecting result:

AutoML Workflow
Select AutoML here:

Auto tuning:

Final Result:

Choosing dataset and Classification ML workflow

AdaBoost in non-autoML workflow

AutoML

final Result


how to get the labels for own datasets (21 numbers)

@PeterZheFu @btekin
I am still confused with this part, for the label file in LINEMOD, how to get the 18 coordinates?
Currently, I got the Camera intrinsic, mesh models and  the dimension of the my object (Length, Width, Height), I intuitively set the object centroid to be (0,0,0), so I can also get the coordinates of the 8 corners. (For example, Coke Can: L= 6.2cm, W= 6.2cm, H= 10.2cm . If the centriod is (0,0,0), the corners should be P1=(3.1,3.1,5.1), P2=(3.1,3.1,-5.1)... so on  )  Is this intuitive method correct?
If it is correct, How do I project these 8 points on the 2D image? By the way, How to set the x, y range? please give me a help, thank you very much.

Please see: https://github.com/Microsoft/singleshotpose/issues/37#issuecomment-422360436


Sources cleanup

📝 Changed routes:

/api/verify

Commit 93bc6225e00862315aedc5938227f439bab16d06 (https://thirdweb-l9rk3f0md-thirdweb.vercel.app).


Fails on VSCode 1.21.0 / Windows 10

Hi guys!
Just installed the plugin and got a failure with ouput:
 An assembly specified in the application dependencies manifest (mech.deps.json) was not found:
    package: 'FSharp.Compiler.Service', version: '17.0.1'
    path: 'lib/netstandard1.6/FSharp.Compiler.Service.dll'

What other information do you need?

looks like we need to call .net publish

Ok, so we should add a new target on Mechanic repo.
Perhaps we should profit of this to call publish in Release configuration ?

I did that manually as quickfix @whitetigle can you please try 0.0.9?

sure 👍

It works now

ok it works for me 👍

Great job guys! 👏


⚠ Bofrost.es has degraded performance

In 3340461, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1626 ms

Resolved: Bofrost.es performance has improved in b01d3fe after 13 minutes.


Unify the unix/Win32 packages

<IP_ADDRESS> - https://github.com/ghc/packages-unix
<IP_ADDRESS> - https://github.com/ghc/packages-Win32
Unify these two packages into an 'os' package as Java abstracts from the OS.

Temporarily closing this issue since we are just removing the unix dependencies from the packages that have it.


ctrl+z doesn't call to   handleKeyCommand

Do you want to request a feature or report a bug?
Bug
What is the current behavior?
When I press ctrl+y I got in to handleKeyCommand with parameter command = ''redo''
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem. You can use this jsfiddle to get started: https://jsfiddle.net/stopachka/m6z0xn4r/.
type something in the editor, press ctrl+z and try to stop in handleKeyCommand  (in debug)
What is the expected behavior?
ctrl+z will get in handleKeyCommand with command "undo"
Which versions of Draft.js, and which browser / OS are affected by this issue? Did this work in previous versions of Draft.js?
ersions of Draft.js is "0.10.1"

You need to override the key binding in keyBindingFn & then use that in handleKeyCommand:
if (
  e.keyCode === 90 /* `Z` key */ &&
  hasCommandModifier(e) &&
  !e.shiftKey
) {
  return 'my-editor-undo';
}


Legend loc in figures

I would like to have legends under figures as:
plt.legend(loc='upper center', bbox_to_anchor=(0.5, -0.1))

included in pull request #14

Slightly adjusted. Legend is in test file, so not directly in figures function.


Psydac on fedora:latest, import problem

I am trying to run Psydac on the latest Fedora docker image, which uses Python 3.11 by default.  After cloning the repo, the installation from devel works fine. However, when launching Python and trying to import, I get (note that the numpy import works, I tested also several other imports, such as struphy):
[...]# python3
Python 3.11.4 (main, Jun  7 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import psydac
dd1baa5f5873:pid267.python3: Failed to get eth0 (unit 0) cpu set
dd1baa5f5873:pid267: PSM3 can't open nic unit: 0 (err=23)
PMIx Log Report:[267]: (nic/PSM)[267]: PSM3 can't open nic unit: 0 (err=23)
dd1baa5f5873:pid267.python3: Failed to get eth0 (unit 0) cpu set
dd1baa5f5873:pid267: PSM3 can't open nic unit: 0 (err=23)
PMIx Log Report:[267]: (nic/PSM)[267]: PSM3 can't open nic unit: 0 (err=23)

I have tried also a virtual environment (even launched with python3.10 -m venv env), but get the same errors.

I have never seen an error like this before! 😅  For what it's worth, my intuition suggests that this is an MPI problem. Have you tried to import mpi4py?

Nice that the import of mpi4py works. Have you tried to use it together with mpirun? At a minimum you should write from mpi4py import MPI, and try some basic instructions (e.g. having each process write their MPI rank to the terminal, in order).

@spossann Do you have any updates on this issue?

I have created an issue on this in the fedora-image Github page. Let's see.

Adding these environment variables seemed to resolve this issue for me.
OMPI_MCA_pml=ob1
OMPI_MCA_btl=tcp,self

@spossann Have you tried the suggestion posted above?


The Armor & Health Value will sometimes display improperly.

As the picture shows, the AP Value didn't display properly (The number "0" is cut half?), and sometimes this also happens on HP value.

Should be fixed as of 31d4be9


Can i use kendo window with AngularsJs 1.5x Component?

Can i use kendo window with AngularsJs 1.5x Component?
Like uibModal does?  put resolve on bindings { resolve : "<" } of angularjs component.

Can you explain further? Like with an example
http://plnkr.co/edit/UD0mxA5ilaWMyGYGmZFr?p=preview


Fix ChatMessage.text() method that is deprecated

The dev.langchain4j.data.message.ChatMessage has been updated, and this PR should use the new API with Azure OpenAI

Sorry, my Git history was completely broken by IntelliJ.... Closing this and opening a new PR, with this fix (and the Base64 issue solved)


[cctag] update to v1.0.4

It updates cctag to the new 1.0.4 version.
No patch is required now as the issues with the new boost version 1.85 have been integrated.
Incidentally, it removes the line that removes the findTbb.cmake as it is not shipped anymore.

[x] Changes comply with the maintainer guide.
[x] SHA512s are updated for each updated download.
[x] The "supports" clause reflects platforms that may be fixed by this new version.
[x] Any fixed CI baseline entries are removed from that file.
[x] Any patches that are no longer applied are deleted from the port's directory.
[x] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[x] Only one version is added to each modified port's versions file.

@simogasp, Thanks for your PR, when I tested the feature by command ./vcpkg install cctag[*]:x64-windows I get this error.
Please take a look:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\include\crt/host_config.h(153): fatal error C1189: #error:  -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2022 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.

nvcc warning : The 'compute_35', 'compute_37', 'sm_35', and 'sm_37' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).

CMake Error at CCTag_generated_assist.cu.obj.Debug.cmake:220 (message):
  Error generating
  F:/Feature-test/cctag/buildtrees/cctag/x64-windows-dbg/src/CMakeFiles/CCTag.dir/./cctag/cuda/./CCTag_generated_assist.cu.obj

The VS I used for local testing is 2022, but it doesn't recognize .

I will try later when I have access to my Windows machine, but I suspect
that you have a too recent version of VS that it is not supported by that
version of cuda. Are you able to compile other packages that uses cuda?
(E.g. popsift)
On Mon, Jun 24, 2024, 10:12 JonLiu1993 @.***> wrote:

@simogasp https://github.com/simogasp, Thanks for your PR, when I
tested the feature by command ./vcpkg install cctag[*]:x64-windows I get
this error.
Please take a look:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\include\crt/host_config.h(153): fatal error C1189: #error:  -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2022 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
nvcc warning : The 'compute_35', 'compute_37', 'sm_35', and 'sm_37' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
CMake Error at CCTag_generated_assist.cu.obj.Debug.cmake:220 (message):
Error generating
F:/Feature-test/cctag/buildtrees/cctag/x64-windows-dbg/src/CMakeFiles/CCTag.dir/./cctag/cuda/./CCTag_generated_assist.cu.obj
The VS I used for local testing is 2022, but it doesn't recognize .
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vcpkg/pull/39467#issuecomment-2185881254,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKFEIHKKYV2LOFGXFXNTWLZI7IGHAVCNFSM6AAAAABJYRUAT6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBVHA4DCMRVGQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

@simogasp, Thanks for your PR, when I tested the feature by command ./vcpkg install cctag[*]:x64-windows I get this error. Please take a look:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\include\crt/host_config.h(153): fatal error C1189: #error:  -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2022 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.

nvcc warning : The 'compute_35', 'compute_37', 'sm_35', and 'sm_37' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).

CMake Error at CCTag_generated_assist.cu.obj.Debug.cmake:220 (message):
  Error generating
  F:/Feature-test/cctag/buildtrees/cctag/x64-windows-dbg/src/CMakeFiles/CCTag.dir/./cctag/cuda/./CCTag_generated_assist.cu.obj

The VS I used for local testing is 2022, but it doesn't recognize .

can you check the conditions around line 153 of C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\include\crt/host_config.h?
I should be something
#if defined(_WIN32)

#if _MSC_VER < 1910 || _MSC_VER >= 1930

#error -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2019 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.

this is mine for cuda 11.5.
Can you check the value _MSC_VER >= XXXX is in your version of cuda and which exact version of VS you are using?

@JonLiu1993 CI isn't detecting this probably because it uses CUDA 12.1 rather than 11.7 as in your example. You probably need to update your copy of CUDA to a 12.x release for VS2022.
https://github.com/microsoft/vcpkg/blob/89e222a90fc2847600be430a85783b8943f31f6c/scripts/azure-pipelines/windows/deploy-cuda.ps1 from https://github.com/microsoft/vcpkg/pull/39325 will install CUDA 12.4 for you which might help

Thanks @BillyONeal @simogasp, I updated my local computer cuda to 12.5 and successfully tested the feature.
All features are tested successfully in the following triplet:

x86-windows
x64-windows
x64-windows-static

Thanks for the update!


Closing a tab or the window does not result in a logout

I have Debian installed (uname -a returns Linux Arcadia 4.19.104-microsoft-standard #1 SMP Wed Feb 19 06:37:35 UTC 2020 x86_64 GNU/Linux), and I was in the process of getting all of my SSH stuff setup when I noticed that my .bash_logout (where I kill the instance of ssh-agent that was started on login) wasn't getting executed if I closed a tab or exited Windows Terminal.  After a couple of opens and closes, I've got five or six instances of ssh-agent -s running.
If I explicitly call logout, the window closes and I can open a new one to see that the previous instance was correctly killed as desired.
Environment
Windows build number: 10.0.19042.0
Windows Terminal version (if applicable): 1.1.2233.0

Steps to reproduce

Add the following to ~/.bashrc:
if [ -z "$SSH_AUTH_SOCK" ] ; then
  eval `ssh-agent -s`
fi

Add the following to ~/.bash_logout
if [ -n "$SSH_AUTH_SOCK" ] ; then
  eval `/usr/bin/ssh-agent -k`
fi

Close Windows Terminal.
Open Windows Terminal.
Close Windows Terminal.
Open Windows Terminal.
Execute ps aux | grep ssh-agent.
Expected behavior

There should be only one instance of ssh-agent running.
Actual behavior

There are multiple instances of ssh-agent still running.

Is this not what happens when you close an xterm or gnome-terminal? I am not certain that shells typically react to SIGHUP by processing their logout scripts.
Does this work as expected elsewhere?

My Debian box autostarts ssh-agent via x-session-manager, so I can't test this exact scenario.  However, one big difference is that if I try to logout on that box, I get bash: logout: not login shell: use 'exit', whereas on WSL2, if I enter logout, the tab closes (or the window if it's the only tab).  If Windows Terminal is opening a login shell, shouldn't .bash_logout be run when the window is closed?

So, alright. There's a combination of issues here.

wsl -d X starts your login shell as a login shell (automatically)
It is not standard behavior for a shell to run logout scripts when its controlling terminal exits

This behavior can be recapitulated in gnome-terminal if you toggle the "start ... as login shell" option: exiting the tab unceremoniously does not result in .bash_logout running.
I'm going to close this bug as by-design, but with a quick suggestion:
It may be possible to cache the output of ssh-agent such that you don't need to rely on a shell to exit gracefully and tear down the running instance. My personal agent startup looks a bit like...
function {
        function _agent_add_keys() {
                ssh-add &> /dev/null
                ssh-add ~/.ssh/id_rsa_howett.net &> /dev/null
        }

        if [[ ! -z "${SSH_AUTH_SOCK}" ]]; then
                if ! ssh-add -l &> /dev/null; then
                        _agent_add_keys
                fi
                unfunction _agent_add_keys
                return
        fi

        local _agent_env
        _agent_env="$HOME/.ssh/agent-env-$HOST"
        function _start_agent() {
                ssh-agent -s > "${_agent_env}"
                chmod 600 "${_agent_env}"
                . "${_agent_env}" &> /dev/null
                _agent_add_keys
        }

        ssh-add -l &> /dev/null
        if [[ $? -eq 2 ]]; then
                [[ -f "${_agent_env}" ]] && . "${_agent_env}" &> /dev/null
                ssh-add -l &> /dev/null
                if [[ $? -eq 2 ]]; then
                        _start_agent
                fi
        fi

        unfunction _start_agent
        unfunction _agent_add_keys
}

(admittedly, this is for zsh instead of bash.) It caches the agent info in ~/.ssh/agent-env-LIBRA or whatever my machine name suggests it should be, and each instance of my shell imports that info. It detects that the agent has exited because it'll fail to add a key. If it does so, it restarts it.
This seems a little cleaner than having one agent per shell instance, and presuming that I launch a lot of shells it's a reasonable savings of resources spent.


[AutoPR azure-iot-deviceupdate] add sample readme.typescript.md

Create to sync https://github.com/openapi-env-test/azure-rest-api-specs/pull/3122
You can install the use using pip install of the artificats.

API change check for azure-iot-deviceupdate
API changes have been detected in azure-iot-deviceupdate. You can review API changes here


How to convert the pre-model ir-50 to the onnx model of OpenCV DNN?

As shown in the title
How to use backbone_ir50_xxxx.pth in Opencv DNN

https://deci.ai/blog/how-to-convert-a-pytorch-model-to-onnx/


HTTP ERROR 500

Attempted Debugging

[X] I have read the debugging page

Searched GitHub Issues

[X] I have searched GitHub for the issue.

Describe the Scenario
Describe the bug
HTTP ERROR 500 when going to site.
Steps To Reproduce
Going to site, I get HTTP ERROR 500
Expected behavior
I would intend for the site to load.
Exact BookStack Version
BookStack v21.12.3
Log Content
PHP Fatal error:  Uncaught Error: Call to undefined method ComposerAutoloaderInit4e5fbf84e4a2bdf796f705349341e084::getLoader() in bookstack/vendor/autoload.php
PHP Version
PHP 7.4.26 (cli) (built: Nov 19 2021 06:24:55) ( NTS )
Hosting Environment
Debian 12, Apache

Hi @KapustaIKwas,

Can you confirm the version of composer in use? (Output of running composer -V)
Do you get any errors during the composer install --no-dev step of the update commands?

Hey @ssddanbrown
Composer version is:  2.2.4 2022-01-08 12:30:42
When running the install command im getting two PHP Warnings:

PHP Warning:  PHP Startup: Unable to load dynamic library 'apc.so'
PHP Warning:  PHP Startup: Unable to load dynamic library 'memcached.so'

@KapustaIKwas Thanks, I'm assuming there were not other errors running the command and that if you re-run it you see the following line within the output?:
Nothing to install, update or remove

You could trying deleting the vendor/ directory within your BookStack install folder, then re-run the composer install --no-dev step to see if the issue remains afterwards.

@ssddanbrown yeah the issue still remains after deleting and reinstalling the composer dependencies.
Could it have something to do with the fact that I'm using debian 12?
Just installed bookstack on a ubuntu 18 system and it works fine.

Could it have something to do with the fact that I'm using debian 12

Potentially, always an added risk of using pre-release operating systems. Could be something funky in a caching element of the PHP setup.

Just installed bookstack on a ubuntu 18 system and it works fine.

Okay, is that instead of the debian instance? Alright for me to therefore close this off?

Yeah sure, thanks for the quick help!


help center: Update "SCIM provisioning" page.

Updates the SCIM provisioning page to match help center documentation patterns, and to use the newly added macros.
Fixes: #22890.
Concerns:
The wording should be double-checked for accuracy as I'm not familiar with SCIM provisioning.
Screenshots and screen captures:

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability

Communicate decisions, questions, and potential concerns.

[x] Calls out remaining decisions and concerns.
[x] Automated tests.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[x] Visual appearance of the changes.
[x] Links.

@alya This PR is ready for your review.

@mateuszmandera  Could you please review this one before I take a pass?

Looks good to me other than the comments above!

I've updated this PR in response to your feedback and updated the screenshots in the PR description.

Hm, the screenshots and the PR don't seem to be fully updated? I still see several open comments.

All right, submitted now. Sigh.

I think this PR is pretty much ready for integration.  We just need to confirm whether "assign" is a label in the Okta UI to know whether we should use bold formatting in the last step of the "Configure SCIM" section:

Now that the  integration is ready to manage Zulip user accounts, assign users to the SCIM app.

@mateuszmandera can you double-check the above?

@drrosa Yes, "assign" is the specific term Okta uses in their UI for this:

@timabbott I think this one is ready for you! Reviewed by @mateuszmandera and me.

Merged, after rewriting some of the introductory paragraphs to make it sound less like "We have an Okta SCIM integration" and instead clearly convey "we have an SCIM integration; SCIM is kinda a pain to setup on the provider side, and we only documented that in detail for Okta".


Warning on converting 32bit to 30bit (bitfield)

@kleunen I need your help.
The CI is broken now since several time ago. The warning check isn't working.
After many try and errors, I re-enabled warnings. But I got many warnings (as errors).
I'm fixing it step by step.
And I stuck subscription_map.hpp
See:
https://github.com/redboltz/mqtt_cpp/runs/1466271881?check_suite_focus=true#step:5:66
/home/runner/work/mqtt_cpp/mqtt_cpp/test/subscription_map.hpp:97:68: error: conversion to ‘unsigned int:30’ from ‘uint32_t {aka unsigned int}’ may alter its value [-Werror=conversion]
         : value(value), has_hash_child(false), has_plus_child(false)

template<>
struct count_storage<4>
{
    std::uint32_t value : 30;
    std::uint32_t has_hash_child : 1;
    std::uint32_t has_plus_child : 1;

    count_storage(std::uint32_t value = 1)
        : value(value), has_hash_child(false), has_plus_child(false)
    { }

    static constexpr std::size_t max() { return std::numeric_limits<uint32_t>::max() >> 2; }
};

This is wandbox demo https://wandbox.org/permlink/sy7upObBvNtMLG4q
g++ reports warning.
Could you investigate how to solve it?

It seems that I found a solution.
https://wandbox.org/permlink/VqJpTQ2o1IbS0sbO


fix clipskip

reference PR https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/10804

Thank you!
it'd be nice if there were p.clip_skip and etc. fields available so we didn't have to awkwardly mess with shared.opts or adapt to unrelated upstream changes

before get_conds_with_caching() you could just use apply_override()
I did consider switching to apply_override for clip but I want to kep the changes as minimum as possble
you can switch to apply_override, is just in order to keeptrack take advantage of get_conds_with_caching(), an extra var need use for memory will have to be used for keeping track of the current clip skip vlaue
get_conds_with_caching() can also be modified
but I didn't want to make changes to script unless nessery
btw lots of other fields can also use apply_override(), I think AUTO hase switch some over to apply_override since then
since "don't fix what isn't borke" so I didn't make the switch


Gradle should support file watching on OverlayFS

Expected Behavior
Continuous builds should work from a docker container
Current Behavior (optional)
Running any build with -t inside a docker container using an overlay FS gives:
Configuration on demand is an incubating feature.
Watching the file system is not supported.

FAILURE: Build failed with an exception.

* What went wrong:
Continuous build does not work when file system watching is disabled

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s

What's interesting is that the home directory is using XFS -- GRADLE_HOME and the repository in question are both on the XFS mount, which seems to be supported, but I believe OverlayFS is not.
Context
Using gradle 7.6.2.
We have development environments in the cloud, running on kubernetes, and it seems that file watching / continuous mode is not supported there.

I tried to use --debug to see if there's more information about what directory / FS isn't supported, but there's not any more information as far as I can tell.

Thank you for your interest in Gradle!
This feature request is in the backlog of the relevant team, but this area of Gradle is currently not in focus. It might take a while before it gets implemented.

What OS are you running on? Could you also maybe try with 8.6-rc-1 with --info log enabled? I've added some more verbose logging about why FSW is disabled.

@lptr macOS Sonoma 14.2.1
Also, as far as I can tell there is no official image for Gradle 8.6* https://hub.docker.com/_/gradle/tags. Presumably because it's a release candidate. Are you folks able to publish that image to Docker Hub, or point me to the right direction so I can build it myself?

With Gradle 8.6-rc-4
Initialized native services in: /home/gradle/.gradle/native
Initialized jansi services in: /home/gradle/.gradle/native
Received JVM installation metadata from '/opt/java/openjdk': {JAVA_HOME=/opt/java/openjdk, JAVA_VERSION=21.0.2, JAVA_VENDOR=Eclipse Adoptium, RUNTIME_NAME=OpenJDK Runtime Environment, RUNTIME_VERSION=21.0.2+13-LTS, VM_NAME=OpenJDK 64-Bit Server VM, VM_VERSION=21.0.2+13-LTS, VM_VENDOR=Eclipse Adoptium, OS_ARCH=aarch64}
Removing 0 daemon stop events from registry
Starting a Gradle Daemon (subsequent builds will be faster)
Starting process 'Gradle build daemon'. Working directory: /home/gradle/.gradle/daemon/8.6-rc-4 Command: /opt/java/openjdk/bin/java --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED -XX:MaxMetaspaceSize=384m -XX:+HeapDumpOnOutOfMemoryError -Xms256m -Xmx512m -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /opt/gradle/lib/gradle-launcher-8.6.jar -javaagent:/opt/gradle/lib/agents/gradle-instrumentation-agent-8.6.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 8.6-rc-4
Successfully started process 'Gradle build daemon'
An attempt to start the daemon took 0.494 secs.
The client will now receive all logging from the daemon (pid: 45). The daemon log file: /home/gradle/.gradle/daemon/8.6-rc-4/daemon-45.out.log
Starting build in new daemon [memory: 512 MiB]
Using 10 worker leases.
Received JVM installation metadata from '/opt/java/openjdk': {JAVA_HOME=/opt/java/openjdk, JAVA_VERSION=21.0.2, JAVA_VENDOR=Eclipse Adoptium, RUNTIME_NAME=OpenJDK Runtime Environment, RUNTIME_VERSION=21.0.2+13-LTS, VM_NAME=OpenJDK 64-Bit Server VM, VM_VERSION=21.0.2+13-LTS, VM_VENDOR=Eclipse Adoptium, OS_ARCH=aarch64}
Watching the file system is configured to be enabled
VFS> Statistics since last build:
VFS> > Stat: Executed stat() x 0. getUnixMode() x 0
VFS> > FileHasher: Hashed 0 files (0 bytes)
VFS> > DirectorySnapshotter: Snapshot 0 directory hierarchies (visited 0 directories, 0 files and 0 failed files)
Now considering [/Users/julichm/workspace/my-project] as hierarchies to watch
File system watching is active
Starting Build
Compiling settings file '/Users/julichm/workspace/my-project/settings.gradle' using SubsetScriptTransformer.
Compiling settings file '/Users/julichm/workspace/my-project/settings.gradle' using BuildScriptTransformer.
Settings evaluated using settings file '/Users/julichm/workspace/my-project/settings.gradle'.
Projects loaded. Root project using build file '/Users/julichm/workspace/my-project/build.gradle'.
Included projects: [root project 'my-project']

My setup looks something like this https://github.com/jmera/gradle-fsw


Make Diagnostics->Symbols more prominent in the iOS Symbolication guide

Describe the solution you'd like
The iOS Symbolication guide mentions that you can upload symbols manually by navigating to the Diagnostics->Symbols page, but it would be nice if the other sections mention that you can use that page to verify that you've successfully uploaded symbols.
Describe alternatives you've considered
You could also make the Diagnostics->Symbols page more prominent in the page's navigation. It's currently only visible once you've already clicked on Diagnostics, if you then mouse over Diagnostics again, which makes it less than easily discoverable.
Additional context
I missed the existence of that page when I was first writing a script to upload symbols using the REST API, since I skipped right to that section. It would have been very helpful if that part of the guide advised that I could see the symbols list there. I imagine this may also be a helpful suggestion in the App Center CLI section.

Has this been implemented?

Thank you for the feature request, @abbeycode! Currently, we are focusing on improving the reliability and performance of the service and new feature work is significantly reduced. To keep the team focused we are closing feature requests with no activity in the past 30-60 days. If you believe this is still relevant, feel free to create a new issue and tag this item so that we can review it. Sorry for the inconvenience.


Handle SIGTERM signal

To debug error 143 , the python worker process should handle SIGTERM signal and log the current memory usage before exiting.
We need this until we have some other way of getting memory status from the container.

Hey Bala, I don't have too much context, could you attach a link to error 143 here?

This is on the roadmap for the next few months.

There are places exit with 143, worker dies before the host (maybe memory issue).
Logs through STDERR, needs to implement SIGTERM handler.

@balag0 is this still relevant?

Yes.


LA 2253, LA-2255 code customizing to our need

We are now changing the code to our library needs.
"JSONViewer" code is now removed from "JSONEditor" component. it will not have the functionality of viewer which is not required in our library. removed the import from code.

Please add proper titles to the PR's.

We should not remove the functionality! We can make it configurable/controllable.

@Nikmahajan Ok Nikhil I will work on it to make it more configurable and controlable

@Nikmahajan ,adding back viewer functionality code on the JSONE editor that will be props-driven.

Added the code for JSON viewer, If a user needs the dual view both for editor and viewer he could have by passing prop "view"={'dual'} .


HomeKit Icon

At first! Thank you for you awesome work! but I have a question. When I add the button everything works just fine. The only thing is that the button icon on HomeKit stays greyed out when I toggle the button?
ps. do you have a PayPal account, I want to buy you a :coffee: for you excellent work!

It seems that you need a switch service instead.
Thank you for your support. My paypal is https://www.paypal.me/MaximKulkin

No, I know hen I Change the service to switch, the HomeKit icon allso changes. And you van see a switch Flip in the icon. But heen you toggle the button doesn’t it had to change to purpel?
https://ksassets.timeincuk.net/wp/uploads/sites/54/2017/02/Apple-Home-4.png

Sorry doe the typo’s; typing on my iPhone and it has the dutch dictionary on. Hope you like you ::koffee::

Thank you for tipping.
Actually, I'm not sure about mechanics of a button, the idea of a button is that it is usually not interact with it through Home.app. You just press physical button and it generates event that you can base your automation on.

@maximkulkin #offtopic
Apple buys self-driving car startup Drive.ai
Congrats?

Unfortunately, I'm not part of the deal. But I have already started in another self-driving company

Congrats With your new Job @Plus.AI :exclamation: :thumbsup:


Audit should ignore groups that the app is only a member of

If the app service account is a member (not admin) of a group on AnVIL that is not in the app, the record should be ignored.

Closed by #411


Add SIP for signing Spin releases

Rendered SIP

cc @vdice since you are the most familiar with the release process.


Updates wording and adds docs links to 2.5 changelog.

Summary
I updated the 2.5 changelog with my wording edits and added documentation links as well.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Adds full screen landscape support

This PR is based of the 4.4.0 tag instead of the latest 5.0.0 release since we can't update our app right now to offer Android X support.
This branch should not be merged into master, rather, a new release tag should be created from it after it is reviewed.
Add an icon in the playerControlView that toggles switching to fullscreen
Instead of creating a new Activity to handle the fullscreen video, use a modal dialog with a Theme_Black_NoTitleBar_Fullscreen
When toggling through fullscreen, the existing exoPlayerView (and controls) are added/removed to/from the main activity's view/dialog

Hi @carlosalmonte04 @bpolonia,
Thanks for reviewing&approving this PR!
Unfortunately, during some additional tests, I've discovered a scenario that crashes the app, when the remote video URL causes the views to be re-created, the app will crash due to an index out of bounds.
I'll hold a bit in merging it until we can find a proper solution.
I'll ping you once the PR gets updated.
Thanks!

Hi @carlosalmonte04 @bpolonia,
I've added a
        if (indexOfExoPlayerView == -1 || getChildCount() == 0 ) {             return;         }
check for the case mentioned above.
Let me know what you think!


Error with Android 8.0 and Android Platform 7.1

Actual Behaviour
The app doesn't working after upgrade Android and Push Notifications for compatibility with Android 8
Reproduce Scenario
The error happens after updating the Android platform to 7.1.0 and the push plugin to 2.2.3
Steps to Reproduce
Start the app and the plugin executes the error event directly

Platform and Version
Android 8.0

(Android) What device vendor
Google Pixel

Cordova CLI version and cordova platform version
cordova --version                         7.1.0
cordova platform version android          7.1.1

Plugin version
cordova plugin version | grep phonegap-plugin-push   2.2.3

Sample Code that illustrates the problem
const pushOptions = {
  android: {
      senderID: "xxxxxxxxxx",
      forceShow: true
    },
    ios: {
      "alert": "true",
      "badge": "true",
      "sound": "true",
      "clearBadge": "false"
  }
}
 const pushObject = this.push.init(pushOptions);

pushObject.on('error').subscribe(error => {
      console.error('Error with Push plugin', JSON.stringify(error));
      let toastOptions: ToastOptions = {
        message: 'Your device is not compatible with Push Notifications',
        duration: 4000,
        position: 'top',
        cssClass: 'toast-error'
      };
      let toast = this.toastCtrl.create(toastOptions);
      toast.present();
    });

The issue seems related to Google Play Services


Secondary user

What's done

Adding a new request definition for creating an external user

How to test
cd examples/example-node-api-client && yarn start
Tasks

[ ] Have you written tests for new code (if applicable)?
[x] Have you tested the changes on all the platforms?

[ ] iOS
[ ] Android
[ ] Web
[x] Node

[ ] Have you added stories to demonstrate the new functionality (if there is any)?

Demo (screenshots, recordings, etc.)

@marlon-wiprud you have to run yarn changeset in the root to generate a new version of the api package


Skip invalid items of collections instead of raising ValidationError

Checks

[x] I added a descriptive title to this issue
[x] I have searched (google, github) for similar issues and couldn't find anything
[x] I have read and followed the docs and still think this feature/change is needed
[x] After submitting this, I commit to one of:

Look through open issues and helped at least one other person
Hit the "watch" button on this repo to receive notifications and I commit to help at least 2 people that ask questions in the future
Implement a Pull Request for a confirmed bug

Feature Request
When parsing data, you may encounter some invalid items in a list. In some cases, it's not worth to through away all of the data because of just one invalid item.
What I need is to skip invalid items if a list and leave only valid ones.
Sure, this can be done using @validator(pre=True), but this needs to be default behaviour of my models and to accomplish this I will basically have to write custom validator for every single list by hand, which is not very handy.
Related issues
#800
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
             pydantic version: 1.6.1
            pydantic compiled: True
                 install path: /Users/rocky/.local/share/virtualenvs/.../lib/python3.6/site-packages/pydantic
               python version: 3.6.8 (v3.6.8:3c6b436a57, Dec 24 2018, 02:04:31)  [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
                     platform: Darwin-20.1.0-x86_64-i386-64bit
     optional deps. installed: ['typing-extensions']

Here's how I expect it to work:
from pydantic import BaseModel
from pydantic.types import OptionalList

class Model(BaseModel):
    a: OptionalList[int]

m = Model(a=[1, '2', '?'])
assert m.a == [1, 2]
assert m.__skipped__ == [{'loc': ('a', 2), 'msg': 'value is not a valid integer', 'type': 'type_error.integer'}]
# skipped list should be returned by `validate_model()` just like errors
...

Things to think of:

Support ofconlist, conset etc... and interaction with them
Default values instead of invalid skipped ones

Another possible syntax:
a: List[int] = Field(skip_invalid=True)

class Config:
    skip_invalid_collections_items = True

Hey fellow person who needs to only optionally validate some fields!
This has been a huge pain point for me in the past and I now have a working solution (for pydantic 1.4) that is extremely hacky and quite slow, as it delegates type creation to the runtime for optional types.
Some concerns about your proposal:
Storing your validation notices on the model instance will break in cases where required fields are also invalid
Quite straight forward: if you have an invalid required field, it will still raise a validation error and you'll never get the model instance, thus m.__skipped__ doesn't exist.
My solution here was to raise these optional fields as warnings. That works every time regardless of the total outcome of the validation.
Some downsides are...

Warnings in Python are dumb and you need to once again create the warning types at runtime to give them a payload (The validation message).
Collecting warnings in Python are not thread-safe but that#s more of an academic point in Python-land.

loc is a lie, sort of
This one was driving me nuts for ages! The loc is only created when the total list of nested validation errors is flattened the final flat list. I had to reach deep into the guts of the BaseModel class to hack the propagation of the actual loc into it, because of how pydantic operates this also has some very real implications as it introduces essentially state on certain types, where there should rather be one, namely the ModelField classes. Those essentially now get polluted by the loc and you need to clean up after yourself and reset it. This also forbids recursive type uses in most cases, which doesn't affect my use case but isn't good enough for the general goals of this library.
IMHO, what needs to come first before this can be a reality is moving the loc generation into the whole validation flow for all validation first, only then can you really start extracting validation information anywhere in the validation flow, which is a necessary but insufficient prerequisite for this feature to work.

Hey fellow person who needs to only optionally validate some fields!
This has been a huge pain point for me in the past and I now have a working solution (for pydantic 1.4) that is extremely hacky and quite slow, as it delegates type creation to the runtime for optional types.
Some concerns about your proposal:
Storing your validation notices on the model instance will break in cases where required fields are also invalid
Quite straight forward: if you have an invalid required field, it will still raise a validation error and you'll never get the model instance, thus m.__skipped__ doesn't exist.
My solution here was to raise these optional fields as warnings. That works every time regardless of the total outcome of the validation.
Some downsides are...

Warnings in Python are dumb and you need to once again create the warning types at runtime to give them a payload (The validation message).
Collecting warnings in Python are not thread-safe but that#s more of an academic point in Python-land.

loc is a lie, sort of
This one was driving me nuts for ages! The loc is only created when the total list of nested validation errors is flattened the final flat list. I had to reach deep into the guts of the BaseModel class to hack the propagation of the actual loc into it, because of how pydantic operates this also has some very real implications as it introduces essentially state on certain types, where there should rather be one, namely the ModelField classes. Those essentially now get polluted by the loc and you need to clean up after yourself and reset it. This also forbids recursive type uses in most cases, which doesn't affect my use case but isn't good enough for the general goals of this library.
IMHO, what needs to come first before this can be a reality is moving the loc generation into the whole validation flow for all validation first, only then can you really start extracting validation information anywhere in the validation flow, which is a necessary but insufficient prerequisite for this feature to work.

Quite straight forward: if you have an invalid required field, it will still raise a validation error and you'll never get the model instance, thus m.__skipped__ doesn't exist.

I mentioned that skipped values should be returned by validate_model(), just like errors. At this point, we can pass them into raised ValidationError and display as regular errors with some special flag, like so:
[
    {'loc': ('a', 2), 'msg': 'value is not a valid integer', 'type': 'type_error.integer', 'skipped': True},
    {'loc': ('b',), 'msg': 'field required', 'type': 'value_error.missing'},
]

My solution here was to raise these optional fields as warnings.

As for warnings, I think, they are too specific to fit in every workflow. And personally, I'd preferred some other way to gather skipped fields.

IMHO, what needs to come first before this can be a reality is moving the loc generation into the whole validation flow for all validation first, only then can you really start extracting validation information anywhere in the validation flow, which is a necessary but insufficient prerequisite for this feature to work.

Agreed.

EDIT: Oh yeah, what's extra interesting for you, to make lists work in a reasonable manner I had to iterate over them and validate every item one by one and only add the valid ones back to the resulting list.

That's exactly what I don't want to do every time and that's why I decided to open this issue in the first place :)

Quite straight forward: if you have an invalid required field, it will still raise a validation error and you'll never get the model instance, thus m.__skipped__ doesn't exist.

I mentioned that skipped values should be returned by validate_model(), just like errors. At this point, we can pass them into raised ValidationError and display as regular errors with some special flag, like so:
[
    {'loc': ('a', 2), 'msg': 'value is not a valid integer', 'type': 'type_error.integer', 'skipped': True},
    {'loc': ('b',), 'msg': 'field required', 'type': 'value_error.missing'},
]

My solution here was to raise these optional fields as warnings.

As for warnings, I think, they are too specific to fit in every workflow. And personally, I'd preferred some other way to gather skipped fields.

IMHO, what needs to come first before this can be a reality is moving the loc generation into the whole validation flow for all validation first, only then can you really start extracting validation information anywhere in the validation flow, which is a necessary but insufficient prerequisite for this feature to work.

Agreed.

EDIT: Oh yeah, what's extra interesting for you, to make lists work in a reasonable manner I had to iterate over them and validate every item one by one and only add the valid ones back to the resulting list.

That's exactly what I don't want to do every time and that's why I decided to open this issue in the first place :)

Here's my temporary implementation. It works as expected with GenericModel and generate schema identical to typing.List.
from typing import TYPE_CHECKING, Any, List, Optional, TypeVar, cast

from pydantic import BaseModel
from pydantic.fields import ModelField, Undefined
from pydantic.validators import list_validator

if TYPE_CHECKING:
    from pydantic.typing import DictStrAny

ItemT = TypeVar('ItemT')

class LenientList(List[ItemT]):
    """
    LenientList[T] is same as List[T], but will skip invalid items instead of raising error

    At some point this behaviour might be implemented in pydantic:
    https://github.com/samuelcolvin/pydantic/issues/2274
    """

    _item_field: ModelField

    def __class_getitem__(cls, type_):
        """
        Returned type must be subclass of LenientList for validation to work,
        But it also needs to smell like typing.List[T] for pydantic magic to work properly
        """
        if isinstance(type_, tuple):
            (type_,) = type_
        elif hasattr(cls, '_item_field'):
            raise TypeError(f'{cls.__name__} is already concrete')

        if isinstance(type_, TypeVar):
            pydantic_type = type(f'LenientList[{type_}]', (cls,), {})
        else:
            item_field = ModelField.infer(
                name='item',
                value=Undefined,
                annotation=type_,
                class_validators=None,
                config=BaseModel.__config__,
            )
            t_name = getattr(type_, "__name__", None) or type_.__class__.__name__
            pydantic_type = type(f'LenientList[{t_name}]', (cls,), {'_item_field': item_field})

        generic_alias = super().__class_getitem__(type_)
        # this will enable support of GenericModel and json schema
        pydantic_type.__origin__ = generic_alias.__origin__
        pydantic_type.__parameters__ = generic_alias.__parameters__
        pydantic_type.__args__ = generic_alias.__args__
        return pydantic_type

    @classmethod
    def __get_validators__(cls):
        yield cls._list_validator

    @classmethod
    def _list_validator(cls, raw_value: Any, values: 'DictStrAny', field: ModelField) -> Optional[List[ItemT]]:
        if raw_value is None and not field.required:
            return None
        list_value: List[Any] = list_validator(raw_value)
        parsed: List[ItemT] = []
        for item in list_value:
            value, error = cls._item_field.validate(item, values, loc=())
            if error is None:
                parsed.append(cast(ItemT, value))
        return parsed

class Root(BaseModel):
    items: LenientList[int]

print(Root(items=[1, 'noo', 3]))  # items=[1, 3]

Here's my temporary implementation. It works as expected with GenericModel and generates schema identical to typing.List.
from typing import TYPE_CHECKING, Any, List, Optional, TypeVar, cast

from pydantic import BaseModel
from pydantic.fields import ModelField, Undefined
from pydantic.validators import list_validator

if TYPE_CHECKING:
    from pydantic.typing import DictStrAny

ItemT = TypeVar('ItemT')

class LenientList(List[ItemT]):
    """
    LenientList[T] is same as List[T], but will skip invalid items instead of raising error

    At some point this behaviour might be implemented in pydantic:
    https://github.com/samuelcolvin/pydantic/issues/2274
    """

    _item_field: ModelField

    def __class_getitem__(cls, type_):
        """
        Returned type must be subclass of LenientList for validation to work,
        But it also needs to smell like typing.List[T] for pydantic magic to work properly
        """
        if isinstance(type_, tuple):
            (type_,) = type_
        elif hasattr(cls, '_item_field'):
            raise TypeError(f'{cls.__name__} is already concrete')

        if isinstance(type_, TypeVar):
            pydantic_type = type(f'LenientList[{type_}]', (cls,), {})
        else:
            item_field = ModelField.infer(
                name='item',
                value=Undefined,
                annotation=type_,
                class_validators=None,
                config=BaseModel.__config__,
            )
            t_name = getattr(type_, "__name__", None) or type_.__class__.__name__
            pydantic_type = type(f'LenientList[{t_name}]', (cls,), {'_item_field': item_field})

        generic_alias = super().__class_getitem__(type_)
        # this will enable support of GenericModel and json schema
        pydantic_type.__origin__ = generic_alias.__origin__
        pydantic_type.__parameters__ = generic_alias.__parameters__
        pydantic_type.__args__ = generic_alias.__args__
        return pydantic_type

    @classmethod
    def __get_validators__(cls):
        yield cls._list_validator

    @classmethod
    def _list_validator(cls, raw_value: Any, values: 'DictStrAny', field: ModelField) -> Optional[List[ItemT]]:
        if raw_value is None and not field.required:
            return None
        list_value: List[Any] = list_validator(raw_value)
        parsed: List[ItemT] = []
        for item in list_value:
            value, error = cls._item_field.validate(item, values, loc=())
            if error is None:
                parsed.append(cast(ItemT, value))
        return parsed

class Root(BaseModel):
    items: LenientList[int]

print(Root(items=[1, 'noo', 3]))  # items=[1, 3]

How would it work for a single item? to implement a pagination

A very useful suggestion. I'm working with an old and poorly documented API, and I can't predict in advance all possible responses and exception handling, ignoring and logging artifacts in this case is a great strategy. The code remains clean and understandable

Here's my temporary implementation. It works as expected with GenericModel and generates schema identical to typing.List.
from typing import TYPE_CHECKING, Any, List, Optional, TypeVar, cast

from pydantic import BaseModel
from pydantic.fields import ModelField, Undefined
from pydantic.validators import list_validator

if TYPE_CHECKING:
    from pydantic.typing import DictStrAny

ItemT = TypeVar('ItemT')

class LenientList(List[ItemT]):
    """
    LenientList[T] is same as List[T], but will skip invalid items instead of raising error

    At some point this behaviour might be implemented in pydantic:
    https://github.com/samuelcolvin/pydantic/issues/2274
    """

    _item_field: ModelField

    def __class_getitem__(cls, type_):
        """
        Returned type must be subclass of LenientList for validation to work,
        But it also needs to smell like typing.List[T] for pydantic magic to work properly
        """
        if isinstance(type_, tuple):
            (type_,) = type_
        elif hasattr(cls, '_item_field'):
            raise TypeError(f'{cls.__name__} is already concrete')

        if isinstance(type_, TypeVar):
            pydantic_type = type(f'LenientList[{type_}]', (cls,), {})
        else:
            item_field = ModelField.infer(
                name='item',
                value=Undefined,
                annotation=type_,
                class_validators=None,
                config=BaseModel.__config__,
            )
            t_name = getattr(type_, "__name__", None) or type_.__class__.__name__
            pydantic_type = type(f'LenientList[{t_name}]', (cls,), {'_item_field': item_field})

        generic_alias = super().__class_getitem__(type_)
        # this will enable support of GenericModel and json schema
        pydantic_type.__origin__ = generic_alias.__origin__
        pydantic_type.__parameters__ = generic_alias.__parameters__
        pydantic_type.__args__ = generic_alias.__args__
        return pydantic_type

    @classmethod
    def __get_validators__(cls):
        yield cls._list_validator

    @classmethod
    def _list_validator(cls, raw_value: Any, values: 'DictStrAny', field: ModelField) -> Optional[List[ItemT]]:
        if raw_value is None and not field.required:
            return None
        list_value: List[Any] = list_validator(raw_value)
        parsed: List[ItemT] = []
        for item in list_value:
            value, error = cls._item_field.validate(item, values, loc=())
            if error is None:
                parsed.append(cast(ItemT, value))
        return parsed

class Root(BaseModel):
    items: LenientList[int]

print(Root(items=[1, 'noo', 3]))  # items=[1, 3]

work fine, thank you

In case anyone is waiting for feedback on this, LenientList as implemented here by Bobronium works for me in a complex schema, with pydantic 1.10.2. I don't know the internals of pydantic enough to give a proper code review though.

A property to collect the errors on LenientList would be very handy, so they can be logged for later inspection.

A property to collect the errors on LenientList would be very handy, so they can be logged for later inspection.

collecting errors should become possible in V2 with context, see here.

This should be possible in V2 with on_error='omit' but we should make sure it's actually possible before release.

@Bobronium it should be possible now using __get_pydantic_validation_schema__, any chance you could try it out? PR welcome to add this type for V2.

@samuelcolvin, thanks for a heads up!
Yeah, I totally can try to implement it.
Do you have any concrete time frame for this PR in mind?

No hurry, some time before the end of March would be great.

Hi @Bobronium, is there any chance you've had an opportunity to work on this? Can I help in any way?

@Mason3k, I haven't even started yet, but planned later this month, as was discussed above.
I would be glad to accept any help on this! And if you'd like to work on this yourself, I'd be glad to provide my help. We can code pair, for example and I could assist you if you'd have any difficulties.

I have some good news, after thinking about this for a bit I think this not only works in V2 but is also super powerful and re-usable:
from dataclasses import dataclass
from typing import Annotated, Any, Callable, List, TypeVar

from pydantic_core import ValidationError
from pydantic_core import core_schema as cs

_ERROR = object()

@dataclass
class ErorrItemsMarker:
    def __get_pydantic_core_schema__(
        self, source_type: Any, handler: Callable[[Any], cs.CoreSchema]
    ) -> cs.CoreSchema:
        schema = handler(source_type)
        def val(v: Any, handler: cs.ValidatorFunctionWrapHandler) -> Any:
            try:
                return handler(v)
            except ValidationError:
                return _ERROR

        return cs.no_info_wrap_validator_function(val, schema, serialization=schema.get('serialization'))

@dataclass
class ListErrorFilter:
    def __get_pydantic_core_schema__(
        self, source_type: Any, handler: Callable[[Any], cs.CoreSchema]
    ) -> cs.CoreSchema:
        schema = handler(source_type)

        def val(v: List[Any]) -> List[Any]:
            return [item for item in v if item is not _ERROR]

        return cs.no_info_after_validator_function(val, schema, serialization=schema.get('serialization'))

T = TypeVar('T')

LenientList = Annotated[List[Annotated[T, ErorrItemsMarker()]], ListErrorFilter()]

from pydantic import BaseModel

class Model(BaseModel):
    x: LenientList[int]
    y: LenientList[str]
    z: LenientList[List[int]]

print(Model(x=[1, '2', 'c'], y=['a', 'b', 3], z=[['a'], ['1']]))
#> x=[1, 2] y=['a', 'b'] z=[[1]]

Obviously you may want slightly different behavior, but I think this general template can be adapted.

It's also supported out of the box in pydantic-core, see a test example here and the schema to configure it here.
I implemented this months ago thinking of this issue, but forgot to reference it and we haven't added support in python yet.

@samuelcolvin that doesn't expose a way to track the errors does it? I implemented a slight tweak to @adriangb's very good suggestion here: https://gist.github.com/dmontagu/7f0cef76e5e0e04198dd608ad7219573, this adds an attribute to LenientList called errors that lets you retrieve the validation errors, and has a method with_errors() that returns the items and errors in the order corresponding to validation.
The implementation has some warts, but perhaps someone will find it useful.
Either way, this is another good example of how to create a pydantic core schema for a custom generic type.

Is there any example anywhere of using this on_error='omit' in v2?

Bump

OnErrorOmit can be used to use on_error='omit'.

Is there any example anywhere of using this on_error='omit' in v2?

There's an example here 🙂 https://github.com/pydantic/pydantic/blob/cd3b0ad3943829a00d22da9a9156256b97fad9ae/tests/test_types.py#L6656
or for a list List[OnErrorOmit[<type>]]

Is there any example anywhere of using this on_error='omit' in v2?

There's an example here 🙂
https://github.com/pydantic/pydantic/blob/cd3b0ad3943829a00d22da9a9156256b97fad9ae/tests/test_types.py#L6656
or for a list List[OnErrorOmit[<type>]]

This is what I am looking for.. thanks @adriangb @leonardzou.. Anyway, how we can get list errors for further development? Lets say I want it to put on logging.error message


Weird behavior of custom inference.py script

Hi,
I'm deploying the custom inference.py script below
import os

import tensorflow as tf
from transformers import TFAutoModelForQuestionAnswering, AutoTokenizer

def load_fn(model_dir):
    """this function reads the model from disk"""

    print('load_fn dir view:')
    print(os.listdir())

    # load model
    model = TFAutoModelForQuestionAnswering.from_pretrained(os.environ['MODEL_FOLDER'])

    # load tokenizer
    tokenizer = AutoTokenizer.from_pretrained(os.environ['TOKENIZER_FOLDER'])

    return model, tokenizer

def preprocess_fn(input_data, content_type):
    """this function pre-processes the input.
       payload format is {"inputs": {"question": XX, "text": XX}}"""

    print('input_data received:')
    print(input_data)
    return input_data['inputs']['question'], input_data['inputs']['text']

def predict(processed_data)
    """this function runs inference"""

    print('processed_data received: ')
    print(processed_data)

    question, text = processed_data

    input_dict = tokenizer(question, text, return_tensors='tf')
    outputs = model(input_dict)
    start_logits = outputs.start_logits
    end_logits = outputs.end_logits

    all_tokens = tokenizer.convert_ids_to_tokens(input_dict["input_ids"].numpy()[0])
    answer = ' '.join(all_tokens[tf.math.argmax(start_logits, 1)[0] : tf.math.argmax(end_logits, 1)[0]+1])

    return answer

There are 2 weirds things happening:

the print statements are not showing up in CloudWatch
the inference errors with W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - mms.service.PredictionException: invalid syntax (inference.py, line 39) : 400

Anything wrong in the code?

the print statements are not showing up in CloudWatch

@vdantu can you share your infernece.py where you can see the print statements?

the inference errors with W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - mms.service.PredictionException: invalid syntax (inference.py, line 39) : 400

@la-cruche what is line 39?
Also, I am not sure if your preprocess function is correct since you are not parsing the JSON.

Also, why are you writing your own inference code for question-answering and not using HF_TASK and let the inference-toolkit handle everything?

Hi thanks @philschmid I am parsing the input here no?
input_data['inputs']['question'], input_data['inputs']['text']
my thought was that input_data would be a dict in memory. Otherwise what's the point of SageMaker knowing "content_type" if we have to do the deser ourselves for each content type?
the reason I'm doing my own inference code is because that's the thing customers are going to do the most ; they will always want to finely control what happens, what the result is (probas, strings, top-N), and they will also often chain multiple things in the endpoint (eg embedding + kNN, ensembles...). As part of bug bash I just wanted to check that this advanced yet frequent pattern would work fine :)

Hi thanks @philschmid I am parsing the input here no?
input_data['inputs']['question'], input_data['inputs']['text']
my thought was that input_data would be a dict in memory. Otherwise what's the point of SageMaker knowing "content_type" if we have to do the deser ourselves for each content type?

input_fn or preprocess receive the raw HTTP-Body + the content-type to be able to parse the body. E.g. If a customer wants to sent a png he could implement in the input_fn based on the content-type his preprocess function for handling the png.
You always need to parse the body. Take a look here. Furthermore it is documented here or here

ok so my predict(processed_data) missed a : (!). Let me fix that and see what happens.

mm ok so with that new script below I now get a W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - mms.service.PredictionException: ("You need to define one of the following ['feature-extraction', 'text-classification', 'token-classification', 'question-answering', 'table-question-answering', 'fill-mask', 'summarization', 'translation', 'text2text-generation', 'text-generation', 'zero-shot-classification', 'conversational', 'image-classification'] as env 'TASK'.", 403) : 400
import os

import tensorflow as tf
from transformers import TFAutoModelForQuestionAnswering, AutoTokenizer

def load(model_dir):
    """this function reads the model from disk"""

    print('load_fn dir view:')
    print(os.listdir())

    # load model
    model = TFAutoModelForQuestionAnswering.from_pretrained(os.environ['MODEL_FOLDER'])

    # load tokenizer
    tokenizer = AutoTokenizer.from_pretrained(os.environ['TOKENIZER_FOLDER'])

    return model, tokenizer

def predict(processed_data):
    """this function runs inference"""

    print('processed_data received: ')
    print(processed_data)

    question, text = processed_data['input']['question'], processed_data['input']['text']

    input_dict = tokenizer(question, text, return_tensors='tf')
    outputs = model(input_dict)
    start_logits = outputs.start_logits
    end_logits = outputs.end_logits

    all_tokens = tokenizer.convert_ids_to_tokens(input_dict["input_ids"].numpy()[0])
    answer = ' '.join(all_tokens[tf.math.argmax(start_logits, 1)[0] : tf.math.argmax(end_logits, 1)[0]+1])

    return answer

Maybe that's because I didn't specify the task in my SDK call? Since my config.json (below) contains couple hints that it's doing QA I was thinking I wouldn't need to specify the task
{
  "_name_or_path": "distilbert-base-uncased-distilled-squad",
  "activation": "gelu",
  "architectures": [
    "DistilBertForQuestionAnswering"
  ],
  "attention_dropout": 0.1,
  "dim": 768,
  "dropout": 0.1,
  "hidden_dim": 3072,
  "initializer_range": 0.02,
  "max_position_embeddings": 512,
  "model_type": "distilbert",
  "n_heads": 12,
  "n_layers": 6,
  "pad_token_id": 0,
  "qa_dropout": 0.1,
  "seq_classif_dropout": 0.2,
  "sinusoidal_pos_embds": false,
  "tie_weights_": true,
  "transformers_version": "4.8.1",
  "vocab_size": 30522
}

The inference toolkit will automatically detect the TASK based on architectures in the config.json. If you use the model without custom modules it should work.
You receive the error because you didn't overwrite the load function. To overwrite the load function you need to define the load_fn not load. That's, why it is using the regular load function from the default handler. From your other issue #9 I expect that your model_dir structure is wrong.

ok I confirm the print were ignored because I was using a load function. With a load_fn, the print work correctly. nice!

note - the new contract for model loading is model_fn. above-described issue fixed here: https://github.com/aws/sagemaker-huggingface-inference-toolkit/issues/10#issuecomment-870404588


fix: update my collection artwork status post submission

This PR resolves ONYX-999
Description
Update my collection artwork status post submission
https://github.com/artsy/eigen/assets/11945712/43737e25-4ba3-488e-9f82-7c39f9fe33a4

PR Checklist

[x] I have tested my changes on iOS and Android.
[x] I hid my changes behind a feature flag, or they don't need one.
[x] I have included screenshots or videos, or I have not changed the UI.
[x] I have added tests, or my changes don't require any.
[x] I added an app state migration, or my changes do not require one.
[x] I have documented any follow-up work that this PR will require, or it does not require any.
[x] I have added a changelog entry below, or my changes do not require one.

To the reviewers 👀

[ ] I would like at least one of the reviewers to run this PR on the simulator or device.

Changelog updates
#nochangelog
Need help with something? Have a look at our docs, or get in touch with us.

Warnings

:warning:

✅ No changelog changes

  Generated by :no_entry_sign: dangerJS against 7dd2f9338232c587714078b900c5dca0e88f2224


Project Completion QA Checklist: Healthy and type 2 diabetes pancreas

Project UUID: ae71be1d-ddd8-4feb-9bed-24c3ddb6e1ad
Project Title: Single-cell RNA-seq analysis of human pancreas from healthy individuals and type 2 diabetes patients
Project Short Name: Healthy and type 2 diabetes pancreas
Submission UUID: c81f7d54-a27f-4212-a6df-88dde947f7cc
Environment: Production

[ ] Process QA of bundles
[ ] Metadata tsv validation: Run metadatatsv_validator.py

Biocuration is needed for dissociation_protocol fields.
Current metadata:
dissociation_protocol.method.ontology = EFO:0009108
dissociation_protocol.method.ontology_label = fluorescence-activated   cell sorting
dissociation_protocol.method.text = fluorescence-activated cell sorting
dissociation_protocol.protocol_core.protocol_description = Islets were dissociated into single-cell suspension and viable individual cells were distributed by FACS into 384-well plates containing lysis buffer.
FACS is likely the method for single cell isolation from the cell suspension into a plate well. There should also be a method of tissue dissociation (mechanical, enzymatic, etc.) captured in these fields.


test different trie height

add test #528
Description
test different trie height
Changes:

add unit test for different trie height.

Types of changes

Other (please describe):  Add more test

Testing
Requires testing: Yes
Did you write tests??: Yes

Hi @IronGauntlets,
When I put a key to a trie, (the key's length is longer than the trie's height), it work normal as the longer key was truncked in func feltToBitSet().
And while I get() this key's value, it still be truncked. So the finial result works noraml., no panic as we expect in issue.
And I have 2 proposals for this problem, first one is treat this as normal, just like assign uint16 to uint8.
Second method is adding a check in func feltToBitSet(), panic while key out of length.
Which one is do you agree? or is there any better method?

Closing in favor of #668

sorry about this


Allow TextSymbolizer that ignores placement of other data, which scales up and down with zoom

In discussion with brandon, couple of thoughts here

we'd like to inject our own user-created annotations into the rendered basemap
we would like these annotations (E.g. a text layer) to not be automatically hidden like typical map features/labels are — like figma, users should be able to place annotations wherever they want and handle overlaps themselves
api might look something like, in styles.ts:

  {
    dataLayer: `annotation_label`,
    symbolizer: new CenteredTextSymbolizer({
      fill: params.annotationLabel,
      align: 'center',
      font: (z, p) => `...`, // dynamically sizes down/up based on some custom logic
      ignorePlacement: true,
    }),
}

Yup we're thinking the placement and moving would happen in a separate DOM-based layer that is fast and handles all interactions. Then, we want to translate that into a read-only set of MVT data that protomaps can render.
I believe the limitation right now is that without something like ignorePlacement, we cannot explicit control which text gets rendered. We want to allow overlapping text in these kind of situations.

It's not totally clear to me yet what the advantage would be of ever displaying the annotations inside the basemap via protomaps.js, unless the goal is for the annotation labels to have mandatory text display that effectively "knocks out" underlying basemap labels. One way that could work is you do your annotation typing etc via DOM manipulation, and then a "commit" event happens where you inject a pseudo-label bounding box into the basemap that clears out that area for the annotation. This wouldn't be realtime, so the "commit" event could happen after the user is done typing or dragging the annotation.

We could use https://github.com/mapbox/geojson-vt to convert client-side annotation data into vector tiles for display in protomaps.js, but this seems like a roundabout design where the editing data structure is very different than the display data structure. Besides Leaflet, other reference points:
https://openseadragon.github.io/examples/ui-overlays/
iD editor on https://www.openstreetmap.org/edit (requires login)

It's not totally clear to me yet what the advantage would be of ever displaying the annotations inside the basemap via protomaps.js

here are the primary constraints/goals we have which led us to considering this exploration

user-scalable and positionable text > unlike a "leafletmarker" which just repositions a div on the screen, we are syncing the scale in addition to position to the map zoom, and allowing users to further scale the text with some control
fractional zoom levels > we really want the map to feel smooth, with google maps online as a basis. so as you zoom, the text should scale with the map at the tiny fractional zoom levels. however, the way leaflet broadcasts its zoom and the way we do performant animations on our elements, it causes a "lag" where our elements are scaling slightly behind the zoom.
performance > we're using SVG text now and the performance drops precipitously, especially as you zoom in and the SVG gets very large and the browser is redrawing a ton of pixels. other alternatives we've tried to mixed results: using a textarea, div with content editable.

in 99% of the time we expect people to just be panning around and READING the map, vs the 1% time someone is EDITING text
the big theory behind this feature request is that canvas is more performant than SVG, as it can be more intelligent about which parts of text to draw, especially given tiling/zooming
we don't care about knocking out underlying basemap labels bc the way we style the text (colored, maybe stroked) and basemap (subdued) is already enough to make labels behind it a non-issue. instead of injecting a pseudo-label bounding box we'd rather just inject the text itself with its position and scale.
the API might look like this
function useTextTool() {
  function onCreate() {
    // start rendering DOM version to allow user to freely edit, scale adjust
    addElement({...})
  }

  function onSave() {
    // hide the DOM version of the label
    hideElement({...})

    // we call to protomaps to start rendering the text, it handles the positioning, tiling, zooming
    protomaps.addLabel({
      text: string,
      position: LatLngTuple,
      rotation: number, // deg
      baseZoomLevel: number, // the zoom level at which we created it
      scale: number, // further user adjusted scale on top of the baseZoomLevel, where the final scale is Math.pow(2, (baseZoomLevel - map.getZoom()) * scale)
    });
  }

  function onEdit() {
    // show the DOM editable version again and hide the protomaps version
    showElement({...});
    protomaps.hideLabel({id: element.id});
  }
}

does that make sense? are there any flawed assumptions in there? one that jumps out at me was your pushback

Yeah, that makes sense, but my approach would be to use a canvas element on top of the basemap, instead of working inside the basemap layer directly (right now implemented as a grid of Canvas squares).
I think the internal mechanics of editable text will have way different demands than basemap Symbolizers so the code should be separate; you may want manual line breaking for annotations, or a11y; so the 1% SVG + 99% canvas case seems viable, the tricky parts may be to get visual parity across SVG + canvas across browsers.
For the lag you are seeing with animating elements, are you using the leaflet zoomanim event? Syncing animation to leaflet zoom might require you to reverse-engineer the CSS bezier transformation via https://github.com/mapbox/unitbezier , you'll also need to listen to the move event for property animating pinch on mobile.

To elaborate on another train of thought above, I think MVT tiling should be irrelevant to the annotation feature, unless there is multiple megabytes of annotation data. I would store the annotation data as a quad tree (probably https://github.com/mourner/rbush) so you can do windowed queries when panning/zooming

woof, i wish there was a fun youtube education person who did an r-tree
video...
On Sep 7, 2021 at 10:59:43 AM, Brandon Liu @.***> wrote:

To elaborate on another train of thought above, I think MVT tiling should
be irrelevant to the annotation feature, unless there is multiple megabytes
of annotation data. I would store the annotation data as a quad tree
(probably https://github.com/mourner/rbush) so you can do windowed
queries when panning/zooming
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/protomaps/protomaps.js/issues/42#issuecomment-914430272,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAMN5TJ6QVTLJ67FDJCGYCDUAYZG7ANCNFSM5DCSA34A
.

examples of other "canvas + leaflet" approaches:
https://github.com/manubb/Leaflet.PixiOverlay - pretty good sync, because it compromises by transitioning the canvas with CSS + only resizing the canvas at the end of the animation
tangram: https://github.com/tangrams/tangram/blob/master/src/leaflet_layer.js#L387 works around this by replacing leaflet CSS transitions with javascript animations
In general, the syncing of the default leaflet transition to javascript is challenging (maybe impossible?) because CSS transitions happen off the main thread. worth investigating more, but this is a core motivation of why the WebGL mapping libraries (google maps GL version, mapboxGL) use GL all the way instead of letting the browser do compositing.


feat: docker in docker

Codecov Report

Merging #38 (3da84b9) into master (c272c19) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #38   +/-   ##
=======================================
  Coverage   72.89%   72.89%
=======================================
  Files          55       55
  Lines        2446     2446
=======================================
  Hits         1783     1783
  Misses        663      663

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.