[Epic] Metrics Explorer: implement the MVP metrics API backend prototype

Initiative
implement a new backend response structure that works with #311. This should (1) leverage the priority queue so that we can cancel / enqueue lots of updates and (2) should run queries and return the response.
Description
Implement a new backend response structure.
Problem Statement
We need robust API calls that power the front end explore experience.
Definition of Done
This should (1) leverage the priority queue so that we can cancel / enqueue lots of updates and (2) should run queries and return the response.
Resources
https://www.notion.so/rilldata/Implement-the-MVP-metrics-API-backend-prototype-c7df760666674b718bd49c9e58170580
Checklist of Issues
We will need the following API calls. Some of these are contingent on a metrics definition to be created.

[ ] #322
[ ] #323
[ ] #324
[ ] #441
[ ] #442

@AdityaHegde , could you put an update in this epic letting us know how it is going?  I see two of these issues in review, is that part of your ongoing discussion with @bcolloran ?  If not, where is your work related to his asks documented in issues or epics?

There is a working branch for all these tickets : https://github.com/rilldata/rill-developer/pull/383. I also have backend APIs for metrics definition editor in the same PR along with the UI for metric explore.

All the changes are in the working branch #383 along with some basic UI from #204


Support typedefs in idlharness

@zcorpan
This should finish fixing w3c/web-platform-tests#5034.  Let me know if there are any other idlharness changes you want, while I'm thinking about it.

This change is

@zcorpan @jgraham Is someone going to merge this?

Oops, now merged. Thanks!


feat(weekday): support weekday plugin

Dayjs weekday plugin uses weekStart property in the locale config which is not provided currently.
This PR tries to sync the locale config with the official fa locale.
The appropriate test is added for weekday plugin as well.

I think instead of adding locale values to the plugin we can extend official fa locale in line 64 of plugin.js
  dayjs.locale('fa', Object.assign({}, dayjs.Ls.fa, C.fa), true)

@farnabaz This seems to be the correct way but it requires people to load the fa locale before using jalaliday. Otherwise, fa won't be available in dayjs.Ls.

@alimo You're right. What if we load fa locale in the plugin and extend it.
In the end, we need to register fa locale.

@farnabaz I pushed the changes. But since we're importing something from dayjs, size limit package includes the dayjs in the bundle and shows a size limit error. I increased the size limit to ignore this error unless you have a better idea.

Thanks


Swift - Code missing from didReceiveRemoteNotification

Hi,
My project has an AppDelegate.swift instead of AppDelegate.m
I carefully followed the installation instruction. When finished I didn't see any mention of code need to be added to didReceiveRemoteNotification (in appDelegate).
But looking at your example app I can see that in fact there is a code there:
// Required for the notification event.
- (void)application:(UIApplication *)application didReceiveRemoteNotification:(NSDictionary *)notification {
  [RCTOneSignal didReceiveRemoteNotification:notification];
}

Did I miss any step?
Should the link command have added some code (but malfunctioned because I am using swift)?

@yaronlevi This project is intended to work with Objective-C as there would be additional steps to bridge to Swift. Can you switch your project to Objective-C?

Unfortunately, no as there is a healthy amount of code and functionality written in Swift. Is it possible you can provide me the missing steps, or better add them to the docs?
If it is a non-trivial amount of work, it could be a deal breaker for us, and I'll need to implement the Onesignal iOS SDK manually (which in that case I would happily send you a PR once it is working)


Readcomp

@afmckinney @halatmit Note: This change is already in production and works as expected.

Something went wrong...


"Cannot load image from server. Might be CORS related."

this message comes up when attempting to review individual images from a trace. it occurs both in Safari and Firefox, and occurs both with uploading tracks and with tracks that are complete and processed.

@rwelty1889 could you please provide the track id for further check?

http://www.openstreetcam.org/details/3093894/1/track-info
http://www.openstreetcam.org/details/2951802/1/track-info
these are both failing in safari and in firefox on my MacBook air.

thank you, we are checking on this

@rwelty1889 please use link without www to visit before it got fixed
https://openstreetcam.org/details/3093894/1/track-info
https://openstreetcam.org/details/2951802/1/track-info

@tigren
could you please check the tracks
https://openstreetcam.org/details/3097290/0/track-info
https://openstreetcam.org/details/3093090/0/track-info
https://openstreetcam.org/details/3082662/0/track-info
https://openstreetcam.org/details/3082378/0/track-info
https://openstreetcam.org/details/3082350/0/track-info
https://openstreetcam.org/details/3081790/0/track-info
https://openstreetcam.org/details/3075302/0/track-info
https://openstreetcam.org/details/3073770/0/track-info
https://openstreetcam.org/details/3071814/0/track-info
https://openstreetcam.org/details/3068698/0/track-info
https://openstreetcam.org/details/3068634/0/track-info
best regards

@ALFURJANI any problems with the tracks?

@tigren
tracks can't show up on the map !!!
the same problem for both contributors
https://openstreetcam.org/user/aal-ldyn-bn-saayd
and
https://openstreetcam.org/user/mohamedkenan

@ALFURJANI noted, we're checking on this

@tigren
could you please solve the same issue for this list
tracks can't show up
https://openstreetcam.org/user/abdalla-altaboni
https://openstreetcam.org/user/salah-alseleni
https://openstreetcam.org/user/ahmed-elgehani
https://openstreetcam.org/user/saad-abu-matary
https://openstreetcam.org/user/noredeen-imto
https://openstreetcam.org/user/elhusadi

@ALFURJANI has this issue be fixed? I checked some tracks which can show up now.

@rwelty1889 , are you still suffering CORS issue when you open two tracks below?
https://openstreetcam.org/details/3093894/1/track-info
https://openstreetcam.org/details/2951802/1/track-info
If not, could you please close this ticket?

Here's another that suffers CORS issues: https://openstreetcam.org/details/3183478/1/track-info

@joeybab3 , just checked, track 3183478 is being processed.

@JiudanZheng Thanks, as far as the other tracks on my profile that still have 0 points or haven't processed on the main map, should I create another issue so that we can keep this one specific to CORS issues?

@joeybab3 , yes, that's better. Thanks.

@JiudanZheng
still not solved
https://openstreetcam.org/user/mohamedkenan

@ALFURJANI , just checked tracks under mohamedkenan. All have been processed successfully. Could you please give some track numbers? Thanks.

@JiudanZheng
3060138
3196162
3060618
3060690
3061410
3061614
3064478
3067218
3068474
3068518
3075302
3073770
3071814
3068698
3068634
3081790
3082350
3082378
3082662
3093090

@ALFURJANI , I checked all the tracks you listed. I could see all of them on the web map. Except for tracks 3061410 and 3068634. For these two tracks, cuz some photos of them got matching failing, there is no purple line on the map. But if you zoom out up to 18z, you could see the green dot for each photo. We have improved the mechanism of map matching. Right now, the failing matching should be very rare.

I have seen this before. I am the replacement account for #243 and #244 #204 - This is because of a server error. Check back again later.

@brmonaghan , also, about the green dot, I've talked with the team. It's normal as those photos were not on the road. Please check all sequences you have thought have issues. If these issues are confirmed to have been fixed, let's close this ticket.

Ok thank you.

@brmonaghan , could we close this ticket now?

Tianduo Zhai - Grab

Reopening as this still occurs.
Cannot load image from server. Might be CORS related.

https://kartaview.org/details/3395193/0/track-info

Reopening as this still occurs.
Cannot load image from server. Might be CORS related.

https://kartaview.org/details/3395193/0/track-info

+Feng Zhang<EMAIL_ADDRESS>Hi Feng
Could you please help check this? Thanks
On Tue, Jan 26, 2021 at 5:13 AM Bryce Monaghan<EMAIL_ADDRESS>wrote:

Reopening as this still occurs.
Cannot load image from server. Might be CORS related.
[image: image]
https://user-images.githubusercontent.com/71337477/105766862-44a61e00-5f28-11eb-9dfd-c2bbc7ef2aee.png
https://kartaview.org/details/3395193/0/track-info
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/kartaview/openstreetcam.org/issues/294#issuecomment-767116479,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABIEO62CHWOXDHO4E2SBYHTS3XNJJANCNFSM4R7HCR5Q
.

--
[image: Grab (Beijing)] https://htmlsig.com/t/000001BKA99J
[image: Twitter]  https://htmlsig.com/t/000001BKDVDC [image: Facebook]
https://htmlsig.com/t/000001BF8J9Q [image: LinkedIn]
https://htmlsig.com/t/000001BKYJ3R [image: Instagram]
https://htmlsig.com/t/000001BH4CH1 [image: Youtube]
https://htmlsig.com/t/0000001BMMNPF
Tianduo Zhai / Software Engineer
<EMAIL_ADDRESS>/ +86<PHONE_NUMBER>4
Grab (Beijing)
Room 212, Building A, Raycom Infotech Park, 2 Kexueyuan South Road, Haidian
District, Beijing China
https://www.grab.com
[image: Download on the App Store]
https://htmlsig.com/t/000001BEBS2K [image:
Get it on Google Play]  https://htmlsig.com/t/000001BFTMV9
This e-mail message may contain confidential or legally privileged
information and is intended only for the use of the intended recipient(s).
Any unauthorized disclosure, dissemination, distribution, copying or the
taking of any action in reliance on the information herein is prohibited.
E-mails are not secure and cannot be guaranteed to be error free as they
can be intercepted, amended, or contain viruses. Anyone who communicates
with us by e-mail is deemed to have accepted these risks. Company Name is
not responsible for errors or omissions in this message and denies any
responsibility for any damage arising from the use of e-mail. Any opinion
and other statement contained in this message and any attachment are solely
those of the author and do not necessarily represent those of the company.
--
By communicating with Grab Inc and/or its subsidiaries, associate
companies and jointly controlled entities (“Grab Group”), you are deemed to
have consented to the processing of your personal data as set out in the
Privacy Notice which can be viewed at https://grab.com/privacy/
https://grab.com/privacy/
This email contains confidential information
and is only for the intended recipient(s). If you are not the intended
recipient(s), please do not disseminate, distribute or copy this email
Please notify Grab Group immediately if you have received this by mistake
and delete this email from your system. Email transmission cannot be
guaranteed to be secure or error-free as any information therein could be
intercepted, corrupted, lost, destroyed, delayed or incomplete, or contain
viruses. Grab Group do not accept liability for any errors or omissions in
the contents of this email arises as a result of email transmission. All
intellectual property rights in this email and attachments therein shall
remain vested in Grab Group, unless otherwise provided by law.

+Feng Zhang<EMAIL_ADDRESS>Hi Feng
Could you please help check this? Thanks
On Tue, Jan 26, 2021 at 5:13 AM Bryce Monaghan<EMAIL_ADDRESS>wrote:

Reopening as this still occurs.
Cannot load image from server. Might be CORS related.
[image: image]
https://user-images.githubusercontent.com/71337477/105766862-44a61e00-5f28-11eb-9dfd-c2bbc7ef2aee.png
https://kartaview.org/details/3395193/0/track-info
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/kartaview/openstreetcam.org/issues/294#issuecomment-767116479,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABIEO62CHWOXDHO4E2SBYHTS3XNJJANCNFSM4R7HCR5Q
.

--
[image: Grab (Beijing)] https://htmlsig.com/t/000001BKA99J
[image: Twitter]  https://htmlsig.com/t/000001BKDVDC [image: Facebook]
https://htmlsig.com/t/000001BF8J9Q [image: LinkedIn]
https://htmlsig.com/t/000001BKYJ3R [image: Instagram]
https://htmlsig.com/t/000001BH4CH1 [image: Youtube]
https://htmlsig.com/t/0000001BMMNPF
Tianduo Zhai / Software Engineer
<EMAIL_ADDRESS>/ +86<PHONE_NUMBER>4
Grab (Beijing)
Room 212, Building A, Raycom Infotech Park, 2 Kexueyuan South Road, Haidian
District, Beijing China
https://www.grab.com
[image: Download on the App Store]
https://htmlsig.com/t/000001BEBS2K [image:
Get it on Google Play]  https://htmlsig.com/t/000001BFTMV9
This e-mail message may contain confidential or legally privileged
information and is intended only for the use of the intended recipient(s).
Any unauthorized disclosure, dissemination, distribution, copying or the
taking of any action in reliance on the information herein is prohibited.
E-mails are not secure and cannot be guaranteed to be error free as they
can be intercepted, amended, or contain viruses. Anyone who communicates
with us by e-mail is deemed to have accepted these risks. Company Name is
not responsible for errors or omissions in this message and denies any
responsibility for any damage arising from the use of e-mail. Any opinion
and other statement contained in this message and any attachment are solely
those of the author and do not necessarily represent those of the company.
--
By communicating with Grab Inc and/or its subsidiaries, associate
companies and jointly controlled entities (“Grab Group”), you are deemed to
have consented to the processing of your personal data as set out in the
Privacy Notice which can be viewed at https://grab.com/privacy/
https://grab.com/privacy/
This email contains confidential information
and is only for the intended recipient(s). If you are not the intended
recipient(s), please do not disseminate, distribute or copy this email
Please notify Grab Group immediately if you have received this by mistake
and delete this email from your system. Email transmission cannot be
guaranteed to be secure or error-free as any information therein could be
intercepted, corrupted, lost, destroyed, delayed or incomplete, or contain
viruses. Grab Group do not accept liability for any errors or omissions in
the contents of this email arises as a result of email transmission. All
intellectual property rights in this email and attachments therein shall
remain vested in Grab Group, unless otherwise provided by law.

Nevermind, i got this to work. Had to wait for processing to complete.

Nevermind, i got this to work. Had to wait for processing to complete.

Thanks!
We just resolved the delay of processing.
On Tue, Jan 26, 2021 at 9:11 PM Bryce Monaghan<EMAIL_ADDRESS>wrote:

Nevermind, i got this to work. Had to wait for processing to complete.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/kartaview/openstreetcam.org/issues/294#issuecomment-767531435,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABIEO62JYCMP5SXNZDCYJSLS325ORANCNFSM4R7HCR5Q
.

--
[image: Grab (Beijing)] https://htmlsig.com/t/000001BKA99J
[image: Twitter]  https://htmlsig.com/t/000001BKDVDC [image: Facebook]
https://htmlsig.com/t/000001BF8J9Q [image: LinkedIn]
https://htmlsig.com/t/000001BKYJ3R [image: Instagram]
https://htmlsig.com/t/000001BH4CH1 [image: Youtube]
https://htmlsig.com/t/0000001BMMNPF
Tianduo Zhai / Software Engineer
<EMAIL_ADDRESS>/ +86<PHONE_NUMBER>4
Grab (Beijing)
Room 212, Building A, Raycom Infotech Park, 2 Kexueyuan South Road, Haidian
District, Beijing China
https://www.grab.com
[image: Download on the App Store]
https://htmlsig.com/t/000001BEBS2K [image:
Get it on Google Play]  https://htmlsig.com/t/000001BFTMV9
This e-mail message may contain confidential or legally privileged
information and is intended only for the use of the intended recipient(s).
Any unauthorized disclosure, dissemination, distribution, copying or the
taking of any action in reliance on the information herein is prohibited.
E-mails are not secure and cannot be guaranteed to be error free as they
can be intercepted, amended, or contain viruses. Anyone who communicates
with us by e-mail is deemed to have accepted these risks. Company Name is
not responsible for errors or omissions in this message and denies any
responsibility for any damage arising from the use of e-mail. Any opinion
and other statement contained in this message and any attachment are solely
those of the author and do not necessarily represent those of the company.
--
By communicating with Grab Inc and/or its subsidiaries, associate
companies and jointly controlled entities (“Grab Group”), you are deemed to
have consented to the processing of your personal data as set out in the
Privacy Notice which can be viewed at https://grab.com/privacy/
https://grab.com/privacy/
This email contains confidential information
and is only for the intended recipient(s). If you are not the intended
recipient(s), please do not disseminate, distribute or copy this email
Please notify Grab Group immediately if you have received this by mistake
and delete this email from your system. Email transmission cannot be
guaranteed to be secure or error-free as any information therein could be
intercepted, corrupted, lost, destroyed, delayed or incomplete, or contain
viruses. Grab Group do not accept liability for any errors or omissions in
the contents of this email arises as a result of email transmission. All
intellectual property rights in this email and attachments therein shall
remain vested in Grab Group, unless otherwise provided by law.

Thanks!
We just resolved the delay of processing.
On Tue, Jan 26, 2021 at 9:11 PM Bryce Monaghan<EMAIL_ADDRESS>wrote:

Nevermind, i got this to work. Had to wait for processing to complete.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/kartaview/openstreetcam.org/issues/294#issuecomment-767531435,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABIEO62JYCMP5SXNZDCYJSLS325ORANCNFSM4R7HCR5Q
.

--
[image: Grab (Beijing)] https://htmlsig.com/t/000001BKA99J
[image: Twitter]  https://htmlsig.com/t/000001BKDVDC [image: Facebook]
https://htmlsig.com/t/000001BF8J9Q [image: LinkedIn]
https://htmlsig.com/t/000001BKYJ3R [image: Instagram]
https://htmlsig.com/t/000001BH4CH1 [image: Youtube]
https://htmlsig.com/t/0000001BMMNPF
Tianduo Zhai / Software Engineer
<EMAIL_ADDRESS>/ +86<PHONE_NUMBER>4
Grab (Beijing)
Room 212, Building A, Raycom Infotech Park, 2 Kexueyuan South Road, Haidian
District, Beijing China
https://www.grab.com
[image: Download on the App Store]
https://htmlsig.com/t/000001BEBS2K [image:
Get it on Google Play]  https://htmlsig.com/t/000001BFTMV9
This e-mail message may contain confidential or legally privileged
information and is intended only for the use of the intended recipient(s).
Any unauthorized disclosure, dissemination, distribution, copying or the
taking of any action in reliance on the information herein is prohibited.
E-mails are not secure and cannot be guaranteed to be error free as they
can be intercepted, amended, or contain viruses. Anyone who communicates
with us by e-mail is deemed to have accepted these risks. Company Name is
not responsible for errors or omissions in this message and denies any
responsibility for any damage arising from the use of e-mail. Any opinion
and other statement contained in this message and any attachment are solely
those of the author and do not necessarily represent those of the company.
--
By communicating with Grab Inc and/or its subsidiaries, associate
companies and jointly controlled entities (“Grab Group”), you are deemed to
have consented to the processing of your personal data as set out in the
Privacy Notice which can be viewed at https://grab.com/privacy/
https://grab.com/privacy/
This email contains confidential information
and is only for the intended recipient(s). If you are not the intended
recipient(s), please do not disseminate, distribute or copy this email
Please notify Grab Group immediately if you have received this by mistake
and delete this email from your system. Email transmission cannot be
guaranteed to be secure or error-free as any information therein could be
intercepted, corrupted, lost, destroyed, delayed or incomplete, or contain
viruses. Grab Group do not accept liability for any errors or omissions in
the contents of this email arises as a result of email transmission. All
intellectual property rights in this email and attachments therein shall
remain vested in Grab Group, unless otherwise provided by law.

Its not working today.
https://kartaview.org/details/3617729/0/track-info
Is there a problem with the server @zhaitianduo ?
When do you expect this to be resolved?
Thanks

Just ran into this today with a track under processing.
I think there are multiple issues here:

from what I recall, it was possible to access imagery of a track under processing, so the fact that it is not possible at the moment would be a bug
the message itself: users are unlikely to know what "CORS" is, the acronym is to short for a web search to be productive and the message is unlikely to help users take action; in case there's nothing a user could do, the error message could be something like "error accessing image (error code #12345: please use the code when reporting service issues)"

Same here, I am heading to NY tomorrow night and ran into this issue like the last time I uploaded tracks on I-99.

Just ran into this today with a track under processing.
I think there are multiple issues here:

from what I recall, it was possible to access imagery of a track under processing, so the fact that it is not possible at the moment would be a bug
the message itself: users are unlikely to know what "CORS" is, the acronym is to short for a web search to be productive and the message is unlikely to help users take action; in case there's nothing a user could do, the error message could be something like "error accessing image (error code #12345: please use the code when reporting service issues)"

It now says Ups! Sequence Cannot Be Loaded. Reload or Go Back 🤔

Just encountered the CORS message again, track https://kartaview.org/details/6233465/0/track-info

Seeing this again with https://kartaview.org/details/9378345/1/track-info

...anyone?

...anyone?

No one from Grab cares about this project anymore, your last message was from 2022 and got no response from kartaview... Commenting "anyone?" won't do anything except send notifications to everyone who's commented on this issue unfortunately

Fair enough, although it's a shame: if it was a bit better, it would attract more data, might attract more investor attention, might attract some proper funding...but at the very least, this is a good motivator to take a look at https://en.wikipedia.org/wiki/List_of_street_view_services and find something with a bit of life to it. Thanks for the comment.


Issue 45164: Improve HTML error parsing

Rationale
It's possible to have both bad HTML and script
Changes

Assume that bad HTML may have script

Need null check before adding warning?

Indeed. Added


bad

it sucks pretty bad where's the multithreading

Doesn't matter because of rate limits lol, the script is already very fast


Pixelmon 8.2.0 arisen issues

To start, after the new update to 8.2.0, quite a few people voted (including myself), and from that, my vote crate gave 16 ultraballs (where I spent some on a cufant). Since the reversion back before 8.2.0, the vote keys did not restore, thus I had lost some money.
Another issue with the reversion is that I spent 10k on a heart scale to teach one of my legendaries an otherwise unobtainable move, but since the reversion, I had lost the 10k, and the move was reverted as if the heart scale was bought albeit not used.
(It probably says on my logs or the server logs, idk)

Can you give me your username and the list of the stuff lost? As far as I understand, you need 16 ultra balls, and 10k.

Since there was no response I assume it was just as Ras says, 16 Ultra Balls and 10k?

Oops, sorry... That was essentially all, yeah (lol)


allow loading .goplicate.yaml targets from template repository

Hey,
great project - I'd really love to use this in our company and I have been looking for such a thing for a long time :)
One question: In our use case, it would be very helpful to be able to import a set of targets from a remote template repository - or alternatively, sync the .goplicate.yaml file using the tool itself!? (or is this inception? :D )
Would you be open for such a change or have some specific ideas about it? if yes, I could also prepare a PR.
All the best,
Sebastian

Hi @skurfuerst! I'm awfully sorry that it took me 28 days to respond to you (I need to fix my notifications).
First, every suggestion is welcome at this stage as it's an early-stage project with many bright possible evolutions.
To see if I understand you correctly, syncing/importing a target from a remote template repository would mean that after the sync, goplicate would run on these targets, but in order for the target to be changed in a meaningful way, it has to contain the annotations in order to update the relevant blocks in the target files, right? So, I'm not sure how it will work exactly.
Can you elaborate with an example use case? 🤩

Hey @ilaif ,
no worries :)
You are right, the target needs to contain the annotations, if it exists.
I imagine the following scenario:

we have a template repository in v1.
we create a project where we use goplicate.
now, in the template, a few new files appear (think of nginx config files, Github Actions files, ...)
When I run "goplicate" again in my project, I want these new files to appear in the project as well :)

=> Right now, as far as I understand it, this won't work because the .goplicate.yaml needs to contain the new files for that.
That's why I thought as a workaround, of syncing the .goplicate.yaml itself using goplicate -> so on first run, you would update the .goplicate.yaml, and on second run, it would create the new files.
Does the scenario make sense to you, or not? :)
All the best,
Sebastian

I really like that! I think the core functionality that is missing today is the syncing of a whole file.
I'd be happy for you to open a PR and we'll push it together into a fast deployment at your company.
So, feel free to open a PR (draft or not), and let's get this rolling 🤘🏼
We can also start discussing the new API this requires as I feel this will require some thought to scale way with future additions.
I'd also appreciate any honest feedback about the current codebase and what can be improved 🙏🏼

Awesome :) I will check it out and prepare a PR.
It might take a few weeks until I get back to you, as I currently have lots of open todos on my table.
All the best,
Sebastian

Hi @skurfuerst,
How is it going?
I wanted to share that we have released v0.2.0 that adds support for remote git URLs (instead of assuming your projects are already checked out locally).
This mainly helps with the automation of the whole flow and reduces errors since the fresh clone is always clean 🧹
Let me know if you already had some time to take a look at your use case 🙏🏼

Hi @skurfuerst,
I've allowed myself to finish this proposed feature in v0.2.1.
Basically, you can now add a config-sync option to your .goplicate.yaml and have it sync itself from a source target. See https://github.com/ilaif/goplicate/tree/main/examples/sync-config for an example.
I'm reminding you that all source paths can now be synced from a repository, which makes it stateless and clean!
Please let me know if that works for your use case 🙏🏼 🎉

Awesome, really cool! Thanks so much :):)
I will test it in the upcoming days and give feedback :)
All the best,
Sebastian

Reopening until @skurfuerst confirms this works 🙏🏼

Sorry, did not get around doing this over the holidays - a happy new year to you 😊 I will test it soon.
All the best,
Sebastian

Happy new year, @skurfuerst !


Cannot find the reference label_map_util in init.py. init.py file is empty ? how to solve this ? all the init files are empty !

@imuralice Recheck the line where error occured for brackets. Make sure no extra brackets are there.

@imuralice Recheck the line where error occured for brackets. Make sure no extra brackets are there.


Reordering of predicates

Is your feature request related to a problem or challenge?
The optimizer reorders the evaluation of the predicates in a WHERE clause according to their assigned execution cost.
If the first predicate (that is, the cheapest) in an OR optimization evaluates to TRUE, the more expensive predicates (that is, any that follow) do not have to be evaluated. Similarly, in an AND optimization, if the first predicate evaluates to FALSE, the more expensive predicates do not have to be evaluated.
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response

@Ted-Jiang
I think this is in your todo also.

@Ted-Jiang I think this is in your todo also.

@mingmwang  Thanks for ping me! I will look this later.

I think we can construct a bypass plan.


social login exception

After the login success, back to the login page, the use of social login, will be an exception.
Page display:
[Un authorized operation]
[You are not authorized to perform this operation]
I think we should be able to directly use the social login.

What can be done to achieve this effect?

Can you share stacktrace, full exception message and which template you are using?

Is not an exception error, should be logged in the process of BUG.
In the case has been successfully logged in, and then request a social login, this problem will appear.
This could be caused by UseCookieAuthentication. Because the login Cookie already exists.
My current solution is to enter the Login Action, the use of _signInManager.SignOutAllAsync (), and then re-enter the Login Action.
But I feel that this is not a good solution.

When user clicks to social login button, you should logout the user before continue. I think this is reasonable, why you didn't like it?
BTW, are you using AspNet Core version with module-zero? If so, it may be related to https://github.com/aspnet/Announcements/issues/201

I see.
Thank you.


Bloomberg Feedback

At Bloomberg, we have been using lazy module evaluation in production for a long time. We have proved multiple times that it's essential for better application startup performance. Hence we are very happy to see this proposal and will invest in its standardization.
This proposal, as it stands, is generally inline with our view. Please see our detailed feedback below.
Existing Use Cases
Our module loader speaks AMD. Historically, AMD has also been the authoring syntax for modules. This has changed in the last few years. Modules are now authored in ESM syntax by default and AMD is considered legacy as an authoring format. ES modules can interoperate with AMD and vice versa. In terms of interop, it would be fair to consider an AMD module as an ES module with a single default export.
Currently we have two primary features that give users lazy evaluation capabilities. These are designed to be used more in the producer modules when building module exports.

In AMD, a programmatic API lets users create objects with getter functions as a proxy to other modules, i.e. re-exports.
In ESM, a compiler-based feature that makes all re-exports of a module lazy. This feature is heavily restricted.

Both of these are used in critical parts of the infrastructure. They don't take special care for asynchronous modules, i.e. ESM using top-level await, or AMD using a special async formulation. Users are expected to create safe dependency graphs by knowing what children to preload ahead of time. This makes the feature very brittle, which we are actively working to address (see below).
1. Default-exported Object with Lazy Properties
In this pattern, the default-exported object contains getter functions that load different target modules on first access.
define(["make-lazy", "sync-require"], (makeLazy, require) => {
    const exports = {};

    makeLazy(exports, "Label", "./label", require);
    makeLazy(exports, "Button", "./button", require);

    return exports;
});

Using the current proposal, this can be implemented as follows:
import * as Label from "./label" with { lazyInit: true };
import * as Button from "./button" with { lazyInit: true };

export default {
    get Label() { return Label.default },
    get Button() { return Button.default },
};

This requires .default for each namespace which is a little inconvenient.
2. Lazy Namespace of (Lazy) Namespaces
In this pattern, a mega lazy namespace is constructed by re-exporting other namespaces. A special pragma tells the build tooling to transform eager static re-exports to lazy evaluation.
/* "use special pragma to make re-exports lazy at build time" */
export * as TableUtils from "./table-utils";
export * as WindowUtils from "./window-utils";

This is possible to implement using the current proposal as it's allowed to pass the DeferredModuleNamespace exotic object around without breaking its lazy nature.
import * as TableUtils from "./table-utils" with { lazyInit: true };
import * as WindowUtils from "./window-utils" with { lazyInit: true };

export { TableUtils, WindowUtils };

It cannot be expressed concisely only using re-export statements, though.
3. Lazy Namespace of Named Re-exports
The same pragma can be used to construct a namespace where only selected named bindings are re-exported lazily. Although this pattern is supported, we don't have many users of it today.
/* "use special pragma to make re-exports lazy at build time" */
export { Table } from "./table";
export { Window } from "./window";

This cannot be expressed in the current proposal as it only allows namespace imports and there is no way to export a function that works like a getter.
Implementation of Deferred Module Evaluation
We have implemented deferred module evaluation in our module system. It is now used by a few early adopters in production. As background, this module system is not known to the JS engine. It is almost entirely implemented in JS and is constructed during an initial bootstrapping phase that runs before the main application module is loaded.
Most of the complexity we found when implementing this feature arises due to handling of asynchronous modules in the graph. Supporting top-level await is important in our system, which is why we have invested in standardization of it in the past. So this was a critical requirement for us. Handling of asynchronous modules is achieved by a combination of metadata collected at build time and preloading asynchronous nodes in the runtime. Build-time metadata is reliable because we enforce static analyzability of dependency graphs. This metadata enables an optimization that makes the preloading traversal in the runtime more efficient by avoiding parsing modules which makes discovery of asynchronous modules possible without the need to explore the whole static module graph.
We are aligning with the current state of the proposal as much as we can by

using a comment based syntax in import declarations with a goal of moving to Import Attributes and eventually to the final standard syntax
only allowing namespace imports - named imports are banned

Thanks for the feedback -- closing to do some cleanup :)


description test

add description property

Dont merge dev into main!

Please create branch and merge this into main (and update dev afterwards)


[BUG] Dict fields not shown on session viewer

Instructions
Describe the problem
Adding a dict field to a Sample works fine in the code, but the session viewer does not show any values for the field, also grouping it under "other". I'm not clear if this is a bug or if I misread the docs, because it was happening before to me with some Keypoints until I created them correctly, but from what I understand in the docs the dict is base type that should be shown correctly like strings and lists. Also, with a list of mixed types I am getting the same problem.
Code to reproduce issue
import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")
view = dataset.take(1, seed=51)
sample = view.first()

sample["dict_example_not_showing"] = {"test_1": 3, "test_2": 4}
sample["list_example_not_showing"] = [1, "even_strings", 3.14]
sample["list_example_ok"] = [1, 2, 3.14]
sample["string_example_ok"] = "perfection"
sample.save()

session = fo.launch_app(desktop=True)
session.view = view
session.wait()

System information

OS Platform and Distribution (e.g., Linux Ubuntu 22.04): Ubuntu 22.04.3 LTS
Python version (python --version): Python 3.10.12
FiftyOne version (fiftyone --version): FiftyOne v0.23.2, Voxel51, Inc.
FiftyOne installed from (pip or source): pip

Other info/logs
If I debug the example code just before the line to view the session, I can see that the dict is correctly saved in the sample field like the lists and strings example, but it is not shown correctly in the viewer (see images below)

Willingness to contribute
I am not working in a big organization, but would be willing to make a small contribution, the problem is I have no idea how the viewer works or where this problem could be.

[ ] Yes. I can contribute a fix for this bug independently
[X] Yes. I would be willing to contribute a fix for this bug with guidance
from the FiftyOne community
[ ] No. I cannot contribute a bug fix at this time

Encounter the same issue.


updates most name changes, need to clarify on others

Why
Need to update tech docs to reflect new name change. This is happening in tandem with Design for Adoption team.
What
Updates name across tech docs.
Technical writer support
This is getting tech writer support.
How to review
Check name is updated across site.

Updates most name changes - need to clarify on others (e.g. anchor links), and also need to update diagrams


Adding some quantum information functions to qi module

Description
Added some additional features to qi module and improved some older functions:

partial trace more efficient if input is a state vector
Added several entropy and mutual information functions
Added functions for generating random density matrices
fixed a couple of bugs with chop function, and for list inputs of state vectors

How Has This Been Tested?
Unit tests pass on Python 3.6
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

Thanks  @chriseclectic - merging!


52 create glossary page

Description

Added a glossary page with the following functionality:

Shows a list of glossary entries which consists of a term, tags and a description
Two buttons to collapse and expand all entries
A filter component to filter entries for specific tags
An index component to filter entries for their first letter
A searchbar to search for a string inside all entries

Type of changes

[ ] Bugfix
[x] Feature
[ ] Refactoring
[ ] Optimization
[ ] Documentation
[ ] CI/CD
[ ] Other

Requirements

[x] Everything build out of Material UI components
[x] New Material UI components capsulated via dependency injection in "@common/components"
[x] Provide basic information pieces in comments
[x] Establish interface to backend (if necessary)
[x] Use Skeleton components if something has to be fetched (if necessary)
[x] Error management: Components should handle undefined inputs
[x] Components should be resuable (if possible)
[x] Component Requirements are met

SonarCloud beim Erkennen von Security Issues:


Issue with simple Gaussian distribution

Hi,
First of all thanks for this wonderful implementation.
I am now trying to work a simple example of using SSGE for estimating score function for N(0, 1) function. Please find below my code:
import torch
import gpytorch
from ssge import SSGE

def test_gaussian():
    import matplotlib.pyplot as plt
    plt.style.use('seaborn')

    dist = torch.distributions.normal.Normal(
        torch.tensor(0.0), torch.tensor(1.0)
    )

    kernel = gpytorch.kernels.RBFKernel()

    # estimator = SSGE(gpytorch.kernels.ScaleKernel(gpytorch.kernels.RBFKernel()))
    estimator = SSGE(
        gpytorch.kernels.ScaleKernel(gpytorch.kernels.RBFKernel()),
        noise=1e-4
    )

    sample = dist.sample((100, 1))
    logp = dist.log_prob(sample)
    sample.requires_grad_(True)
    estimator.fit(sample)

    # test_points = dist.sample((25, 1))
    test_points = torch.linspace(-2, +2, 250)
    test_points.requires_grad_(True)
    grad_the = -test_points
    grad_est = estimator(test_points)

    # Plot results
    fig, ax = plt.subplots(1, 1)
    ax.scatter(sample.detach().numpy(), logp.detach().numpy(),
               marker='o', s=2.0, label='Sample points')
    ax.plot(test_points.detach().numpy(), grad_the.detach().numpy(),
            c='red', lw=1, label=r"$\nabla_x")
    ax.plot(test_points.detach().numpy(), grad_est.detach().numpy(),
            c='blue', lw=1,  label=r"$\hat{\nabla_x}")
    plt.savefig("out/test_gaussian.png")

test_gaussian()

However, when I run the example, the estimated gradients are very much different than the actual ones. (See the figure attached)
How this can be improved?

Thank you for your interest to this repo. I am sorry for the inconviniences. I will work on this issue in this week.

I assume problem is at the core algorithm. I check it again.

Sorry for the late response. In the test you generated, average value of the estimated gradient (blue line) resembles log density, not the derivative of log density. It is highly probable that I implemented something wrong and I am analyzing the original paper again.
However, I guess noise in gradient estimate are expected even in the complete implementation, because of 2 reasons:

Optimization with respect to Kernel Hyperparameter is needed. In the simulated example like analytical distributions, for optimization, loss can be just MSE of estimated gradient and theoretical gradient. I don't understand what is the loss for the implicit distribution for now.
Small sample size.


Milestone: preserve data in the browser

Milestone: preserve data in the browser

When the user changes the any input field, the data is saved to the local storage.
When loads the page, if there is any data in the local storage the input fields are pre-filled with this data.

👍 Thank for any suggestion to improve this project

Congrats @jicamargo enjoy your meeting tomorrow.
Javier.-


Add feature to assign and unassign tags to contacts

Fixes 3/4 and 4/4 of #39.

Codecov Report

Merging #83 (f9c5432) into master (251534e) will decrease coverage by 0.24%.
The diff coverage is 66.66%.

@@             Coverage Diff              @@
##             master      #83      +/-   ##
============================================
- Coverage     70.55%   70.31%   -0.25%
- Complexity      541      566      +25
============================================
  Files            93       97       +4
  Lines          1722     1836     +114
  Branches        196      215      +19
============================================
+ Hits           1215     1291      +76
- Misses          442      472      +30
- Partials         65       73       +8

Impacted Files
Coverage Δ

...seedu/address/logic/commands/CreateTagCommand.java
100.00% <ø> (ø)

.../seedu/address/logic/parser/AddressBookParser.java
61.53% <0.00%> (-5.13%)
:arrow_down:

...u/address/logic/parser/AssignTagCommandParser.java
0.00% <0.00%> (ø)

...address/logic/parser/UnassignTagCommandParser.java
0.00% <0.00%> (ø)

...seedu/address/logic/commands/AssignTagCommand.java
90.47% <90.47%> (ø)

...edu/address/logic/commands/UnassignTagCommand.java
90.47% <90.47%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 251534e...f9c5432. Read the comment docs.


Document GKE cloud DNS deployment using LRP

Extend the LRP doc with the GKE Cloud DNS deployment setup similar to other use cases like node-local DNS - https://docs.cilium.io/en/latest/network/kubernetes/local-redirect-policy/#use-cases.
Context - https://github.com/cilium/cilium/pull/33721.

Check in an LRP spec with AddressMatcher and skipRedirectFromBackend flag set to true.
Document steps to enable this in GKE deployments.

/cc @ysksuzuki

@aditighag Please assign this to me.


MongoDB findUnique() or findUniqueOrThrow() leads to COLSCAN if multiple queries happen at the same time

Bug description
in mongodb a simple findUnique or findUniqueOrThrow can lead to a colscan if multiple queries to the same collection happen at the same time:
prisma.user
    .findUnique({
      where: {
        id: "some-id"
      },
      select: {
        roles: true,
      },
    })

prisma.user
    .findUnique({
      where: {
        id: "some-id-2"
      },

    })

it creates this query:

[
      {
        "$match": {
          "$expr": {
            "$and": [
              {
                "$in": [
                  "$_id",
                  {
                    "$literal": [
                      "some-id",
                      "some-id-2"
                    ]
                  }
                ]
              },
              {
                "$ne": [
                  "$_id",
                  "$$REMOVE"
                ]
              }
            ]
          }
        }
      },
      {
        "$project": {
          "_id": 1,
          //...
        }
      }
    ],

which leads to a colscan because of this bug: https://github.com/prisma/prisma/issues/21453 which can take several seconds depending on the collection size and load
while this query which you would expect takes only some miliseconds:
{

          "$expr": {
            "$and": [
              {
               "_id": "some-id"
              },
              {
                "$ne": [
                  "$_id",
                  "$$REMOVE"
                ]
              }
            ]
          }

}

findUnique or findUniqueOrThrow do not have this problem
How to reproduce
The error happens if there are multiple queries at the same time
i assume this is because an attempted optimization in prisma: prisma batches multiple queries on the same collection and this leads to a id: {$in: [..]} aggregation. This is slow in prisma https://github.com/prisma/prisma/issues/21453
This would also explain why findFirst does not have this issue, since you can't batch that in the same way.
EDIT: even findFirst sometimes creates faulty aggregation that lead to a COLLSCAN.
Expected behavior
findUnique and findUniqueOrThrow never do a colscan. They are expected to be fast and efficient
Prisma information
see above
Environment & setup

OS: macos
Database: mongodb atlas
Node.js version: 18

Prisma Version
prisma                  : 5.9.1
@prisma/client          : 5.9.1
Computed binaryTarget   : darwin-arm64
Operating System        : darwin
Architecture            : arm64
Node.js                 : v20.11.0
Query Engine (Node-API) : libquery-engine 23fdc5965b1e05fc54e5f26ed3de66776b93de64 (at<EMAIL_ADDRESS>Schema Engine           : schema-engine-cli 23fdc5965b1e05fc54e5f26ed3de66776b93de64 (at node_modules/@prisma/engines/schema-engine-darwin-arm64)
Schema Wasm             : @prisma/prisma-schema-wasm 5.9.0-32.23fdc5965b1e05fc54e5f26ed3de66776b93de64
Default Engines Hash    : 23fdc5965b1e05fc54e5f26ed3de66776b93de64
Studio                  : 0.497.0

seems like the same problem as mentioned here https://github.com/prisma/prisma/issues/17142#issuecomment-1648791852
this is shocking to say the least. How can a simple findUnique ON THE ID lead to a collscan? And how can such a serious problem land in a release version of prisma?

i did further research and it seems removing select does not fix it.
it seems the bug is that prisma accidentially passes the same id multiple times:
db.users.aggregate([ { $match: { $expr: { $and: [ { $in: [ "$_id", { $literal: [ "some-id", "some-id", ], }, ], }, { $ne: [ "$_id", "$$REMOVE", ], }, ], }, }, }, { $project: { _id: 1, /* other fields...*/ }, }, ])
^[

```

another one for https://github.com/prisma/prisma/issues/14871 ...

its a very weird bug and now i also observed it with findUniqueOrThrow. It seems to be some concurrency problem and definitly a serious bug in the query engine. If there are multiple queries at the same time, they sometijmes receive multiple ids. I think this is an attempt of optimization so that multiple queries at the same time do not lead to multiple requests
however due to very poor optimizations of mongodb queries in prisma this leads to a full table scan (colscan)

@Druue i saw that you labeled the issue. I just wanted to inform you that i added more information and i have a possible explanation why this happens (prisma's batching attempt going wrong). I think its critical issue and I really hoped this gets fixed with priority

Now that you understand the problem, can you please describe it again in clear words?
Does this only happen when Prisma's batching kicks in? Or also in other cases? If yes, what is is a good reproduction scenario to see this ourselves? We would like to run one query and see it being fast, and then another set of queries where we can see it is slow.
If it is indeed the batching, we can possibly just disable it for these cases where this leads to problems.

Now that you understand the problem, can you please describe it again in clear words?
Does this only happen when Prisma's batching kicks in? Or also in other cases? If yes, what is is a good reproduction scenario to see this ourselves? We would like to run one query and see it being fast, and then another set of queries where we can see it is slow.
If it is indeed the batching, we can possibly just disable it for these cases where this leads to problems.

I am pretty sure its batching, so it should be easy to reproduce by just calling multiple findUnique on the same collection without awaiting them

set

export const prisma = new PrismaClient({
  log: ["query"],
});

then:

await Promise.all(["id1", "id2", "id3"].map(id => prisma.MyModel.findUnique({where: {id}})))`

you then see that it creates quite complicated queries using $in in a way that cannot use the index ($in can use the index, but not how prisma generates those queries)
to see that those queries are indeed lead to a colscan, you can force mongodb to not allow colscan https://www.mongodb.com/docs/manual/reference/parameters/#param.notablescan
then, compare that with using findFirst. Those queries do not use $in, since they can't be batched the same way.
Instead of disabling batching, it would actually be better to fix the underlying issue which is documented here https://github.com/prisma/prisma/issues/19955
i understand that mongodb queries are harder to optimize when multiple collections need to be accessed at the same time.
But in cases where an indexed field is in the selector, they should be never a colscan. Colscan can be devasting not only for performance but also for costs.

@janpio is it possible to disable batching? This bug is very serious and we need to take immediate action

@macrozone if you want to opt out of batching, you can use findFirst(OrThrow) instead of findUnique(OrThrow).
If you are using composite unique you will need to flatten where:
prisma.model.findUnique({
  where: {
     name_email: {
         name,
         email
     }
  }
})

becomes:
prisma.model.findUnique({
  where: {
      name,
      email
  }
})

For plain unique, just replacing findUnique with findFirst is enough.

@macrozone if you want to opt out of batching, you can use findFirst(OrThrow) instead of findUnique(OrThrow).
If you are using composite unique you will need to flatten where:
prisma.model.findUnique({
  where: {
     name_email: {
         name,
         email
     }
  }
})

becomes:
prisma.model.findUnique({
  where: {
      name,
      email
  }
})

For plain unique, just replacing findUnique with findFirst is enough.

yeah thank you, I had the same observation.
However i would like to know whether there is a way to disable it completly, because It's hard to explain to devs why they should not use findUnique when using mongodb. Also when this bug is fixed, I would like to revert it back. So disabling it on the client level would be nice.

@macrozone you could do a client extension that would convert findUnique to findFirst automatically.

@macrozone you could do a client extension that would convert findUnique to findFirst automatically.

yeah, i also thought about that, or creating a custom eslint rule ;-)

but i think prisma should just just disable the batching for mongodb until https://github.com/prisma/prisma/issues/19955 is fixed

@macrozone If the queries are also slow when not being batched, please look for existing issues that refer to that case or open a new one with the full information. Thank you.

@macrozone If the queries are also slow when not being batched, please look for existing issues that refer to that case or open a new one with the full information. Thank you.

Thank you @janpio
I removed all findUnique calls and replaced them with findFirst, but I still see slow query logs like this. The codebase has no findUnique anymore. Those are guaranteed findFirst (or findFirstOrThrow) calls. However I can't reproduce it locally, there i don't see those queries (by using prisma's log feature).
If prisma would at least create $eq expressions instead of $in if there is only one _id, it would already mitigate this problem, since mongodb would then use the index.
I think a developer of the query engine needs to look at this, since its probably a rather easy fix. Also they probably know under which conditions a $in is used (batching or some attempted optimization).

I remember our discussions.
I am serious though, a quick issue from you with a plain and obvious reproduction would go a long way. You clearly have a reproduction case now, and as a new issue we would also try to reproduce and then potentially prioritize that.

I am serious though, a quick issue from you with a plain and obvious reproduction would go a long way. You clearly have a reproduction case now, and as a new issue we would also try to reproduce and then potentially prioritize that.

The initial description is still correct. findUnique(OrThrow) lead to a colscan due to wrongly implemented batching.  I hope its clear how problematic this issue is. (and many of the other reported and well documented mongodb performance problems.
I can offer to create a repo with some test cases though, if the prisma team does not have a setup to test mongodb.

We are facing similar issues with these non-optimized queries. Some of our collection grew a lot in the recent weeks and now these COLLSCAN became a serious issue. A simple findUnique is taking +30 seconds to resolve on a 2.5mi records collection.
By enabling the debug on Prisma queries, we can tell all of it is due to the misusage of $expr + $in. Seems like any Prisma query will lazily just wrap your query with the $expr command instead of building a more specific query.
As a workaround, we manage to replace some of our queries with a Prisma OR operator.
So instead of something like
prisma.users.findMany({ where: { id: { in: sortedMemberIds } })

We replaced with
prisma.users.findMany({ where: { OR: sortedMemberIds.map((id) => ({ id })) }})

It's a silly workaround and doesn't work for all the cases, but it helps for now. The problem is even worse in our project since we expose a GraphQL endpoint, and rely on Prisma/Nexus/PalJS generated resolvers. All of those will generate the offender queries. To mitigate, we introduced custom resolvers that will do the workaround query described above. It's a huge drawback to the project, as we need to handle each individual resolver manually now.
From quickly scanning Prisma engine codebase, I think calls like these here are the culprit (I'm nowhere near to understand Rust code 😅). Instead of just passing the query to the $expr command, it should be building the mongodb specific query depending on the requested filters. Not a trivial task, I guess, but it's what I'd expect from a data access library.
 We manage to work around some of them, but it's becoming unsustainable. It was first introduced because we wanted to better integrate with our new GraphQL services and not having to write resolvers manually for everything. As it stands now, we are back on writing new code with mongoose.

findUnique(OrThrow) lead to a colscan due to wrongly implemented batching.

I don't understand. I thought you replaced all findUnique() queries (which are batched) with other queries that are not batched, and you are still seing the problem - which indicates that batching is not at fault. What am I missing?

I can offer to create a repo with some test cases though, maybe i can do that over the weekend.

That would be very welcome, thank you.

I don't understand. I thought you replaced all findUnique() queries (which are batched) with other queries that are not batched, and you are still seing the problem - which indicates that batching is not at fault. What am I missing?

I still see colscans on production for unique ids without having findUnique. (only findFirst and firndFirstAndThrow).
But i could not reproduce that locally yet, while I could reproduce the findUnique cases.
So I would argue that we should keep the issue here about findUnique(andThrow), were there is strong evidence and a guess why this happens (batching).
My hope is that this bug here is fixed in a short time, since it is rather obvious and will affect all users of prisma and mongodb.
In the meantime I check why I still see collscans for "findFirst" and if i find other evidence and will open a new issue for that.

I checked the logs again from the production, made sure that it does not run any old version of the code where findUnique or findUniqueAndThrow is used and still see quite a lot of COLLSCANS from $in-queries on the primary key. I created therefore a separate issue https://github.com/prisma/prisma/issues/23775

@janpio i created a repo that show the bug described here. it also shows that findFirst should not have this issue
https://github.com/macrozone/prisma-mongodb-collscan-repro
Why i still have https://github.com/prisma/prisma/issues/23775 is up for investigation

Our release 5.14.0 significantly improved the performance of certain queries when using MongoDB, specifically ones with include and in should now not cause a collection scan anymore and use an index instead to find the correct documents. I assume this was also the underlying problem that was reported in this issue - so I am going to close this for now.
If you still encounter any performance problems with Prisma and MongoDB, please open a new GitHub bug issue. Thanks!


Type definitions on enums

I have a enum, where some of its variants contain nested data. Some of that data are foreign structs that do not impl the TS trait.
Inside structs we have the possibility to use attributes like #[ts(type = "Vec<usize>)] for this case, but this attribute fails on enums telling me type is unknown and only these are allowed: "rename", "rename_all", "inline", "skip", "untagged".
I could implement TS on these structs, but that would require me to write some boilerplate.
Here is a minimal setup of my usecase:
use serde::{Deserialize, Serialize};
use time::{Duration, Weekday};
use ts_rs::TS;

#[derive(Debug, Clone, Serialize, Deserialize, TS)]
#[ts(export)]
pub enum MyEnum {
    // #[ts(type = "Vec<isize>")]
    Variant1(Duration), // time::Duration does not impl TS
    Variant2(usize),
    // #[ts(as = "[usize, String]")]
    Variant3((usize, Weekday)), // time::Weekday does not impl TS
}

fn main() {
    println!("Hello, world!");
}

[package]
name = "rust-playground"
version = "0.1.0"
edition = "2021"

[dependencies]
ts-rs = "9.0.1"
serde = { version = "1.0.182", features = ["derive"] }
time = { version = "0.3.31", features = ["serde", "local-offset", "parsing", "formatting"] }

Hi @JonasFocke01! There's a PR for that (#284), can you check if your code works with it?


2 keys for aim&trigger bot Updated

Just an update for the last Version

Nice Job

could you update？？it dosent work now.thanks！


Target group create_before_destroy

what

Add create_before_destroy lifecycle to target group

why

Prevent ResourceInUse error with the target group when making changes

references

Closes https://github.com/cloudposse/terraform-aws-alb-ingress/issues/24

/test all


[main] Upgrade to latest dependencies

testing update deps -dprotaso
/cc knative/operations-wg-leads
/assign knative/operations-wg-leads
Produced by: knative-sandbox/knobots/actions/update-deps
Details:
bumping knative.dev/pkg 6c9c1c6...ed7ab1d:
  > ed7ab1d upgrade to latest dependencies (# 2559)
bumping golang.org/x/sync 0de741c...886fb93:
  > 886fb93 A+C: delete AUTHORS and CONTRIBUTORS
bumping knative.dev/serving 808923d...2a22c44:
  > 2a22c44 upgrade to latest dependencies (# 13147)
bumping knative.dev/networking 1f4c9ce...cddb0ff:
  > cddb0ff Stop using knative.dev/pkg/pool, use errgroup instead (# 695)
bumping knative.dev/caching 783e336...cab258d:
  > cab258d pull in latest knative.dev/hack changes (# 662)

/lgtm
/approve


Command Prompt closes

When I excecute stadiacontroller.exe a short command prompt window appears (accompanied by a windows connecting jiggle sound), and the windows closes immediately after than.  Earlier today it worked fine (the cmd window stayed opened), but after a fresh Windows 10 Pro installation, it just closes and my controller isn't working in games anymore.
Any ideas?

Is ViGEm installed on your fresh installation?

Is ViGEm installed on your fresh installation?

ViGEmBusSetup_x64.msi is installed. I did it before and it worked with the old Windows 10 installation.

Got it. You could try starting your own cmd.exe prompt and executing stadiacontroller.exe from there; if there is an error, it should be displayed and the prompt won't close on its own before you can read it.

Got it. You could try starting your own cmd.exe prompt and executing stadiacontroller.exe from there; if there is an error, it should be displayed and the prompt won't close on its own before you can read it.

Yeah, good idea. I got an error indeed.

"Overlapping E/A-process in progress/the working."

I'm not sure about this. I remember having a bug about "overlapping" a while ago, but can't remember exactly what it was or how it was fixed. If I had to guess, I'd say some other process is taking exclusive control of the emulated X360 controller. Perhaps Steam?

Nevermind, it works now. Maybe one of that Windows Updates I did yesterday did the trick. Who knows.


Page header transparency

Description
We should be able to change the transparency of the top bar on the page.
It should either have it's own setting in the customization page, or it could follow the app opacity slider.
(I think I'd prefer the header to have it's own opacity slider.)
Priority
Low (Nice-to-have)

Out of consistency with the past versions, it should follow the app's opacity settings.
That's how it behaved previously.
All that is available to change in custom CSS though.

Still relevant as it's a feature request.


Add more colors

[ ] Add up and down arrows next to values in Ticker Data table (and other tables, if appropriate) that are colored either green or red if the value has increased or decreased since t - 1
[x] POS/NEUT/NEG sentiment values should be colored appropriately. BUY/SELL signals should be colored appropriately

up and down arrows

We'll need to import a special charset/people will need to use a patched font like I have for zsh...
Maybe this isn't a good idea.

Or this: https://www.npmjs.com/package/figures


[intel-mkl] Modify the lib storage path under Windows

Fixes https://github.com/microsoft/vcpkg/issues/35671

Change the library storage path for the port under Windows from ${CURRENT_PACKAGES_DIR}/lib/intel64 to ${CURRENT_PACKAGES_DIR}/lib.
Update the lib paths in the PC files under Windows.

[x] Changes comply with the maintainer guide
[x] SHA512s are updated for each updated download
[ ] The "supports" clause reflects platforms that may be fixed by this new version
[ ] Any fixed CI baseline entries are removed from that file.
[ ] Any patches that are no longer applied are deleted from the port's directory.
[x] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[x] Only one version is added to each modified port's versions file.

AFAIU intel-mkl was intentionally added und accepted with this installation layout in
https://github.com/microsoft/vcpkg/pull/29437
CC @Neumann-A

This modification is solely intended to fix the issues in the following scenarios:

Run the vcpkg integrate install command.
Directly invoke MKL in a Visual Studio Console program.

I understand for which effects the change is done. My point is that there may be unintented side effects, such as breaking official find modules, user find scripts, or even msbuild autolinking.

You'll get symbol conflicts with LAPACK and BLAS.... just saying.

We're using the msbuild autolinking with vcpkg, how should symbol conflicts and finding of these libs be handled.
Is there some way to prepend this search path for these symbols?
We're not building / installing either BLAS or LAPACK in the vcpkg installed folder.
Is there a way to disable the autolink and we manually specify which libs we wish to link to.
That would seem preferable to inadvertently linking the wrong library if there's a chance of conflicts.
Not up to speed on the msvc linking magic with vcpkg.
I also see the ilp64 only libs installed into lib\intel64.
I believe this requires MKL_ILP64 to be defined. But I don't see any reference to this in the scripts.
What would the prefered method be to set this so all dependent ports are built with this set?


Syllabification improperly handles "suavissimum"

In an email from Debra:

"Suavissimum" is being split into too many syllables here. "sua" should be just one in this case. But in the case of the single word "sua" it would be two syllables.
https://cantusdatabase.org/chant/588598
Is our system able to handle this?

Catching up on this now....
From @annamorphism's email, it looks like there are differences in the syllabification of this word (and of suav constructions generally) that we want to maintain, and that this will be a place where manual editing in one case or another will be necessary.
Is that a correct reading of the discussion?
@annamorphism, can you confirm that "su-a" is what we want as default and "sua" is an 'exception' that we will make manually? I think that's what your email says, but before I close this issue I want to make sure.

Correct: as far as I can tell, su-a is applicable in more cases than "sua" (both in more words and in more examples of each of those words), so it should remain the default. (It would be interesting to have some sort of way of flagging the exceptions: "an extra syllable was found in the following chants" or maybe even "the following chants have unmatched Volpiano on the word 'suavissimum'." But that's a problem for another time!)

I think this type of message is captured under #5 (so a very useful enhancement, but definitely an enhancement).
Thanks for confirming...closing this since we've determined we want to keep current behaviour with this construction.


Convert option examples from single to double dashes

This PR resolves #156 (single dashes in the cpu_check README) as well as the same problem in check_user_group.

Codecov Report

Merging #160 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #160   +/-   ##
=======================================
  Coverage   82.54%   82.54%
=======================================
  Files          19       19
  Lines         653      653
=======================================
  Hits          539      539
  Misses        111      111
  Partials        3        3

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d7400ec...1d545c3. Read the comment docs.


fix: sources API

This PR fixes the sources HTTP API reload capabilities. Functionally it is a step back as now all the pipelines will be restarted without fine grained checks.
Anyway we can recover such level of functionalities later on if required.

Hello @muka! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file src/ambianic/config_mgm/config_diff.py:

Line 333:80: E501 line too long (81 > 79 characters)
Line 383:80: E501 line too long (101 > 79 characters)
Line 404:80: E501 line too long (87 > 79 characters)

In the file src/ambianic/pipeline/interpreter.py:

Line 73:58: W291 trailing whitespace
Line 151:5: E303 too many blank lines (2)
Line 155:12: E111 indentation is not a multiple of four
Line 164:80: E501 line too long (84 > 79 characters)
Line 233:45: W291 trailing whitespace

In the file src/ambianic/server.py:

Line 189:80: E501 line too long (107 > 79 characters)

In the file src/ambianic/webapp/server/config_sources.py:

Line 52:1: W293 blank line contains whitespace

@muka Have you noticed that codecov is failing? Would you be able to add a few more tests to keep the coverage at a good level.

I noticed but considered just the overall, will check this

I think the PR is good for review. That codecov patch check is too picky for me!

Awesome. Merging. Thank you!

:tada: This PR is included in version 1.5.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Update for mypy 0.4

http://mypy-lang.blogspot.jp/2016/05/mypy-04-released.html

[ ] --fast-parser enables the new, faster parser (see above). This is still experimental but worth a try!
[ ] --disallow-untyped-defs generates errors for functions without type annotations. Consider using this if you tend to forget to annotate some functions.
[ ] --disallow-untyped-calls causes mypy to complain about calls to untyped functions. This is a boon for static typing purists, together with --disallow-untyped-defs :-)
[ ] --line-count-report DIRECTORY generates a report of annotated line counts.
[ ] --almost-silent behaves like --silent-imports, but it also reports every module that was silently ignored. This is useful for troubleshooting if mypy doesn’t seem to find all the code you are expecting.

It's to specify these options


GPU required?

The README implies that AlphaFold would at least prefer to use GPUs. However, it doesn't mention whether or not GPUs are required.

Thanks Alphafold team!
I have a similar question: does AlphaFold require GPUs?
Another question: if our HPC cluster does not run docker, any other way to run the software?

Based on my experience, it is working fine only with CPUs. Also, you may enforce the script to run without GPUs using
run_docker.py --use_gpu=False + additional arguments

Based on my experience, it is working fine only with CPUs. Also, you may enforce the script to run without GPUs using
run_docker.py --use_gpu=False + additional arguments

That's very good to know.
How long does it take to make a single prediction using CPU? And how much RAM does it use on average?

I have not checked about the RAM usage for the inference, but the input feature generation step requires significantly because of sequence searching against the BFD.
For the timing using CPU, it took more than predictions using GPUs. For a ~140 residue protein, it took ~25 minutes x 5 models for model buildings + ~25 minutes for the input feature generation.

For the timing using CPU, it took more than predictions using GPUs

Which CPU are you referring to? It would be of great help if I had a benchmark for my cpu

I have two Intel Xeon Silver 4214 @ 2.2GHz (24 threads for each) on a node. I have no idea how many threads were used for the inference.

I have two Intel Xeon Silver 4214 @ 2.2GHz (24 threads for each) on a node. I have no idea how many threads were used for the inference.

That's good enough, thank you so much for sharing this detail

I have not checked about the RAM usage for the inference, but the input feature generation step requires significantly because of sequence searching against the BFD.
For the timing using CPU, it took more than predictions using GPUs. For a ~140 residue protein, it took ~25 minutes x 5 models for model buildings + ~25 minutes for the input feature generation.

Thanks, this is super helpful.
I'd also love to know more about how much memory is required to run on GPUs and time required for GPU-based inference if anyone has specific numbers from their setups.

Thanks Alphafold team!
I have a similar question: does AlphaFold require GPUs?
Another question: if our HPC cluster does not run docker, any other way to run the software?

Hi,
We have created a How-To for non-docker AlphaFold setup and it can be found here https://github.com/kalininalab/alphafold_non_docker
Kind regards,
Sanjay

Thank you for sharing the information,  @huhlim @andreasntr ; @sanjaysrikakulam good to know there is a non-docker version!

Yes, you can run without a GPU (with the --use_gpu=False flag) but it'll be much slower. Thanks to other people for answering and providing timings.

Hi
We don't have docker and we don't use conda too (I built my own singularity recipe).  I want to run cpu version. The flag --use_gpu is marked in the script run_docker.py, and is missing in run_alphafold.py. I tried to use the option --flagfile=flags.txt but it says: Unknown command line flag 'use_gpu'. Is possible to indicate that I don't want to use a gpu to to run_alphafold.py script ?

I suggest that the information about running without GPU be added to README.md.
Should I make a PR or do you (the developers) prefer to add this info based on the comments in this issue?

I am tryin to install alphafold2. I want to know what GPGPUs are compatible with alphafold2?

Should I make a PR or do you (the developers) prefer to add this info based on the comments in this issue?

@jaacoppi, I am no developer, but it’d be great, if you made a merge/pull request.

Hi, I can understand from above conversation that I can run alphafold multimer without GPU. But I want to know is there any minimum configuration of the system or number of cpu, RAM requirement ?
If I want to run it on cloud which VM shall I select ? I am looking for the cheaper option.
Many thanks in advance.


feat: upgrade action/checkout to v4

Codecov Report
Patch coverage has no change and project coverage change: +0.10% :tada:

Comparison is base (7e16d9b) 79.12% compared to head (81c3ff4) 79.22%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2948      +/-   ##
==========================================
+ Coverage   79.12%   79.22%   +0.10%
==========================================
  Files         639      632       -7
  Lines       52615    52308     -307
==========================================
- Hits        41630    41443     -187
+ Misses       8917     8821      -96
+ Partials     2068     2044      -24

Flag
Coverage Δ

go-1.18-386
79.19% <ø> (+0.04%)
:arrow_up:

go-1.18-amd64
79.21% <ø> (+0.02%)
:arrow_up:

go-1.19-386
?

go-1.19-amd64
?

go-1.20-386
?

go-1.20-amd64
?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 8 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Can't find a Custom "menuLayoutClass"  demo

In PagingViewControllerLayout.swift file, you write this comment:
To create your own `PagingViewControllerLayout` you need to override the `menuLayoutClass` property on `PagingViewController`.
Then you can override the methods you normally would to update the  layout attributes for each cell.

I want to implement my own menuLayout.   For Example: vertical's flow layout or other layout I need.
Solution1:
Through your comment. I  must subclass  PagingViewControllerLayout directly.
I have try subclass it , override it's prepare() function, but I can't override PagingViewControllerLayout's  func "createLayoutAttributes", because it is private.
Solution2:
Or  directly subclass UICollectionViewLayout like PagingViewControllerLayout? but menuLayouClass's property is PagingViewControllerLayout type. When init PagingViewController
pagingViewController.menuLayoutClass = CustomLayout.self , CustomLayout must PagingViewControllerLayout's subclass?
I can't find a Demo in your Examples projects for this issue.
Search all issues about "custom menulayout".  but I can't find a good demo or UnitTest for this?
what should I do to implement a custom menuLayout.  Thanks.
Thank you again for your great project, Hope your reply.

Hi @michaeldong! Yeah, the documentation for custom layouts is very lacking. I'll try to improve that when I have some more time 🙂
It's true that need to subclass PagingViewControllerLayout and not UICollectionViewLayout. You basically need to override the same methods that you would if you where subclassing UICollectionViewFlowLayout, e.g. layoutAttributesForElements(in rect:) and layoutAttributesForItem(at indexPath:). There you can modify the layout attributes to return your custom behaviour. That means it's a bit limited what you can customize, as that you are kind of "stuck" with what is returned from createLayoutAttributes.
I think it would be very cool if you could create completely custom layouts, but it's a bit tricky to find a generic solution that works well with the distance calculations that are needed in order to update the content offset while scrolling.
Here is a simple example of a custom layout that rotates all the items in the menu:
class CustomPagingCollectionViewLayout: PagingCollectionViewLayout<ImageItem> {

  override func shouldInvalidateLayout(forBoundsChange newBounds: CGRect) -> Bool {
    return true
  }

  override func layoutAttributesForElements(in rect: CGRect) -> [UICollectionViewLayoutAttributes]? {
    guard let layoutAttributes = super.layoutAttributesForElements(in: rect) else { return nil }

    for attributes in layoutAttributes {
      if attributes.representedElementCategory == .cell {
        attributes.transform = CGAffineTransform(rotationAngle: 0.5)
      }
    }

    return layoutAttributes
  }

  override func layoutAttributesForItem(at indexPath: IndexPath) -> UICollectionViewLayoutAttributes? {
    guard let attributes = super.layoutAttributesForItem(at: indexPath) else { return nil }
    if attributes.representedElementCategory == .cell {
      attributes.transform = CGAffineTransform(rotationAngle: 0.5)
    }
    return attributes
  }

}

@Rechsteiner. Thank you for your detail reply and your demo code. I think your reply solve my question of subclass which class 「PagingViewControllerLayout」.
Next, I will base your 「rotationAngle」code. Try  to solve my problem :「 Left is custom Menu ，Right is pages, pages can up-down scroll.」
Your current demo: Top is menu layout, Bottom is pages, pages can scroll left-right.
I don't want to change menu styles because it's good enough for me. I only want to let menu from horizon to vertical ，pages scroll up-down.
Thank again for you again, Good luck!

wow, Parchment May support vertical or generic layout in future.  Thank you for you and your parchment again. Your sincere reply save me much time! Thanks.


STT0075: get the estimation of time and cost for training a whisper model.

Description
we are going to get an estimation of time and cost for training the whisper model. the whisper model is previously trained on 500 hours of training data but now we have 1500 hours training data so there will be significant improvement in this model compared to last time.
whisper paper
whisper base model code
whisper  paper vedio
Completion Criteria
A detailed estimation of time and cost and the GPU types used for training the stt-whisper model.
Implementaion

Read whisper research paper and understand the architecture of the whisper model.
clone the stt-whisper repo in GCP and set up the environment.
create a mount and transfer the audio segment data from wadwani cloud s3 wav16k storage.
run prepare dataset
train whisper
do hullacination prevention in whisper model.
get estimation of time and cost.

Subtask

[x] understanding the whisper model architecture model from the paper.
[x] set up the gcp environment with mount and stt-whisper repo
[x] transfer the audio segments from cloud to my mounted volume
[x] create train and validation excluding the benchmark
[x] run the prepare dataset
[x] train the whisper model to get time estimation
[ ] get time and cost estimation.

whisper small training on 1500 hours latest training data with new departments compared to 500 hours previously trained limited data variation in training. So we expect a better wer and cer score this time with more training data and variations in training data.
time and spec:
epoch: 10
training data hours: 1500
4 (L4) 24 Gpu for 10 days or
8 (L4) 24 Gpu for 10 days


Difference between downvote and delete

Hi!
Whats the diff. between upvoting and then downvoting,
and just upvoting and deleting the vote?
Makes no sense to me to have a downvoting feature when it's the same as delete (doesn't go into negative)

https://github.com/shanbay/django-vote/blob/master/vote/models.py#L70
vote score can be negative.

Hmm, not working for me.
When count is 0 (untouched object/instance), votes.down returns false for any of my users.

a.vote_score

Maybe reinstall thru pip?

The latest version is 2.1.4 and also check if you have define a custom  save method  on your a model.

I have 2.1.4, and nope, I only have 'save' overriding on my views/forms (CreateView etc.)
a is not a model, just an instance of my Question model.

vote_score doesn't work for me at all, check attachment.
I'll try this out on a new model.

Can you paste your Question model here?

Sure

I believe the relationship between num_vote_up and num_vote_down is messed up.
or something else changed my vote_score variable, and it became a constant.

a is now Newmodel.objects.get(id=1)

num_vote_down doesn't register votes.down at all

Okay I see what the problem is.
The vote_score that shows up in shell and is a model attribute, is different from the one that shows up in the admin panel (the right one)
I don't know why is this happening. Here's my admin panel listing for the same 'a' object as above.

+1
Have the same problem.

@mihovilm @amywoodehy In voting actions we generate a new instance by select_for_update. So the change does't reflect to the original instance, you have to fetch the instance again to see the change.
https://github.com/shanbay/django-vote/blob/master/vote/managers.py#L58

ok, I see change in vote_score, but count still returns the same result as before down vote.
Model.objects.first().votes.down(1)  # True
obj = Model.objects.first()
obj.votes.count()  # 0  -- this still bothers me
obj.vote_score  # -1
obj.num_vote_down  # 1

And Model.objects.first().votes.user_ids() returns empty queryset.

Oh, I understand now. But why was it necessary to make votes.count(action) return number of votes by action? I expected it to return all votes. Same for user_ids(action)

If you just want a score  you can use vote_score field, the count and user_ids  have to distinguish by action because of we support down vote now.

Can any of you give an explanation on how to get negative scores working? Here's what I have:
patch = Patch.objects.filter(id=post_id)[0]
patch.vote_score      #0

if patch.votes.exists(id):
    patch.votes.delete(id)

patch.votes.down(id)
patch.vote_score      #0

patch.save()
new_patch = Patch.objects.filter(id=post_id)[0]
new_patch.vote_score     #0

I know this is an old thread, but I feel like I've looked everywhere for an answer and haven't found one.

@Claytone which version are you using? do you have a custom save method on your Patch model?

I ran into the same (similar) issue, I'm using version 2.1.7 and I have no custom save method on my model. I've realized that it's due to that fact that you cannot save the object for whatever reason until you fetch the new instance:
obj.votes.up(1)
obj.votes.up(2)
obj.votes.down(3)

obj = Model.objects.first()
obj.save()

obj.vote_score #1
obj.votes.count() #2
obj.votes.count(1) #1

but if you flip the save and the fetch
obj.votes.up(1)
obj.votes.up(2)
obj.votes.down(3)

obj.save()
obj = Model.objects.first()

obj.vote_score #0
obj.votes.count() #2
obj.votes.count(1) #1

so although the changes don't reflect in the current instance as you mentioned, this means that you cannot save the instance at any point unless you fetch it again first. I'm not sure this is intentional design because if you're updating several fields along with a vote on a model you have to query for the model again before you save it, however if you follow the rule always fetch first if you have to save it should work as intended. Hope this helps

Hey, sorry for the late reply, but I've tried your solution and it doesn't work. Maybe you can shed some light on this?
    patch = Patch.objects.filter(id=post_id)[0]
    id = request.user.id
    print(patch.vote_score) # 0
    patch.votes.up(id)
    #tried with and without this line
    patch = Patch.objects.filter(id=post_id)[0]
    patch.save()
    print(patch.vote_score) # 0
    print(patch.votes.count(id)) # 0```

Resolved the issue, but I'm not sure what's different. Here's the final block of code, which behaves as expected:
patch = Patch.objects.filter(id=post_id)[0]
    id = request.user.id

    if direction=="up":
        patch.votes.up(id)
    elif direction=="down":
        patch.votes.down(id)
    elif direction == "clear":
        patch.votes.delete(id)
    else:
        print("ERR")

    patch = Patch.objects.filter(id=post_id)[0]
    patch.save()
    print(patch.vote_score)


Latest version of zcl-packet

@Koenkk
Could you please consider a possibility of adding a package dependency to the latest version of zcl-packet (your fork).
I'm interested in the "onWithTimedOff" command, but that bugfix released later.

Done, I moved zigbee-shepherd-converters to zigbee-herdsman (https://github.com/koenkk/zigbee-herdsman) which contains zcl-packet and zcl-id.


Completely insufficient background for Task 2

@MENG2010 @pooyanjamshidi
Once again, I feel completely unprepared to complete Task 2. I've looked at it for hours and I don't know how to get started. This appears to be the same situation we faced in Task 1, wherein we get thrown in the deep end of a swimming pool without being taught how to swim first. It is extremely stressful and frustrating, and honestly it makes me question if anyone is listening to our feedback in this class.
My team has chosen Option 1 because it appears to be the easiest option and the most similar to Task 1. The description of the assignment in the readme offers no helpful detail on how to actually complete the task. Instead, it points us to the Athena research paper section III.F, which also leaves me completely lost and in the dark about how to do this. That paper also cites two other research papers, claiming that it "combines" their methods. It gives a few equations which I do not understand and which have not been explained.
This is a 500-level class, not a 700/800-level one. Most of the students here are undergraduates or are graduate students coming from entirely separate fields. Unless you provide some significant help, most of us simply do not have the background to properly understand research papers whose intended audience was the adversarial ML research community. I'm asking you to please address these topics in class, or provide more detailed instructions about how to accomplish the task, or provide hints/tutorials, or do something to make this understandable.

Unfortunately, I have to agree. And I would say that this is not how 700/800-level courses are like either. I'm very excited about this course and still want to explore more about the topic once it is over, but I absolutely don't feel like I'm the target audience.
I also disagree with the comment in class today that we don't really need to understand the attacks to understand the defense. At least for some methods, understanding both is absolutely essential. For example, PGD/PGD-ADT which was required in the first task. It is like understanding statistical assumptions underlying hypothesis testing. And because this is cutting-edge stuff, the related papers are mostly technical and dense, as Andrew mentioned - including the Athena one.
This is our first contact with these methods. I'm taking this class to learn them. If the class was designed for people who just need to get up to speed with the latest developments, then that was not made clear. However, even if that is the case, consider the following guide by Tensorflow developers: https://www.tensorflow.org/tutorials/generative/adversarial_fgsm
According to themselves, this is targeted to experts. And yet, it is much easier to understand than what we were given to complete the project. That is because 1) the parameter of formulas are described in terms of the problem, and 2) by presenting the code piece by piece, we can write it as we build our own understanding, and then we can actually ask good questions. Had we slowly built Athena ourselves, I think we would not be so confused.

I agree with Raul completely.
In an effort to make this a productive thread and not just a pure rant, I'm going to compile a short list of questions here which, if answered, might begin to clear up my own confusion.  These questions are related to Option 1 for Task 2, since that is the one my team is working on.

I am not sure how much of what we need to do for Option 1 is already implemented in Athena and how much we need to do ourselves.  Ying posted on piazza a few days ago:

Each individual options in task 2 is an advantage extension to the ATHENA project, in which you are expected to do some research and implement the approach on top of ATHENA. This is much more difficult than task 1 and the workload is heavier.

This suggests that the solution of Task 2 requires adding onto Athena rather than just using the existing provided methods to complete the task.  However, the readme says for Option 1:

Possible solutions (already implemented in ATHENA):

Xuanqing Liu, Minhao Cheng, Huan Zhang, Cho-Jui Hsieh. Towards Robust Neural Networks via Random Self-ensemble. ECCV 2018.
Anish Athalye, Logan Engstrom, Andrew Ilyas, Kevin Kwok. Synthesizing Robust Adversarial Examples. ICML 2018

If these methods are already implemented in Athena, then we would not need to add onto Athena--we would just use the existing infrastructure.  That would make sense, since the Athena research paper certainly suggests that white box attacks are already supported.  So which is it?  Are we just supposed to use existing methods? Or do we need to create our own new methods?

If whitebox attacks are indeed already supported in Athena, where exactly is that code located?  It looks to me like the Athena framework only uses attacks provided by ART, and really the framework only provides methods to pass data to/from ART and to define default parameter values for those not specified in the attack configs file.
In particular, I ask because the Athena research paper discusses two ways to create the whitebox AEs: the greedy apporach and the optimization approach (which we are supposed to use).   There is an algorithm given in the paper for the greedy approach.  Where is this algorithm located in Athena?  Likewise, there are equations (6) and (7) associated with the optimization approach.  Where are these equations implemented in Athena?  Are these things actually handled behind the scenes in ART?  Or are we supposed to implement them ourselves? None of this is remotely clear, and I can't make any practical connections between the Athena paper and the provided code which help me understand what we're supposed to do for this assignment.

Thanks, @andrewwunderlich, and @raulcferraz for your suggestions, I agree with you! @andrewwunderlich your questions make sense. Let me try to provide an answer to both questions and @MENG2010 provides more details.
You do not need to add anything to ATHENA (you could if you want, but for T2-Option 1 you would not), rather implementing or reusing a white-box attack out there. We currently tried two types of white-box attacks as explained in the paper. The optimization-based is simply a dynamic attack targetted for ATHENA and these types of targetted attacks are common and are specific to the proposed defense to show how it performs against such strong targetted attacks that pretty much know everything about the defense and in those circumstances, the defense should break, but a good defense increase the cost of the attacker! Ying came up with such a targetted attack by combining EOT and RSE, basically, Eq (7) is the key idea behind this combined white-box attack. @MENG2010 will provide a tutorial showing you how you could implement Eq (7) and then you can use this to generate different types of attacks. I agree that you need to know how this equation implemented and how you can configure it to generate different variations of it.

tutorials have been recorded and uploaded.


[REQUEST] Movies support?

Please describe what you want to be added/changed:
Support for the "Filme" (Movies) tab on Aniworld. It's often used for Specials and Movies (although it's actually mostly specials it seems)

Only the naming of the files might be complicated here because I'm not sure if the order Aniworld has is really trustworthy (in the sense that they might add ones in-between and scramble the order). Maybe just use the titles here? The things on that tab are usually very chaotic anyway, so manual renaming and organization would be needed here anyways.
Possible Solutions if you have any:

Take in "Filme" or similiar for the season field
Use /filme in the URL instead of /staffel-x
Essentially do the same thing as with seasons, but with different file naming.

(If this is already possible, please tell me. I just searched for /filme in the code through GitHub and couldn't find anything)
Would contribute myself, but I have no clue about Python.

@llunawolf you can now use this to download Movies: python .\main.py anime toradora Ger-Sub True
The Bool at the end is the Movie Bool. If its true it will download filme/film-X and if its False it will download staffel-X/episode-X
Please test it and tell me if it works.

Tested and works well. I just started to implement the function as well last evening. Just discovered before your new commit :D
PS: You should check out argparse - that could replace several pieces of your code and also the shell/batch part as it also include checks, defaults, enums and many more.
https://docs.python.org/3/library/argparse.html I use it in nearly all projects at work. It also generated a man page with --help

Hi @alternone .
I will look into it some day. But currently im using 90% of my spar time to work on the DeathgarenRebirth Project.
If you want you can open a PR or i will implement it once i have some time.

So um, the moving downloading works great, but uh... how do I download non-movies now?
python3 "main.py" "anime" "pokmon" "Deutsch" "False" "6"
python3 "main.py" "anime" "pokmon" "Deutsch" "false" "6"
python3 "main.py" "anime" "pokmon" "Deutsch" "True" "6"
python3 "main.py" "anime" "pokmon" "Deutsch" "true" "6"

All of these download the movie.

Temp fix dont use a season override. forgot the logic there...

Actualy not a me bug!
py just doesnt understand Bools as arguments...
(py310_AniWorldScraper) PS D:\python\projects\AniWorldScraper> python .\main.py anime pokmon Ger-Sub 6
DEBUG:src.constants:cli argument detected on position:1 with value:anime
DEBUG:src.constants:cli argument detected on position:2 with value:pokmon
DEBUG:src.constants:cli argument detected on position:3 with value:Ger-Sub
DEBUG:src.constants:cli argument detected on position:4 with value:6

Actualy not a me bug! py just doesnt understand Bools as arguments... (py310_AniWorldScraper) PS D:\python\projects\AniWorldScraper> python .\main.py anime pokmon Ger-Sub 6 DEBUG:src.constants:cli argument detected on position:1 with value:anime DEBUG:src.constants:cli argument detected on position:2 with value:pokmon DEBUG:src.constants:cli argument detected on position:3 with value:Ger-Sub DEBUG:src.constants:cli argument detected on position:4 with value:6

this should help:
movie = (parse_cli_arguments(False, 4).lower() == 'true')  # True = movie. False = episode

@llunawolf Can you please try if the issue is fixed for you as well?
Just implemented @Bramdarl fix which worked on my Machine and Test machine.

I tried the new version with a movie, but got those exceptions in return

I tried the new version with a movie, but got those exceptions in return

@Monokuma11 you are using the Windows Store version of python. This version is from what I know very buggy and you might need to run the script as admin etc because the permissions are not set correctly sometimes.

The error stays :(

@Monokuma11 please try the latest version which was just merged. (Git clone...) If that still does not work create a new issue please with the Anime Name, Your settings and also the log output.

Closed. No Response from original creator.


CWF IP model in OKAN should use MAD

This was just an oversight, pretty sure we have the MAD data for all Columbia (where there is no contrib area outside of BC).

parameter fixed, leaving open as a reminder to check output next time model is run.


Sorry again for bothering

Is there any way to increase the download speed
Like using VPN or something....
The speed of my net is around 8 to 10 mps speed
But the downloading is very slow...

Hi @Sunderlevi, I'd suggest to try the flag -t to enable parallel download, it may gain some speed. Overall, the download speed is throttled on the server side, which I don't have the control of :(