Add support for reading/writing with multiple encodings

With Webnative 0.35 you can only read and write Uint8Arrays.
There should be support for other encoding, such as UTF8 strings.
@matheus23 suggested the encoding option from https://nodejs.org/api/fs.html#filehandlereadfileoptions
We could do something like:
fs.write({ bytes: Uint8Array })
fs.write({ text: "Text", encoding: "utf8" })

This may be a duplicate of #269 now.
I'm biased, but I'd vouch for the 2 positional parameters and an optional third "options" parameter that contains the { encoding: "utf8" } object (as I've suggested in that issue).
For read this looks like: fs.read(path, { encoding: "utf8" }).
This means it's less of a breaking change & it's potentially more familiar to JS developers.
For write, I feel like we have the option of "auto-detecting" the encoding. I.e. if a string is passed, we assume UTF8.
However, that may not be ideal: It may be surprising that .write accepts a string but subsequently returns a Uint8Array if the same file is .read.
So perhaps, it'd make sense to only allow passing a string if the encoding parameter is explicitly set to UTF8.
So, personally I'd be in favor of these types:
fs.read(path: FilePath): Promise<Uint8Array>
fs.read(path: FilePath, options: { encoding: "utf8" }): Promise<string>

fs.write(path: FilePath, bytes: Uint8Array)
fs.write(path: FilePath, text: string, options: { encoding: "utf8" })

Oh whoops, forgot about that issue.
fs.write(path: FilePath, bytes: Uint8Array)
fs.write(path: FilePath, text: string, options: { encoding: "utf8" })

That works too yeah, I have no strong preference.

So perhaps, it'd make sense to only allow passing a string if the encoding parameter is explicitly set to UTF8.

Yes exactly, that's what I was trying to achieve as well. I assumed it'd be easier to do in Typescript with my syntax, rather than yours, but I could be wrong. But yeah, your syntax is more familiar.

Yes exactly, that's what I was trying to achieve as well. I assumed it'd be easier to do in Typescript with my syntax, rather than yours, but I could be wrong. But yeah, your syntax is more familiar.

You are correct that it's easier, but still possible. Here's how node.js defines the readFile type:
https://github.com/DefinitelyTyped/DefinitelyTyped/blob/4805c8405fb64154def301a763c40328ccfb612c/types/node/fs.d.ts#L2532-L2554
Essentially it'd boil down to function type overloading. Something like this in our case:
export function read(path: FilePath): Promise<Uint8Array>;
export function read(path: FilePath, options: { encoding: "utf8" }): Promise<string>;
export function read(path: FilePath, options?: { encoding?: string }): Promise<Uint8Array | string> {
  // impl
}

Oh you can do function overloading in Typescript?! How did I not know this 😲
Ok yeah, that's better 👍


Staging / Prod database

Once nessecary separate staging / prod env for tests.

Done! Duplicated db into new schema that's now being used in local dev & Vercel preview & development branches.


Support for Panel Focus Selection Directional Modifier.

This task is automatically imported from the old Task Issue Board and it was originally created by Mateusz Czapliński.
Original issue is here.

Why
As Enso Developer
I want the Panel Focus Selection Directional Modifier key handled in Component Browser
So that I can implement rendering a blue border around the Panel with Active Focus.
Acceptance Criteria
Scenario:
GIVEN an open Component Browser
WHEN I hold the Panel Focus Selection Directional Modifier and press one of the arrow keys on the keyboard
THEN a debug message is printed in Developer Console informing which Focus Area is selected as Active Focus Area
WHEN I hold the Panel Focus Selection Directional Modifier and press Backtick key or 0 key on the keyboard
THEN a debug message is printed in Developer Console informing that the Expression Input Panel Area is selected as Active Focus Area.

Notes

Focus Areas that need to be supported:

the Component List area of the Component Browser List Panel,
the Section Navigator area of the Component Browser List Panel,
the Expression Input Panel,
the Documentation Panel,
the Visualization Panel.

The Panel Focus Selection Directional Modifier is defined differently on Mac vs. on Windows and Linux. This difference can be encapsulated in a new macro to be used when defining key combinations in default_shortcuts functions.

Comments:
Paweł Grabarz reports a new STANDUP for today (2022-11-07):
Progress: Discussed the arch approach for central focus system, task got estimated. Now implementing focus manager for component browser specifically. It should be finished by 2022-11-14. (Enso Bot - Nov 7, 2022)

**Paweł Grabarz** reports a new **STANDUP** for the provided date (2022-11-02):
Progress: Meeting with Adam on planned focus changes. Investigating application loading times. It should be finished by 2022-11-14. (Enso Bot - Dec 12, 2022)

**Paweł Grabarz** reports a new **STANDUP** for the provided date (2022-11-03):
Progress: Started implementing application-wide focus manager, for now scoped to component browser. It should be finished by 2022-11-14. (Enso Bot - Dec 12, 2022)

**Paweł Grabarz** reports a new **STANDUP** for the provided date (2022-11-04):
Progress: Another meeting with Adam on focus planning. Scrapping focus manager implementation and switching to compile time optimizations task. It should be finished by 2022-11-14. (Enso Bot - Dec 12, 2022)

CB focus design changed.


[WFRP4e System] Upgrade has badly damaged multiple games

Since the most recent update both Warhammer worlds I run are not running properly.  Some characters cannot be accessed at all, others are missing some of the skills (All the skills above Climb on the Basic skills are no longer accessible).  Some combat options no longer seem to work, opposed roles don't seem to work properly for some characters and not others.
I don't know whether this is due to the overall Foundry/Forge upgrade or the upgrade to the Warhammer system.From: Peter Cat #0849wfrp4e: 4.0.6wfrp4e-core: 2.0.4wfrp4e-starter-set: 2.0.2wfrp4e-ua1: 2.0.1

Probably migration issue


Implementation for Undo & Redo Features

Add new feature (undo & redo)
Edited WBSession.ts & WBApexTool.ts

I tested this new feature on your branch and it worked perfectly.
You really did a great job. 👍


Add ntls client and server example; Add API conn.GetVersion()

Only one main func can be defined in the same package . Please move ntls_client and ntls_server into sepereted package (e.p. examples/client/ntls_client.go examples/server/ntls_server.go )

Fixed.


SVG loading issue

When I build boilerplate in dev mode, then I don't see any issue regarding svg files. But when I build in production mode, following error is coming. in webpack config, I see that file-loader is used for SVG files.
ERROR in ./app/components/Header/product.svg
Module parse failed: D:\webui\react-boilerplate\app\components\Header\product
.svg Unexpected token (1:0)
You may need an appropriate loader to handle this file type.
|
|
|     product
@ ./app/components/Header/index.js 14:52-79
Please let me know how I can fix this

@debslab how is the SVG file being imported?

@uzarubin Yes, I am doing like this. For me, it's working in dev build. If I run in prod mode, webpack complains about loader. After seeing your comment, I thought I will try with fresh boilerplate setup and guess what, it started working. Probably I messed up some settings. I will close the bug. Regret the inconvenience.


[ENHANCEMENT] Improve the resourcequota autoscaling

Is your enhancement related to a problem? Please describe.

Backgroud: when VM is under migration, Harvester will scale up & down the related resourcequota to avoid migration is blocked due to quota limitation.
https://github.com/harvester/harvester/blob/992cd4c9ea20b95ed51d6d3a7654cec47255dc83/pkg/controller/master/migration/vmim_controller.go#L54
Following log is observed from one environment:
./harvester-system.virt-controller-6bc767746-qr5rm.virt-controller.20241210.log.gz:{"component":"virt-controller","level":"info","msg":"reenqueuing Migration ns1/kubevirt-evacuation-dfp6l","pos":"migration.go:233","reason":"failed to create vmi migration target pod: pods \"virt-launcher-ns1-4ae21d5d-n2mtw-qgtdp\" is forbidden:

 exceeded quota: default-4tmmk,

requested: limits.cpu=4,015m,  limits.memory= 8,940,039,936,
used:      limits.cpu=28,505m, limits.memory=62,900,279,552,
limited:   limits.cpu=26,      limits.memory=54,242,646Ki",

"timestamp":"2024-12-10T14:36:47.533782Z"}

Per testing, the scaledown happend too early which may affect following migration.
The detailed analysis
https://github.com/w13915984028/harvester-develop-summary/blob/main/v121-vm-migration-with-resources-quota.md
Describe the solution you'd like

Make sure the migration is successful.
Describe alternatives you've considered

Additional context

The ticket has two points to analyze and enhance:
(1) Migration triggers failed to create vmi migration target pod...exceeded quota... error, the reason is known and will be enhanced.
(2) The error message said:
 exceeded quota: default-4tmmk,
...
limited:   limits.cpu=26,      limits.memory=54,242,646Ki",

But the resourcequota is defined as;
  spec:
    hard:
      configmaps: "15"
      limits.cpu: "30"
      limits.memory: 60Gi
      persistentvolumeclaims: "75"
      pods: "50"
      replicationcontrollers: "50"
      requests.cpu: "30"
      requests.memory: 60Gi
      requests.storage: 320Gi

Why it prints  limits.cpu=26,      limits.memory=54,242,646Ki instead the RQ defined value? is still under investigation.
It is not observed on our reproduction: https://github.com/w13915984028/harvester-develop-summary/blob/main/v121-vm-migration-with-resources-quota.md

The scenod phenomenon is explained here:
https://github.com/w13915984028/harvester-develop-summary/blob/main/v121-vm-migration-with-resources-quota.md#another-issue-resourcequota-was-changed-unexpected

The VM-migration was affected by the relacement of POD cattle-cluster-agent , the latter reverted the RQ scaling.

When drain the node, the cattle-cluster-agent POD may also be replaced to another node, and it runs in parallel with VM migration, there is a chance to hit the bug.  sure, it does not always happen.


Add support for Search result color config (#458)

The command like below changes bgcolor of search results
:highlight Search guibg=#FF0000
:highlight Search guibg=blue

Please make PR to develop branch. Thanks.


Remove training images from ailab-images.md

We are not currently making these images available.
Fixes: https://github.com/containers/ai-lab-recipes/issues/643

@cdrage PTAL

@cdrage PTAL

Looks good! If it's not accessible, any idea how to build?


Validate settings read from disk

Add function to check that preference settings are present and valid, use defaults if not.
Resolves #17

Agreed, this is clunky and repetitive. I'm not sure I can use exactly the approach you suggest since I think it may throw an exception before I am ready to catch it. But I have enough of a pointer to go on (pun intended!).

I think by moving it into a function, you can do the catch there and do the correction there (which is why I suggested added the default value to the call).


Need to integrate better in the testgrid

We are on the grid: https://k8s-testgrid.appspot.com/ but clicking our button reveals no results. Other buttons such as "google-aws" have a spread sheet of red and green like this: https://k8s-testgrid.appspot.com/google-aws
Our gubernator results https://k8s-gubernator.appspot.com/builds/canonical-kubernetes-tests/logs/kubernetes-aws-e2e-node also do not show up on the Gubernator home page https://k8s-gubernator.appspot.com/

change the dir format to mmddHHMMSS000 instead of mmdd_HH:MM:SS.000
Looks like right now we filter down to subdirs of gs://canonical-kubernetes-tests/logs/kubernetes-gce-e2e-node that match \d+ (so that we can cast them to int, so that we can sort them correctly). Will look into easing this restriction but it is probably more involved than your changing the dir format if that's an option.

@chuckbutler Do we need the 000 at the end? It looks like we always add that to the timestamp. I could remove that if not needed.

I dont think we can remove it, as that was called out by fejta in their instructions on how ot fix our integration


Programmer should have access to spell damage and range without having to place them in the palette

Working on refactoring the spells for this.  I'll keep a list as I update things.
Finished:
shockWave
confuse
drainLife
earthskin
fear
fling
force-bolt
grow
haste
heal
magicMissile
poisonCloud
raiseDead
regen
root
shrink

These spells are not implemented yet and will be refactored when implemented:
disintegrate
dispel
fireball
flame Armor
goldstorm
invisibility
lightning bolt
summon Burl
summon Undead
swap
teleport

@nemoyatpeace, you can add "- [ ]" before each spell so that they appear with a checkbox.

Now working on the thangs with spells on them:
All books are complete, need to mess with these wands:
Enchanted Stick
Simple Wand
Twisted Pine Wand
Oak Wand
Mahogany Staff
Balsa Staff
Banded Redwood Wand
Oak Sphere Staff
Steel Wand
Lightning Twig
Obsidian Staff
Crystal Wand
Sulphur Staff
Golden Wand
Vine Staff

Ah yeah, those wands/staffs don't exist yet. Now that the Thangs are done, the next step is to go back to all the Components:

Add the new names to the top-level default object with the default values.
Delete the old names from the config schema default and properties.
Delete the old name handling from the spell initialization in the code.
Submit a patch.

Then when all those are ready, I'll merge them all and I'll do the last step, which is to delete the now-unrecognized properties from all the item Thangs.

OK, so refactoring components again here is my todo list:
x confuse
x drainLife
x earthskin
x fear
x fling
x force-bolt
x grow
x haste
x heal
x magicMissile
x poisonCloud
x raiseDead
x regen
x root
x shockWave
x shrink
x timewarp
x windstorm
Complete, ready for you now nick

Phew, done! (Time will tell if we missed any.) Nice work as always, @nemoyatpeace!


Feature: 4K Video Support

Looks like we'll finally get high resolution videos as part of the Apple TV 4K.
https://9to5mac.com/2017/09/11/tvos-11-4k-apple-tv-screensaver/
Can't wait to get them on my iMac

I wonder if anyone has found/published the new URL(s) to these yet?

Great question. I'm looking for the links for the 4K videos, too.

The race is on!

I'm looking as well. Seems like they've obfuscated the URLs in tvOS 11

That just means it's over SSL. Just need to do a MITM attack on a demo ATV and should be clear as day

@AddoSolutions  Haha, "Just". Apple tends to use SSL pinning. Hopefully they didn't in this case, but I wouldn't hold my breath on that route.

Yeah, I was thinking about that in the shower. On the normal device, you
can add a SSL authority, and just do it that way. However, in this case, it
may be more complicated because I don't think you can do that on Apple TV
On Wed, Sep 13, 2017 at 10:25 AM John Coates<EMAIL_ADDRESS>wrote:

@AddoSolutions https://github.com/addosolutions Haha, "Just". Apple
tends to use SSL pinning. Hopefully they didn't in this case, but I
wouldn't hold my breath on that route.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/JohnCoates/Aerial/issues/378#issuecomment-329184539,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADrjCd2t6xlnLFQGsILbkJt_Nil9Uh1_ks5sh-XfgaJpZM4PTUFg
.

--
Nick Artman
+1<PHONE_NUMBER>

Even on regular iOS devices, Apple uses SSL pinning so that even root certs won't fool their critical services. Try opening the App Store on macOS or iOS while running your MITM proxy. You need to disable pinning it with https://github.com/nabla-c0d3/ssl-kill-switch2 for example

Interesting that http://a1.phobos.apple.com/us/r2000/000/Features/atv/AutumnResources/videos/entries.json
Gives an invalid token error vs
http://a1.phobos.apple.com/us/r3000/000/Features/atv/AutumnResources/videos/entries.json
Gives a file not found

Can't you find the screensavers in the IPSW file? I can't check as I'm not a developer. That's what 9to5Mac said though (at least for the screensaver they put on YouTube)

The IPSW has a file called Default.mov in the screensaver app that I was able to find. This is probably what 9to5Mac found as well, but it does not include all of the videos, just that one demo. The rest of the videos probably still get streamed from web locations like with tvOS 10.
I wasn’t able to find any URLs in the IPSW that look like they point to a manifest file, but this is my first time looking through an IPSW and I’m not sure how to extract absolutely everything.
Maybe someone else can find it?

Good news! I got the URLs. It wasn't in the IPSW, and the entries were transmitted using SSL pinning.
These 4k videos are really beautiful. Now I just need to figure out how I'm going to play them.

Any chance you can share the URL(s)? :)

I'd be curious, too!

Yeah! Thats a total tease :)
On Thu, Sep 14, 2017 at 9:48 AM Finn Gaida<EMAIL_ADDRESS>wrote:

I'd be curious, too!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/JohnCoates/Aerial/issues/378#issuecomment-329487146,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADrjCWrrdEABdyx6FI5GvPAzOwy-RwHHks5siS6WgaJpZM4PTUFg
.

--
Nick Artman
+1<PHONE_NUMBER>

Me three! :)
Wondering how you found them as well!
Michael Sheaver
703-200-3008
<EMAIL_ADDRESS>www.michaelsheaver.com

On Sep 14, 2017, at 10:14 AM, Addo Solutions<EMAIL_ADDRESS>wrote:
Yeah! Thats a total tease :)
On Thu, Sep 14, 2017 at 9:48 AM Finn Gaida<EMAIL_ADDRESS>wrote:

I'd be curious, too!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/JohnCoates/Aerial/issues/378#issuecomment-329487146,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADrjCWrrdEABdyx6FI5GvPAzOwy-RwHHks5siS6WgaJpZM4PTUFg
.

--
Nick Artman
+1<PHONE_NUMBER>
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/JohnCoates/Aerial/issues/378#issuecomment-329495514, or mute the thread https://github.com/notifications/unsubscribe-auth/ABXSovyDlb8-3KJRR7Ji86tKnNiW8ruoks5siTTWgaJpZM4PTUFg.

Sounds like he used this https://github.com/nabla-c0d3/ssl-kill-switch2

C'mon, you are killing me!

Please add the new 4K screensavers. They're amazing.

I'll post the URL here as soon as I release support for 4K. They're using HVEC which doesn't come with system support until High Sierra, so it's taking me a little longer.
As for how I found it: It was very hard, took me a long time to figure out, and requires a long explanation. To put it as simple as I can, I had to put my Apple TV running tvOS 11 through a proxy, but since I couldn't use SSL Kill Switch a lot of the connections were hidden. I had to correlate the request for the videos to the request for the sources URL, then since I couldn't see into that request I had to replicate it on the tvOS simulator, where I used a version of SSL Kill Switch that I modified to support this operation. The right modifications were found by using IDA Pro with Hex-Rays to reverse the simulator binaries resposible for checking the trust chain. Then I forged a request as if it came from the new 4K Apple TV.
If you have any questions or want more details feel free to DM me https://twitter.com/johncoatesdev

https://sylvan.apple.com/Aerials/2x/entries.json

https://sylvan.apple.com/Aerials/2x/entries.json

any idea how to play them?

@SuperDOS If you mean as part of the screensaver then all you need to do is be a little patient and let @JohnCoates work his magic. :)
If you mean outwith the screensaver then you can download them and they play in VLC fine.

@alistairmcmillan tried VLC but won't show anything, Potplayer just displays artifacts.

@SuperDOS even Safari/QuickTime player fail to open them on High Sierra. VLC displays just a few colored blocks. Haven't tried https://nightlies.videolan.org/build/macosx-intel/ yet

@SuperDOS @spitfire Try the SDR versions they work fine in VLC right now. For everything else I'm afraid you'll need a little patience. :)

There's no way to play the HDR files currently. I opened up an issue with VLC, https://trac.videolan.org/vlc/ticket/18813
I'll be adding support for the SDR files by using VLC

Yeah, I am able to open the SDR files just fine in VLC.  Will be interesting to see how that affects performance (if my CPU is gonna be at 100% from this on my 2x 4k monitors :P I have 2x 1070s but I kind of doubt VLC is GPU accelerated.

Great point on CPU usage. I'm contemplating supporting conversion to h264, but it takes a really long time to convert each video

So, weird thought, but is anyone using High Sierra? You'd think that apple would add support for the HDR videos (considering iPhones are HDR, and shoot HDR video, etc.) I would have to imagine that QuickTime has been updated to support it??

You'd think so, but I'm on the High Sierra Public GM and the HDR versions won't work with QuickTime or VLC. The SDR 4k ones are fine.

On my 4 core / 8 thread i7, VLC stutters and is almost unusable with the 4k SDR, QuickTime is around 20% CPU and is fine.

Thanks for the heads up @microbug. I'll definitely add in conversion as an option, and default to Quicktime if on High Sierra.

Just release a 2.0 that only supports high sierra and up, the others can use the old one 👍  IMHO.

Uploaded the SDR versions to Youtube (doesn't support HDR) and let them do the conversion :) . downloaded the converted MP4 version and it runs much smoother in VLC without HW decoding.

I believe HDR video makes sense only if the display supports HDR. Software cannot compensate for the lack of HDR support in the display right? Also will 4K be useful if screensaver is being rendered on a 1080p display?
Either ways, hope there will be an option to pick one of the 4 options Apple appears to provide(1080p SDR/1080p HDR/4K SDR/4K HDR). Do let me know if this needs to be put as a separate Enhancement request.

Indeed, most displays (including the built-in displays on all current iDevice and Mac products IIRC) don't support HDR so there's no point in using the HDR videos on them. The 4k screensaver might be crisper on a 1080p display than the original 1080p one, it depends how heavily the original was compressed. Someone could do a comparison between the two? Regardless, you'd probably want 1080p on a 1080p display when away from power as downscaling 4k will drain battery faster than playing 1080p.

@SuperDOS I wouldn't be that keen on using videos that have gone through YouTube. Their heavy compression turns nice videos to garbage. I'd be happy to do some higher-quality transcodes to x264 if needed, I have a spare machine to do it on.

The recent iMac (last 2015) screens have wide-colour-gamut displays (DCI P3). I think this is part of what is improved in HDR.
Presumably an HDR video would look better on a wide-colour-gamut display than an SDR video would?

I don't think transcoding and re-hosting videos breaks the law, there must be a way to use the existing files.

@microbug I though so as well but was quite suprised how the converted 4k versions turned out from youtube. I use this videos as screen saver for kodi on my 4k philips and looks great.
===================== Video =====================
Id                          : 1
Format                      : AVC
Format info                 : Advanced Video Codec
Formatprofile<EMAIL_ADDRESS>Formatsettings, CABAC       : No
Formatsettings, ReFrames    : 2 frames
Codec Id                    : avc1
Codec info                  : Advanced Video Coding
Duration (ms)               : 8mn 0s
Bit rate                    : 10.9 Mbps
Width                       : 3 840 pixels
Height                      : 2 160 pixels
Display aspect ratio        : 16:9
Frame rate mode             : Constant
Frame rate                  : 29.970 (30000/1001) fps
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Bits/(Pixel*Frame)          : 0.044
Stream size                 : 624 MB (100%)
Encoded date                : UTC 2017-09-19 06:35:07
Tagged date                 : UTC 2017-09-19 06:35:07
Color range                 : Limited
Color primaries             : BT.709
Transfer characteristics    : BT.709
Matrix coefficients         : BT.709

As @spitfire said, re-hosting the videos is likely going to violate the law in many jurisdictions; I know it will in the United States. All transcoding, if that is the route that is chosen, will have to be done on local hardware.
Honestly, I think you should support 4K/HDR on High Sierra only, using Apple's built-in playback support. Hardware decoding is an absolute requirement when you start getting to 4K and higher resolutions. All of this talk about transcoding videos is making this way more complicated than it needs to be.

I've been trying to look into @peterrussell's theory that HDR encoded video will look better on the WCG (P3) displays in the iMac and MacBook Pro.
This overview of HDR states:

Furthermore, in HDR TV standards, a given television has to also manage what’s called P3 color, or at least 90% of it. P3 refers to the P3 part of the total color spectrum, which is quite a bit larger than the older Rec.709 color spectrum used in previous 4K TV models.
All of the above amounts to what is also called the “Wide Color Gamut” ideal among 4K TV makers, which means wider coverage of color spectrum and much smoother gradation between all possible shades in that covered spectrum space. It’s an extended part of developing highly realistic display quality and high quality HDR.

Also, this CNET article says:

As I mentioned before, HDR and WCG often appear together, so most HDR material is also in WCG. Almost all 4K Blu-ray discs have wide color gamut. The same goes for HDR streams from places like Nextflix, Amazon and Vudu. To get an idea where you can get WCG content, check out Where can I get HDR TV shows and movies?

So... I think that means yes? I'm still not sure.
Follow up question: does anyone know if you hook up a LG 32UD99-W, will it fully display HDR encoded video?
I recently purchased a TCL 55P607 specifically because The Wirecutter said it was the best HDR display you can get for under a grand. Will my 2016 MacBook Pro be capable of sending it the same HDR content the new Apple TV can?

@huebs IIRC, true HDR requires 10+ bits per channel. Anything less and you just spread the finite colors across the wider gamut. If your display supports both 30-bit color and a wide gamut, then it should benefit at least somewhat from HDR videos.

For those who are wondering, we are able to play "url-4K-HDR" video files found here with QuickTime in High Sierra by changing the fourCC (video codec identifier) dvh1 to hvc1.
vi -b -c "%s/dvh1/hvc1/g" -c "wq" /path/to/video/file

Quality and comparison screenshots

For those who don't want to download pretty ugly (and quite big) HDR videos.

So here's a 2 comparison full screen screenshots between the (almost) same frame on both HDR and SDR video. Both files read by QuickTime on macOS High Sierra, HDR file converted using @maiwennleclercq's technique.

These are JPGs at maximum quality just to reduce file size to upload on GitHub, they were not modified and come straight from the screen capture PNG file

The screenshots actually look like the video in QuickTime

SDR

HDR

Comprehensive review
Viewed on the latest 2017 15" MacBook Pro, it seems like the HDR video is looking like an ungraded video. It mainly lacks contrast & saturation, in a very noticeable way. Whereas the SDR file just looks gorgeous and richer.
This might be due to the fact that QuickTime does not read the HDR file properly. The fact you have to change from dhv1 to hvc1 might be the reason the file doesn't get read properly, especially regarding contrast data, I'm not sure. It might as well be the fact the display is not HDR, not sure either.
TL;DR
HDR file won't get you any better image than SDR. On the contrary, as long as we do not have a proper way of playing HDR files, they will look ugly on your Mac.
Download the SDR.

@habovh Thank you for pointing that out.

@habovh @maiwennleclercq probably some decoder issue, hope that changes in the future, but for now SDR looks better..

@maiwennleclercq thanks for the technique to play HDR
@habovh I appreciate the screenshot comparison
Just wanted to give everyone a heads up. I'm currently working on the update. Sorry it's taking longer than it should, work has kept me very busy. I'm updating from Swift 3 -> Swift 4 and cleaning up the codebase. A great designer has reached out and is working with me to give Aerial a bit of a facelift also. More details soon.

That’s sounds great @johncoates. Patiently waiting for the update!

Thanks for the findings. I've downloaded one of these 4K HDR from the entries.json file to feed it to my LG OLED C7, which can play some of the 4K HDR I've thrown at it, but I believe they need to be in a mp4/mkv container (and HEVC 4096x2160@60P <EMAIL_ADDRESS>Main10<EMAIL_ADDRESS>50Mbps). If there is a video format magician here who could give me a hint as to how best convert from one container to another without losing too much quality, I'd be grateful (neither VLC or Handbrake did help). Thanks.

My AppleTV 4K arrived today. First thing I did was fire up Aerial and... it's gorgeous.
As I said in my last post, I've got a TCL 55P607. It supports HDR and Dolby Vision. When I plugged in the Apple TV, it defaulted to 4K HDR. It then offered to try switching to Dolby Vision and then asked me to confirm it was working.
In the Video Fomats menu, it lists:

4K Dolby Vision 60Hz
4K HDR 60Hz
4K SDR 60Hz
1080p Dolby Vision 60Hz
1080p HDR 60Hz
1080p SDR 60Hz

Then there's an option for "Other Formats", where I find:

4K Dolby Vision 50Hz
4K HDR 50Hz
4K SDR 50Hz
4K Dolby Vision 30Hz
4K HDR 30Hz
4K SDR 30Hz
4K Dolby Vision 25Hz
4K HDR 25Hz
4K SDR 25Hz
4K Dolby Vision 24Hz
4K HDR 24Hz
4K SDR 24Hz
1080p Dolby Vision 50Hz
1080p HDR 50Hz
1080p SDR 50Hz
...and a handful of sub-1080p SDR 60Hz

If I plug in my 2016 15" MacBook Pro with the Radeon Pro 460 4GB graphics card, I'm presented with this:

Unless I'm missing something, it looks like there's there currently no way to drive an HDR display using a 2016 MacBook Pro.
Note that I tried using both a USB-C to HDMI adapter hooked up with the same HDMI cable I used for the Apple TV and a USB-C to HDMI cable.
The cable explicitly says it's only capable of 4K @ 30Hz. The adapter says it supports 4K @ 60Hz and has been, "updated to support 2017 MacBook Pro & iMac with Thunderbolt 3."  Has anyone with either of those newer models tried hooking it up to a 4K HDR display?
My next step is to try this 4K @ 60Hz cable and see if that gets the OS to present different resolution options.

A helpful hint for anyone trying the above: to get my MBP to display an image on the 4K TV, I had to manually disable automatic graphics switching before plugging in either the cable or adapter.

@sebastienbarre The LG C7 will actually play the .mov files natively, once you patch the dvh1 codec identifier back to hevc.
That said, using the latest ffmpeg snapshot build you can remux the Quicktime MPEG-4 into an ISO standard .mp4 and patch the codec identifier in one go with ffmpeg -i input.mov -f hevc -vcodec copy output.mp4

@benwiggins thanks for the tip. Didn't quite work, unfortunately, either with the latest stable ffmpeg 3.3.4, or with the ffmpeg-87630-ge9f9175 snapshot (the included x265 library is a multi-bit version (8-bit, 10-bit, 12-bit)).
ffmpeg version N-87630-ge9f9175-tessus Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-8<IP_ADDRESS>)
  configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libass --enable-libbluray --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzmq --enable-libzvbi --enable-version3 --disable-ffplay
  libavutil      55. 77.101 / 55. 77.101
  libavcodec     57.106.104 / 57.106.104
  libavformat    57. 82.102 / 57. 82.102
  libavdevice    57.  9.101 / 57.  9.101
  libavfilter     6.106.100 /  6.106.100
  libswscale      4.  7.103 /  4.  7.103
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
[dvvideo @ 0x7fca42010c00] could not find dv frame profile
[...]
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Dubai 1.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 0
    compatible_brands: qt
    creation_time   : 2017-07-14T23:32:18.000000Z
  Duration: 00:04:00.24, start: 0.000000, bitrate: 8261 kb/s
    Stream #0:0(und): Video: dvvideo (dvh1 / 0x31687664), yuv420p, 720x576, 8258 kb/s, SAR 1:1 DAR 5:4, 29.97 fps, 29.97 tbr, 30k tbn, 25 tbc (default)
    Metadata:
      creation_time   : 2017-07-15T04:28:28.000000Z
      handler_name    : Core Media Data Handler
      encoder         : H.265
Output #0, hevc, to 'Dubai 1.mp4':
  Metadata:
    major_brand     : qt
    minor_version   : 0
    compatible_brands: qt
    encoder         : Lavf57.82.102
    Stream #0:0(und): Video: dvvideo (dvh1 / 0x31687664), yuv420p, 720x576 [SAR 1:1 DAR 5:4], q=2-31, 8258 kb/s, 29.97 fps, 29.97 tbr, 25 tbn, 25 tbc (default)
    Metadata:
      creation_time   : 2017-07-15T04:28:28.000000Z
      handler_name    : Core Media Data Handler
      encoder         : H.265
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[AVBSFContext @ 0x7fca44101fa0] Codec 'dvvideo' (25) is not supported by the bitstream filter 'hevc_mp4toannexb'. Supported codecs are: hevc (174)
av_interleaved_write_frame(): Invalid argument

(Strangely, seems to stream is 720x576, even though it is the url-4K-HDR entry)

Whoops. I think I might have been running ffmpeg on a file I'd already patched. 😳
You could do vi -b -c "%s/dvh1/hvc1/g" -c "wq" input.mov && ffmpeg -i input.mov -vcodec copy output.mp4 if you still want an ISO MP4 container.

@benwiggins you sir, are a hero. Works great, looks beautiful on the C7. Now to find a way to override the default screensaver...

@benwiggins you sir, are a hero. Works great, looks beautiful on the C7. Now to find a way to override the default screensaver...

Does the new Apple TV display only these 9~10 HDR videos meintioned in the JSON feed? The last one had more than 20 videos.

That's pretty much off-topic @sabaatworld, the issue already has a lot of comments, let's try to only post useful information related to Aerial and its 4K support. There are plenty of other places on the internet for your question (to which I don't know the answer) 😌

https://sylvan.apple.com/Aerials/2x/Videos/DB_D011_C009_4K_HDR_HEVC.mov https://sylvan.apple.com/Aerials/2x/Videos/LA_A006_C008_4K_HDR_HEVC.mov https://sylvan.apple.com/Aerials/2x/Videos/LW_L001_C006_4K_HDR_HEVC.mov https://sylvan.apple.com/Aerials/2x/Videos/DB_D008_C010_4K_HDR_HEVC.mov https://sylvan.apple.com/Aerials/2x/Videos/LA_A009_C009_4K_HDR_HEVC.mov https://sylvan.apple.com/Aerials/2x/Videos/HK_B005_C011_4K_HDR_HEVC.mov https://sylvan.apple.com/Aerials/2x/Videos/DB_D001_C001_4K_HDR_HEVC.mov https://sylvan.apple.com/Aerials/2x/Videos/LA_A005_C009_4K_HDR_HEVC.mov https://sylvan.apple.com/Aerials/2x/Videos/DB_D001_C005_4K_HDR_HEVC.mov

Is there anything I can help with on this? Maybe we can get a V2 Beta 1 released that simply uses the new JSON url and downloads the SDR versions, when on High Sierra? Refactoring to a new version of swift, and updating the UI seems like something to do in another issue/feature.  As aforementioned, this "feature" is already getting bloated enough :)
In my personal case, I see the Arial UI twice, once when I install it and the second when I check the box in screensavers that says "Arial".  I may be speaking for myself, but I would much rather just use the SDR versions for my High Sierra box, just as it is.
Thoughts?

Running with the suggestion by @AddoSolutions I hacked together a quick 4k demo. You can live in the future now at https://github.com/esetnik/Aerial/tree/feature/4k.
Warning this is completely experimental at this point. 👍

@esetnik Would this slow older Macs down?

Yeah I think 4K cpu decoding on macs without decent integrated graphics will cause high cpu usage and battery consumption. I think eventually it’d be nice for Aerial to offer 4K video as an option in settings.

@esetnik I think it would be much more useful allowing the user to convert the videos into 2K and then selecting them, no?

@esetnik I compiled your branch, but I get the dreaded issue from #4 and can't enable 4k. Did you do anything specific to get it to work on 10.13.1?

I have the same problem but it seems worse than #4 because the close and reopen solution doesn’t work. It’s possible that we can’t compile with Xcode 9 🙁

(I'm on Xcode 9.1, I might add)

Didn't work for me either.

@esetnik Your fork is Swift 3 code but bundling the Swift 4 libraries. Hence the error message.
BTW You should open the Issues tab on your fork so people can report bugs against it there instead of here. :)

Thanks for the suggestion @alistairmcmillan. I’ve activated issues tab on my fork. Happy to accept a PR for fixing the swift 4 bundling. I have limited experience building a screensaver so any help on that front is appreciated.

@esetnik Your fork is Swift 3 code but bundling the Swift 4 libraries. Hence the error message.

@alistairmcmillan can you comment on https://github.com/esetnik/Aerial/issues/1 regarding the correct procedure for bundling swift 3 libraries?

I compiled a release of my branch with Xcode 8. If you just want 4k screensavers now you can download it here

@esetnik Thanks for this.
However, I'm looking at cached mov files' details and it seems that all of them are 1080p. Is it supposed to be like that or I'm missing something? Thanks.

@UniverseXXX please direct all issues with my fork to https://github.com/esetnik/Aerial/issues. I want to avoid polluting the original thread.
I am unable to reproduce the issue of cached videos only 1080p. I have updated my release with more detailed logging information. Please try the following:

Install the latest release of my fork here
rm -f ~/Library/Caches/Aerial/*
open Console and filter for Aerial
open System Preferences -> Screen Saver -> Aerial -> Screen Saver Options -> Cache and hit Download Now

For each download file you should see the url being cached in the logs. It looks like default	13:41:18.856364 -0500	System Preferences	uncached: [id=802866E6-4AAF-4A69-96EA-C582651391F1, name=Dubai, type=video, timeofDay=day, url=https://sylvan.apple.com/Aerials/2x/Videos/DB_D001_C005_4K_SDR_HEVC.mov]
You can confirm that the file is indeed 4k by opening in QuickTime directly from the cache or with media-info:
❯ mediainfo ~/Library/Caches/Aerial/DB_D001_C005_4K_SDR_HEVC.mov
General
Complete name                            : /Users/ethan/Library/Caches/Aerial/DB_D001_C005_4K_SDR_HEVC.mov
Format                                   : MPEG-4
Format profile                           : QuickTime
Codec ID                                 : qt   0000.00 (qt  )
File size                                : 446 MiB
Duration                                 : 7 min 30 s
Overall bit rate                         : 8 314 kb/s
Encoded date                             : UTC 2017-07-15 16:53:17
Tagged date                              : UTC 2017-07-16 02:02:24
Writing library                          : Apple QuickTime

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@High
Codec ID                                 : hvc1
Codec ID/Info                            : High Efficiency Video Coding
Duration                                 : 7 min 30 s
Bit rate                                 : 8 310 kb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.033
Stream size                              : 446 MiB (100%)
Title                                    : Core Media Video
Encoded date                             : UTC 2017-07-16 02:02:22
Tagged date                              : UTC 2017-07-16 02:02:24
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Yep, 4K. 🙂
mediainfo ~/Library/Caches/Aerial/DB_D011_C009_4K_SDR_HEVC.mov

General
Complete name                            : ~/Library/Caches/Aerial/DB_D011_C009_4K_SDR_HEVC.mov
Format                                   : MPEG-4
Format profile                           : QuickTime
Codec ID                                 : qt   0000.00 (qt  )
File size                                : 238 MiB
Duration                                 : 4 min 0 s
Overall bit rate                         : 8 294 kb/s
Encoded date                             : UTC 2017-07-14 15:15:56
Tagged date                              : UTC 2017-07-14 20:00:03
Writing library                          : Apple QuickTime

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@High
Codec ID                                 : hvc1
Codec ID/Info                            : High Efficiency Video Coding
Duration                                 : 4 min 0 s
Bit rate                                 : 8 290 kb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.033
Stream size                              : 237 MiB (100%)
Title                                    : Core Media Video
Encoded date                             : UTC 2017-07-14 20:00:02
Tagged date                              : UTC 2017-07-14 20:00:03
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

@SuperDOS @esetnik @scherii it looks like you are happy users of MediaInfo and at the same time of Dolby Vision stuff, FYI we jut have implemented Dolby Vision support in MediaInfo, with detection and display of version/profie/layers, e.g.:
Dolby Vision                             : 1.0, dvhe.stn@fhd30, BL+RPU

You can use the Windows GUI snapshot (updates/other OSes here (mini 20171201)).

Just an update, I've been running the 4K build on macOS high sierra beta, and it's been great, no issues at all! 🎉 https://github.com/JohnCoates/Aerial/issues/378#issuecomment-346098838

@spitfire - hey, was looking at your JSON feed per comment https://github.com/JohnCoates/Aerial/issues/378#issuecomment-329960804 - it doesn't validate properly.
Each line containing "accessibilityLabel":  has a trailing , character which should be dropped.
Tested with https://jsonlint.com/

@carlashley it’s actually a bug in Apple’s feed 🤣. We should file a radar haha.

D’oh! facepalm.png I completely glossed over the Apple-ness of the URL! Sorry @spitfire!
Definitely file a RADAR. Have done it for GarageBand loops feed and it has been corrected in the past.
Tho maybe Apple might decide to make it harder to find the videos 😉

Yeah let’s not let them know we’ve found it just yet. Although it’s not valid JSON it still does parse so it doesn’t pose a real technical challenge.

I installed @esetnik's fork over Thanksgiving and it looks great. Since I've been back in my office this week and plugged into my external monitors, it had my CPU and GPU running at 50% load on my 2016 MBP. It also was dropping frames like crazy. I think the 2017 models have better/native support for decoding HEVC. If you regularly use multiple monitors, I'd avoid 4K for now.

I’m getting a lot of reports related to 4K video playback but it’s odd because the discrete GPU in the 2016 MBP should be capable of decoding HEVC. Can you confirm that playing the cached videos directly in QuickTime also causes performance issues? Do you also have stuttering playback / high cpu usage  4K@60 HEVC video playback in Photos app? I’m trying to figure out if this is a screensaver exclusive issue or extends to other system apps.

@esetnik - my problem was trying to parse it in python with requests (firstly), then with the standard json library :>
Totally agree - definitely don't want to alert Apple to the fact we're onto it. 💯

@esetnik I'll open an issue on your fork and go into more detail there.

@esetnik I was having playback issues with 4K at first, but after I cached all the videos, playback has been smooth. (Late 2016 MBP)
No shutter in QT.

@esetnik
Thank you very much. The 4k videos are a delight!
Strangely, the issue I had earlier on my mbp 13' (with the workaround of having to enable the clock) is no longer seen!
I see that there are 15 videos now. Any estimate of when we might be treated to the rest of the videos?!! Thanks in advance.
Cheers mate!

@hssuhas I’m glad you like it! Unfortunately Apple only shipped the limited set of 4K videos so that’s all we have for now. Maybe they will release more in the future.

 The MBP 2016 skylake doesn't do HEVC HDR, only kabylake does (found in 2017 models)

Closing this as implemented in 1.4


Update circe-core to 0.12.0-M4

Updates io.circe:circe-core from 0.12.0-M3 to 0.12.0-M4.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention @scala-steward in the comments below.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [{ groupId = "io.circe", artifactId = "circe-core" }]

labels: semver-pre-release

Need http4s compliant release. This has been merged but not released.


[CI] Skip make targets if already invoked once

Current Behavior
All makefile targets are run every time a top level make target is invoked. This can slow down the build process significantly and lead to issues in non-idempotent make target actions.
Desired Behavior
To skip prerun targets, that do not need to be run again, each target step should drop a turd (dummy artifact) or a real artifact. If the artifact exists, we can safely skip the step and continue onwards to other targets.

Resources
https://stackoverflow.com/questions/21745816/makefile-make-dependency-only-if-file-doesnt-exist
Alternatives / Additional Context
I am sure there are other ways to achieve the same goals but since my Makefile-fu is weak I cannot find a better way to do it.
CC @kanishkarj

Hi @nitishm, I don't get the issue. If you are saying to skip make command if dependency is already set-up then it is obsolete, as both npm & go-modules checks the manifest of the dependencies before actually fetching them.

According to @kanishkarj there are a lot of targets that are rerun everytime you call a target with other dependencies. Since I am not to privy with the structure yet, I assumed that some dependencies being rebuilt can slow down the process.

Also see - 10.4 Chains of Implicit Rules in https://www.gnu.org/software/make/manual/make.html#Chained-Rules

@leecalcote Suggestions here?

Also see - 10.4 Chains of Implicit Rules in https://www.gnu.org/software/make/manual/make.html#Chained-Rules

I still don't get it, only recurring make command is go mod tidy which is efficient not fetch deps everytime it's run and same for npm engine ?

Thanks for this, @nitishm! Whether its a reduction of duplicative command executions or a considerate organization of current command groups and their names, our Makefiles need some love.

Do we want this instead ?
https://stackoverflow.com/questions/31492675/node-build-from-makefile-only-if-source-changed
In this way what we'd have is if ran make run-local, we'll actually build only if the source files have been changed, if not we used the built version. It would be really helpful in the case of building ui too.

Makefile has been overhauled. It now includes a couple other Makefiles and help output.


It does not work, and it wants excessive write access

Why on earth does that thing need write access to set hooks and write gists just to sign someones CLA?

@Jens-G if signing it should only require read-access to (at most) your email addresses.
Write access is only needed if you want to set it up for your own projects.
Via which link do you try to sign the CLA? Can you please provide the link and I might be able to help you out.

This one
https://cla-assistant.io/jrsoftware/issrc?pullRequest=417
I may add that I revoked the permissions and at the second try it did what you tell. But not so for the first time. Unfortunately I didn't make a screen shot before to prove it, so you have to believe me.

Can confirm, happened to me 2 days ago also.
I just reproduced it by revoke OAuth and sign in again.

open https://cla-assistant.io
click Sign in with GitHub
it ask for email (which is all good)
after approve, it immediately ask for additional permission5.

This is a duplicate of issues #78, #97, #566, #810 etc.
I suggest to deal with this at issue #566, until it has been fully resolved: I.e., until the CLA assistant works properly with cookies disabled in the web-browser.


246 register create storage account for temporary data

Context
Created a toggle for creating the temp data storage account.
Storage account should not be created for review apps so creation is toggled to off for review variables file
Guidance to review
Can set deploy_temp_data_storage_account to true and deploy a review app to see that storage account gets created.
Link to Trello Card
https://trello.com/c/Vy9zOfCL/246-register-create-storage-account-for-temporary-data
Important business

[ ] Does this PR introduce any PII fields that need to be overwritten or deleted in db/scripts/sanitise.sql?
[ ] Does this PR change the database schema? If so, have you updated the config/analytics.yml file and considered whether you need to send 'import_entity' events?
[ ] Do we need to send any updates to DQT as part of the work in this PR?
[ ] Does this PR need an ADR?

NB: Please notify the #twd_data_insights team and ask for a review if new fields are being added to analytics.yml

 Register - Create storage account for temporary data

I assume this will also need adding to the review_aks.tfvars.json file as well

Added!


SAM-2646 samigo will consider a test 'active' if the due date is passed, it doesn't accept late submissions, but there is at least one user who has been given the ability to take a retake

https://jira.sakaiproject.org/browse/SAM-2646
Samigo has a logic flaw that will make a test appear as 'Active' in the authorIndex UI even if the due date is passed and late submissions are not accepted. More details and steps to reproduce in the JIRA.
The user with the available retake will not be able to use this retake, as the due date has passed and the test does not accept late submissions. The problem is a small piece of code in the AuthorActionListerner.isStillAvailable() method:
if (actualNumberRetake < numberRetake) {  isStillAvailable = true; }
This code block does not take into account the quiz's setting for allowing submissions passed the due date, and thus will always return true (the test is active) if the following conditions are met:

the test is passed the due date
any user in the group/site still has available retakes
late submissions are not accepted

To fix this, the conditional just needs a very small tweak:
if (actualNumberRetake < numberRetake && acceptLateSubmission) {  isStillAvailable = true; }
While I was in the file, I did a bit of tidying up; removed some unused code, commented code, imports and variable assignments, added some finals and some logger statements, etc.

Lots of good changes in here. Better logging, better use of methods. Thanks, LGTM


Deduplicate member path usages

This change is

Current dependencies on/for this PR:

main

PR #2664

PR #2665   👈

This comment was auto-generated by Graphite.


Add codecov to clients

Current status of this PR:

Adds code coverage reporting for C++ client
Adds code coverage reporting for Python client
Adds codecov badge to the repo's readme file
Adds codecov PR bot that gives an overview of how the PR will affect the code coverage

Codecov Report

:exclamation: No coverage uploaded for pull request base (develop@2b634eb). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             develop     #173   +/-   ##
==========================================
  Coverage           ?   82.23%
==========================================
  Files              ?       47
  Lines              ?     2617
  Branches           ?        0
==========================================
  Hits               ?     2152
  Misses             ?      465
  Partials           ?        0


[relay][frontend][pytorch]Fix a bug in the _get_pytorch_value_type function

Fixed a bug in _get_pytorch_value_type(): kind should be used to determine if it is an OptionalType, instead of pt_dtype

Thanks, please fix lint.

@masahi  I have  have corrected the lint.

@tvm-bot rerun


Whenever I include it in my gulp-sass project, "error reading values after not"

Hi, very handy and nice plugin, thanks for that :).
I wonder why I my gulp-sass keeps giving me error whenever I @import "modernizr"; in my project's main.scss file.
It says "error reading values after not".
I am using gulp-sass which is wrapper for node-sass and which is based on libsass.

oh, I think it's because of libsass' lack of support for "&(s)" Any workaround?
I can't use Compass since I tried it in last project and compile times were as high as 20s/save.

Okay, I just switched to "gulp-ruby-sass" and same error persists.

Thanks @ManasN!
You're right, libsass doesn't support Sass 3.4 parent selector implementation yet (sass/libsass#548). That's why you're getting the error reading values after not.
It should work with Ruby Sass though. Compass is completely optional. Please, make sure you're using an up-to-date version of gulp-ruby-sass that uses at least Sass 3.4.

Thanks daniel :)

Hi Daniel, Great Mixin! i have however recently switched to a libsass project, and i am experiencing the same issues mentioned above. Is there anyway this could be slightly altered to work within libsass?

Thanks @dawidvdh! :)
Unfortunately we'll have to wait for libsass 3.2. The release is just around the corner: sass/libsass/issues/1020.

Hey guys, as you may already know, Libsass 3.2 has been released. I've pushed a new beta version of the Modernizr mixin with support for Libsass. If you want to give it a try you can find it here: https://github.com/danielguillan/modernizr-mixin/tree/v3.0.5-beta.1.

:+1: looking forward to trying this!


OCPBUGS-18351: fix: update rpm spec file

added restorecon command to spec file for /etc/microshift and /usr/lib/microshift during upgrades from 4.13 added rule for /usr/lib/microshift folder

/jira refresh

/test microshift-metal-tests

/unhold

/hold
Holding this for a moment, in edge-commit upgrades it seems like the restorecon command isn't being run.

Closing PR in favor of adding these changes directly to #2218


Default import is not a function during build and preview

Describe the bug
I am attempting to use the default export of a package I made (https://github.com/knpwrs/envariant) in sveltekit. It works fine during development, but during the build process I see the following output:
TypeError: envariant is not a function
    at file:///<path_redacted>session.server.js:4:18
    at ModuleJob.run (node:internal/modules/esm/module_job:193:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:541:24)
    at async Promise.all (index 0)
    at async prerender (file:///<path_redacted>/node_modules/@sveltejs/kit/src/core/prerender/prerender.js:375:19)

I can see similar issues that others have had (#1526, #704). I was confused about why my app was prerendering given that I'm not using the static adapter, but I figured out that I could disable prerendering for the whole project (I don't need prerendering for my project, only SSR):
const config = {
  kit: {
    adapter: adapter(),
    prerender: {
      enabled: false,
    },
  },
};

This works for the build, but then when I run vite preview and open my browser I'm greeted with a pretty error page:

When I move the call to envariant to be inside the load function rather than a top-level call, I'm greeted with gql is not a function. This is just gql-tag from npm and that function is likewise called inside the loader.
Everything works just fine in development. Is there anything I have to do to get default exports working when running my built app?
Reproduction
This reproduces for the about page: https://stackblitz.com/edit/sveltejs-kit-template-default-ag2fwq?file=src%2Froutes%2Fabout%2F%2Bpage.js&terminal=dev
Logs
No response
System Info
System:
    OS: macOS 12.1
    CPU: (10) arm64 Apple M1 Pro
    Memory: 1.49 GB / 32.00 GB
    Shell: 5.8 - /opt/homebrew/bin/zsh
  Binaries:
    Node: 18.7.0 - /opt/homebrew/bin/node
    Yarn: 1.22.19 - /opt/homebrew/bin/yarn
    npm: 8.15.0 - /opt/homebrew/bin/npm
    Watchman: 2<IP_ADDRESS> - /opt/homebrew/bin/watchman
  Browsers:
    Brave Browser: <IP_ADDRESS>
    Chrome: 105.0.5195.102
    Chrome Canary: 99.0.4828.0
    Firefox: 104.0.2
    Safari: 15.2

Severity
blocking all usage of SvelteKit
Additional Information
I've updated this from "annoyance" to "blocking all usage of SvelteKit" because I originally thought that this would only be a build-time error, but it appears to affect the built app as well.

Your package doesn't follow the Node.js spec. Please see https://kit.svelte.dev/faq#packages

Indeed, that's the problem! I applied the following change to my library's package.json:
diff --git a/package.json b/package.json
index 4a79287..fc86735 100644
--- a/package.json
+++ b/package.json
@@ -6,8 +6,12 @@
     "access": "public"
   },
   "description": "An Invariant for Environment Variables",
-  "main": "dist/commonjs/index.js",
-  "module": "dist/esnext/index.js",
+  "type": "module",
+  "exports": {
+    "import": "./dist/esnext/index.js",
+    "require": "./dist/commonjs/index.js",
+    "default": "./dist/esnext/index.js"
+  },
   "types": "dist/esnext/index.d.ts",
   "sideEffects": false,
   "scripts": {

Granted, the file extension for require is incorrect. I'm not seeing any immediately obvious generalized way for tsc to output cjs files. It seems like rollup or esbuild bundling would be the way to go rather than transpiling a directory with tsc?
And I'll probably end up switching to urql and using the gql export of @urql/core, rather than gql-tag since they appear to be set up incorrectly as well.
Thank you!


chore: update atlantis to 0.19.0

relates to runatlantis/atlantis#2115

0.19.0 has multi-arch docker image support

@chenrui333 Thanks again!  I've been thinking it would be great if we could automate this, let me know if you have any thoughts.

do you mean automate this as part of the atlantis release process?


Remove broken docs links

Fixes #686
The Troubleshooting and FAQ links are currently broken, so I removed them. We've seen lots of interest in sequence diagrams, so I added a link for that.
Before:

After:

LGTM, thanks!

:tada: This PR is included in version 0.79.2 :tada:
The release is available on:

v0.79.2
GitHub release

Your semantic-release bot :package::rocket:


User is not able to add needs as a poi user or poi admin

Field poi is needed but list is empty

A: One need to update poi membership.


Generic Query with generic EntityStore constructor type incompatibility

I'm submitting a...

[ ] Regression (a behavior that used to work and stopped working in a new release)
[x] Bug report
[ ] Performance issue
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request
[ ] Other... Please describe:

Current behavior

I'm building a generic store/service/query on top of QueryEntity and EntityStore.
The problem is that I cannot call the QueryEntity constructor with the GenericStore.
export interface GenericItem {
  id: string;
  aProp: number;
}

export abstract class GenericStore<TEntity  extends GenericItem> extends EntityStore<
  EntityState<TEntity, string>,
  TEntity,
  string
> {}

export abstract class GenericQuery<
  TEntity extends GenericItem,
  T extends EntityState<TEntity, string> = EntityState<TEntity, string>
> extends QueryEntity<T, TEntity, string> {
  constructor(store: GenericStore<TEntity>) {
    super(store);
         ------
  }
}

The problem is following:
Argument of type 'GenericStore<TEntity>' is not assignable to parameter of type 'EntityStore<T, getEntityType<T>, getIDType<T>>'.
  Types of property 'selectEntityAction$' are incompatible.
    Type 'Observable<EntityAction<string>>' is not assignable to type 'Observable<EntityAction<getIDType<T>>>'.
      Type 'EntityAction<string>' is not assignable to type 'EntityAction<getIDType<T>>'.
        Type 'string' is not assignable to type 'getIDType<T>'.

Expected behavior
The constructor should be callable with the genericly typed store.
Minimal reproduction of the problem with instructions
Try to compile the above code.
Or open the code in stackblitz editor
What is the motivation / use case for changing the behavior?
I'm creating an abstract reusable store for my app use. This is a minor issue, since casting
the store in contructor to super(store as EntityStore<T>); is safe and quick workaround.
I understand that this issue is connected to whole QueryEntity <-> Entity Store generic relationship with the ease of use the single <S> generic argument.
Still, I think this issue should be filed, should someone else face the same problem.
Environment

Akita 4.23.2
Typescript 3.8.3

Further details
I dug deeper into the problem and when changing the QueryEntity constructor to:
constructor(store: EntityStore<S, EntityType, IDType>, options?: QueryConfigOptions);
the constructor call also fails to compile due to some other inner incompatibilities of the store.

Typescript expects the same types as you inherit, meaning:

TEntity should use Akita's getEntityType<T>
string should use Akita's getIDType<T>

@danzrou If Akita disallows using different generic types for EntityType and IDType than the defaults for store and query, then these types should not be generict at all.
I'm updating the original question to clarify the need to use TEntity.
The TEntity extends GenericItem is used because it ensures type safety for the operations/query on the store - the query can use aProp in a type safe manner.

@danzrou But thanks for providing an approach for this.
I've dug deeper into approach you're suggesting and the problem (and the reason I started with specifically use TEntity) is that the EntityType is not inferred to child of GenericEntity.
/** Alternative approach */
export interface GenericState<
  EntityType extends GenericItem = any,
  IDtype = any
> extends EntityState<EntityType, IDtype> {}

export abstract class SecondGenericStore<
  TState extends GenericState
> extends EntityStore<TState> {}

export abstract class SecondGenericQuery<
  TState extends GenericState
> extends QueryEntity<TState> {
  constructor(store: SecondGenericStore<TState>) {
    super(store);

    // item is inferred as getEntityType<TState>, item.aProp is not inferred (any)
    this.selectAll({filterBy: item => item.aProp})

  }
}

Sure thing - did that solve it out?

@danzrou Unfortunately no, sorry for not being clear. In the code in previous comment the selectAll unfortunately does not infer the item properties correctly. Since I'm using a generic query with predefined selects for two types of entities it was the reason I rewrote my query to accept the entity type as an generic argument.
I see two options here. Wait and see if the typescript inference gets better (I haven't tried 3.9 yet) or I could try an PR for a wider generic compatibility of the store/query.

I'm not sure if that can help you but I hope it will - below you'll find my implementation for an entity query which inherits Akita's entity query and is generic:
https://gist.github.com/danzrou/6dcedcb483c4f0a900cfb6447710b307

That's quite interesting implementation. It made me look over my code again and I came up with a solution that corrects the inference ignoring the extends type constraint of the EntityType generic parameter. I consider this a workaround of typescript incompleteness.
export interface ThirdGenericState<
  // the `= GenericItem` is used as the inferred
  // if you put Elephant in there, the item in the selectAll function is inferred as Elephant
  EntityType extends GenericItem = GenericItem,
  IDtype = string
> extends EntityState<EntityType, IDtype> {}

export abstract class ThirdGenericStore<
  TState extends GenericState
> extends EntityStore<TState> {}

export abstract class ThirdGenericQuery<
  TState extends ThirdGenericState
> extends QueryEntity<TState> {
  constructor(store: ThirdGenericStore<TState>) {
    super(store);

    // the lack of inference is corrected by the default generic value on line 48.
    this.selectAll({filterBy: item => item.aProp > 1})
  }
}

But still I believe that my first implementation is technically correct and should be compilable - when the Akita query and store have generic parameters, they should be compatible when explicitly set.

I can agree - but I think the gap here is TS's mismatch between getEntityType and others which are totally compatible.

I am closing this issue. If there is any news here, feel free to reopen it.


Fix command enablement by using specific activation contexts for O#, Roslyn standalone, and Roslyn devkit

There were various bugs in the old enablement clauses.  For example

Devkit overrides the useOmnisharp configuration setting.  If useOmnisharp was true and devkit was installed, O# commands were displayed (but would not work since devkit activated, not O#).
Some of the restore commands were only enabled when using Roslyn standalone, but not using O#.

This PR modifies the enablement clauses to look at an activation context.  We explicitly set the activation context after we've run all the other logic to determine which server to actually launch.
Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1891108 (and other non-filed bugs)

@dibarbet Does this mean though that commands aren't visible until the extension has launched? For some that's probably not a big deal, but what about the ones that might trigger the server launch in the first place?


Add separate sets of hard- and soft-enforced linters

Soft-enforced linters will only annotate code changes, hard-enforced linters actually make the workflow fail if they encounter an issue.
You can run the soft linters manually on your machine:
golangci-lint run --config .golangci-soft.yml

Thanks, @muesli!


Add a chart for nsqlookupd

nsqlookupd will be important for us if we want to add more nsqd instances - otherwise known as horizontally scaling.
See http://nsq.io/components/nsqlookupd.html for documentation on nsqlookupd
Notes on horizontally scaling our queueing subsystem
Note that, as of this writing, I have been able to run 4000 concurrent instances of this command:
boom -n 1000 -c 500 ...

In english that test produced 4,000,000 total log messages with up to 2,000,000 of them concurrently entering the queue. A single nsqd pod was able to handle that specific workload.
This test was run on a Google Container Engine cluster of 10 n1-standard-2 (2 vCPUs, 7.5 GB memory) nodes.
When I ran 5000 concurrent instances of the above boom command, the nsq crashed.

So it is completely possible to run nsqlookup within this image. I had that working at first but had the following concerns.

You will need to dynamically look up nsqlookupd pods within the boot script for nsqd. This means you have a dependency on start order (nsqlookupd must start first). That's fine as we can just exit(1) within the bootscript if we dont find any nsqlookupd ips/pods.

You will need to use nsqlookup's pod ip not a service ip.

Im not sure what we will do about scaling the nsqlookup pods (will we ever need to do that?). If you do scale nsqlookupd from 1 to N you will probably want to tell nsqd about that change too. You will definitely want to tell logger/fluentd...

@jchauncey thanks for those points. Comments inline

I think exit(1) is ok. we're currently doing that in a lot of other places like the builder, controller and logger when the controller, database, and nsqd (respectively) are not available, and just letting kubernetes restart the pod as necessary
Why the pod IP?
Good point. I was thinking that this issue would be scoped to just a single nsqlookupd. It still represents a single point of failure, but it would be a good start since a failed lookup node wouldn't stop logs from going into or out of nsqd. For failover, we would certainly need to build some more intelligence into the system to route discovery traffic around failed nodes. Thoughts on ignoring high availability for nsqlookupd nodes in this issue?

So I originally had nsqlookupd and nsqd working but decided if I could figured out how to scale nsqlookupd why not just have 1 nsqd instance.
Maybe we dont need to use the pod ip. Reason I said that was that nsqd needs to know where it can find all nsqlookupd instances. But if you used the service ip here it would just round robin between those pods which might work well.

Closing this for the same reasons as https://github.com/deis/nsq/pull/4#issuecomment-257302307.


Keyframe relative to last frame

Now, I'm creating an animation like this:

_

Move object
Create animation at 0

_

Move, scale object
Create animation at 15

_

Move object
Create animation at 35

_

Reset transform
Create animation at 50

It will be useful, if we could enter relative values:

Move object, +0
Move, scale object, +15
Move object, + 20
Reset transform, + 15.

So, there will be 50 frames.

Hey this is great idea! Set relative keyframe numbers to based last keyframe of animation. Need to check this one after the holidays...


[FLINK-10680][streaming] Fix IllegalArgumentException caused by negative offset in TumblingWindows

What is the purpose of the change
By design, the offset could be either positive or negative to fit in different time zones, but there's a check that ensures the offset is greater than zero.
This PR fixes the IllegalArguementException caused by negative offsets in TumblingEventTimeWindows and TumblingProcessingTimeWindows.
Brief change log

Change the condition of the parameter checks in TumblingEventTimeWindows and TumblingProcessingTimeWindows from 0 <= offset < size to 0 <= abs(offset) < size.

Verifying this change
This change added tests and can be verified as follows:

Updated the tests for invalid parameters for time window assigners.
Added tests for negative offsets of time windows.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): yes
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature?  no
If yes, how is the feature documented? not applicable

Close cause this has been fixed.


Fix: Obsolete version of GraphQLClient.SendAsync forwarding authScheme as token

Whoops! Thanks for catching this.


OLS-632: remove development configuration settings

Description

Type of change

[x] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library

Related Tickets & Documents

Closes https://issues.redhat.com/browse/OLS-632

Checklist before requesting a review

[x] I have performed a self-review of my code.
[x] PR has passed all pre-merge test jobs.
[x] If it is a core feature, I have added thorough tests.

Testing

Please provide detailed steps to perform tests related to this code change.
How were the fix/results from this change verified? Please provide relevant screenshots or results.

/test bundle-e2e

/test bundle-e2e

/test bundle-e2e

/assign @xrajesh

/lgtm
/approve


Publish stuff for undecorating C++ symbols

I would like to see code (or docs) for undecorating C++ symbols made available.
The Visual C++ runtime library contains a function called unDName (or something like that, its referenced all over the CRT source code) but that function is undocumented and no official header for it is available (3rd party headers exist for it but they may not have the correct definitions). Also not everyone working with PDB files can link to the Visual C++ runtime library (even if undname was usable). The Visual Studio 2015 CRT source code includes an undname.cxx file but it is missing required header files (and cant be used anyway because of the license)
There is also a function UnDecorateSymbolName in dbghelp.dll but it doesn't support the full set of decorated names and not everyone working with pdb files is in a position to use dbghelp.dll or the UnDecorateSymbolName function.
There is a tool called undname.exe in the Windows SDK but that cant be redistributed and its a tool, not a library you can call so that isn't an option.
There are 3rd party implementations for undecorating symbols but those may not be complete or accurate.
There is also code claiming to be from Microsoft at this URL https://github.com/sundayliu/windbg/tree/master/langapi/undname but that doesn't have any license attached so we cant use it. Also we dont know how old that is or if it was even legitimately released by Microsoft (its quite possible this code came from one of the various source code thefts Microsoft has experienced in the past (e.g. Windows NT4 source or Windows 2000 source).
I would like to see usable complete source code for undecorating names published (under the same MIT license as the PDB code ideally) or if that is not possible, complete official header files required for using the undname function in Visual C++ libraries.

Wine has an implementation: http://source.winehq.org/git/wine.git/blob/HEAD:/dlls/msvcrt/undname.c

I already mentioned "3rd party implementations" above. My point is that no-one can be sure if the WINE implementation is 100% complete/accurate (including all the weird corner cases) and 100% up to date with the latest compiler bits Microsoft is shipping. (don't get me wrong, the WINE guys do some good work but just because someone else has already reverse engineered this stuff doesn't mean its not worthwhile for Microsoft to at least see how fesable it is to publish the official implementation)

Yes please, this would be excellent.


Filter logs by section

Implementation of #67

[x] Remember selected groups when reopening the filter dialog
[x] Make group name human readable

Coverage decreased (-0.3%) to 16.786% when pulling 0db3e2d006afadbcd0650ce5a474ee79f63f48bc on issue_67/logs_filter into e1d635d62fc770a3c37475e8d9a5d9b848da4982 on master.

Coverage decreased (-0.4%) to 20.677% when pulling 289e2b8432e02d70ba91f6a1d1c479bf50ebf338 on issue_67/logs_filter into d5bc7bab7b91f1f622196a3222bb217c7108895b on master.

Coverage decreased (-0.2%) to 20.855% when pulling 9af6b0565afdac8d20aff5d97d1fc68b0f49b60a on issue_67/logs_filter into d5bc7bab7b91f1f622196a3222bb217c7108895b on master.

Coverage decreased (-0.3%) to 20.733% when pulling 11b1c1d1401ea904a0199717906ab4012c6c39db on issue_67/logs_filter into d5bc7bab7b91f1f622196a3222bb217c7108895b on master.

Coverage increased (+0.4%) to 21.416% when pulling be568a8bc29d81af1fa39a47e530fdb9bef132c8 on issue_67/logs_filter into d5bc7bab7b91f1f622196a3222bb217c7108895b on master.


[BUG?] credential-0.json does not match the VC data model

The VC data model defines the credentialSubject as : a set of objects that contain one or more properties that are each related to a subject of the verifiable credential.
This implies that the credentialSubject property must not be empty, since it has to have one or more properties.
The credential-0.json sample file used in these tests seems to not respect this since it is expecting credentials with empty credentialSubject to be valid.

Confirmed that @mirceanis says is still the case in version 1.1 of the spec: https://www.w3.org/TR/2022/REC-vc-data-model-20220303/

I think we decided that empty object is a value-able negative test care and that it is not a valid VC, test vectors for happy path testing were updated.


Uncaught ReferenceError: ExtendableEvent is not defined

There is a bug when you open page
Observed behavior
Uncaught ReferenceError: ExtendableEvent is not defined
Problem in this line: offline-plugin
Environment
Browser: Chrome version 59.0.3071.115 (64-bit)
OS: MacOS 10.12.5
offline-plugin: 4.8.3
webpack: 3.3.0
P.S.
With offline-plugin: 4.7.0 works fine

Hi @kolima. I'm not sure how that's possible. ExtendableEvent is there in that Chrome version.

Hi @NekR. I hope this helps.

It isn't defined on Window. ExtendableEvent is an constructor on ServiceWorker global object and that code you linked to is running in ServiceWorker.

Thanks for answer, and you right. sw.js is included in index.html and running like js file.

Sorry to comment on closed issue but I have run into this and its not clear how is this fixed
Uncaught ReferenceError: ExtendableEvent is not defined
    at eval (empty-entry.js?5936:4)
    at eval (empty-entry.js?5936:42)
    at Object.../node_modules/offline-plugin/lib/misc/sw-loader.js?json=%7B%22data_var_name%22%3A%22__wpo%22%2C%22cacheMaps%22%3A%5B%5D%2C%22navigationPreload%22%3A%22false%22%7D!../node_modules/offline-plugin/tpls/empty-entry.js (sw.js:310)
    at __webpack_require__ (sw.js:233)
    at sw.js:297
    at sw.js:300

the sw.js is imported as a very first script in index.html
<script src="/build/sw.js"></script>

any help?

I don't remember the problem and how we fixed it, and now we use another plugin https://www.npmjs.com/package/workbox-webpack-plugin

Actually, the issue was to try inject sw.js script directly into html. This would never work because sw.js has node_modules dependencies. For future readers, don't use


A Restaurant Web Application

Is your feature request related to a problem? Please describe.
I have Made a restaurant website using HTML, CSS and Javascript.

@Ayush7614 Please Assign this task to me

I have integrated google maps, you will get detail of every item you see on the site. For such many things, I don't think this should come under level2


fixes a bug where readonly properties would fail to deserialize for TypeScript.

Signed-off-by: Vincent Biret<EMAIL_ADDRESS>

example of the issues it generates https://github.com/microsoftgraph/msgraph-sdk-typescript/actions/runs/3523457834/jobs/5907645592


tricks about result on nuscenes leaderboard

Hi, what strategies are available for submitting results to the nuscenes list? Do you have a strategy for model ensamble or other data augmentation methods that you use? thanks!

I don't know much about tricks. Multimodal Virtual Point is our best entry and it only uses a single model without tta.
AFAIK, you don't need to compare those ensemble models if you do any paper submission so I don't think any tricks are that useful.


✨ transaction execBuy

This feature will handle BUY tx from one single point
PR Type

[x] Feature
[x] Refactoring

Context

[x] Closes #4942

Optional

[x] I've tested it at /snek/gallery/659233203-5

side note

LIST 0.0010
BUY 0.0009
???

@roiLeo what does the side note mean?

@roiLeo what does the side note mean?

On my test /snek/gallery/659233203-5

This is weird, there is a difference between LIST price and BUY price

Tested on
https://deploy-preview-5150--koda-nuxt.netlify.app/rmrk/gallery/9290733-94ae5001bac3833457-PNPRT-NFT_PINUP_GIRL_006-0000000000000005
WFM
I can't sadly buy any nfts on snek, some balance issue

@roiLeo because royalty was calculated
^-^

pay 44 usd


Add OIDCProfile refreshToken for KeyCloak

It seems there is no way to refresh the token currently. I saw the comments in this conversation and thought I would create a ticket.
https://groups.google.com/forum/#!searchin/pac4j-users/refresh|sort:relevance/pac4j-users/ZhX3LhCX1OQ/KyU1-3w3BgAJ
I can contribute but it'll take me some time. I am creating this issue to start a conversation on what would need to be done.

Yes, this is part of the OpenID Connect protocol, but we haven't implemented yet. I'm sure this is completely feasible.
This will be part of pac4j and thus available to play-pac4j.
We are developing the version 3.0.0 of pac4j untill the end of the Year, so you can certainly get in that version.
Please open the same issue on the pac4j project instead.

Thank you. I'll play with the code and re-open it later on then.

Was this issue ever implemented? Did anyone find an elegant way to refresh the tokens in the background?


Add support for unlift strategies

Supersedes #16.

I really like what you did here! +1

Thanks for giving me a nudge in the right direction ;) I wasn't satisfied with not being able to unlift in different threads, but I didn't have a good idea how to tackle this.

Did your run some benchmarks? I am really interested in the numbers.

What numbers exactly do you have in mind? The current benchmarks didn't regress. In fact they're now slightly faster because send doesn't construct the unlifting function, effect handlers do that only when then need to. I also checked performance of stateM from State.Dynamic and it's faster now, presumably due to GHC doing better job at inlining since unlift is now constructed locally in the effect handler.
I'm unsure about two things yet:

withUnliftStrategy resetting for new threads (what do you think?)
having both BoundedConcUnlift and UnboundedConcUnlift

I'll let it sit for a bit.

What numbers exactly do you have in mind? The current benchmarks didn't regress. In fact they're now slightly faster because send doesn't construct the unlifting function, effect handlers do that only when they need to. I also checked performance of stateM from State.Dynamic and it's faster by ~20% now, presumably due to GHC doing better job at inlining since unlift is now constructed locally in the effect handler.

These were the numbers I had in mind. Nice results! :100:

withUnliftStrategy resetting for new threads (what do you think?)

I'll have a look.

having both BoundedConcUnlift and UnboundedConcUnlift

I'm not (yet) sure about this one.

Do you have any thoughts on garbage collecting old ThreadIds from esMap? Will that be an issue?
If so, maybe we can run something like this when the size of the map exceeds a certain threshold :
gc :: Map ThreadId (Env es) -> IO (Map ThreadId (Env es))
gc esMap = do
    let ks = Data.Map.Strict.keys esMap
    alive <- filterM (fmap isAlive . GHC.Conc.threadStatus) ks
    return $ Data.Map.Strict.restrictKeys esMap $ Data.Set.fromList alive
    where
        isAlive :: GHC.Conc.ThreadStatus -> Bool
        isAlive ThreadDied = False
        isAlive ThreadFinished = False
        isAlive _ = True

The map is local to the scope of liftBaseWith/withRunInIO/localUnlift and these scopes tend to be short-lived :slightly_smiling_face:
In theory you can stick the unlifting function in a data structure and decide to use it later, but that isn't going to work terribly well since:

It's parametrized by es
For new threads it'll give you state of the environment at the point of the unlift function's creation, not the current one, which is generally what you'd expect.

I guess what I meant is:
foo k = withRunInIO $ \runInIO -> forever $ do
    x <- UnliftIO.Async.async $ runInIO k
    ...

This is of course an awfully artificial example :wink:

Oh, I see. Right, it's a bit artificial, but still somewhat relevant.
There is another problem with such usage (see here for more info): since the map stores ThreadIdS, dead threads will not be garbage collected.
Ideally we'd like to keep track of Weak ThreadId made with mkWeakThreadId, but Weak objects don't have an Ord instance, so we'd need to store them in an array and then lookup would take linear time :disappointed: But it wouldn't prevent garbage collection and dead threads wouldn't be a problem by design.
Linear lookup is not a problem for BoundedConcUnlift with small bound, but wouldn't work well for Unbounded if there is a lot of threads.

Ok, problem solved. Now the map doesn't store direct references to ThreadIdS and EnvS and cleans itself.

Ah, there is one more thing.
Currently environment changes are persisted between unlift calls within the same thread:
put @Int 0 -- thread-local variant of State
withRunInIO $ \unlift -> forkIO $ do
  unlift $ put @Int 1337
  ...
  unlift $ get @Int -- gives 1337

This is desirable in most cases (otherwise lifting e.g. withUnmask family of functions wouldn't work properly), but not always. E.g. in pooledMapConcurrently from UnliftIO.Async you wouldn't expect environment changes to persist between independent operations, but now they do, because worker threads are reused!
Luckily the variant that doesn't persist the environment between unlift calls is much easier to write than the one that does. But that will increase the number of unlift strategies.

Nice, I was working on this as well. Have a look at https://github.com/mmhat/effectful/commit/ed86c24f0e5078fc1f1002374fbf594cba2ad8a9.
Your work seems to be more advanced; I'll have a look at it throughout the next days (probably tomorrow :smile: ).

Nice, your idea is pretty much the same as mine :+1:  The difference that adds complexity in my case is that I account for possibility of collisions of ids returned from rts_getThreadId (since that's also what Hashable ThreadId uses internally).
I'm not sure if that can happen in practice, but I didn't find a claim that it won't.

I'm not sure if that can happen in practice, but I didn't find a claim that it won't.

TLDR: No, in practice there should be collissions.
I did some research on this and according to this discussion it should be Pracitically Impossible. Note that even though the issue is still open the changes relevant to us got merged in https://gitlab.haskell.org/ghc/ghc/-/commit/e57b7cc6d8b1222e0939d19c265b51d2c3c2b4c0.
Code related to this is:

The rts_getThreadId of the RTS: https://github.com/ghc/ghc/blob/0619fb0fb14a98f04aac5f031f6566419fd27495/rts/Threads.c#L180
Unfortunately this is not exposed; I consider filing an issue in the GHC tracker to change that.
How a new thread id is obtained in the RTS: https://github.com/ghc/ghc/blob/0619fb0fb14a98f04aac5f031f6566419fd27495/rts/Threads.c#L128
The counter supplying the RTS with new thread ids: https://github.com/ghc/ghc/blob/0619fb0fb14a98f04aac5f031f6566419fd27495/rts/Threads.c#L32
The size of that counter: https://github.com/ghc/ghc/blob/0619fb0fb14a98f04aac5f031f6566419fd27495/includes/rts/storage/TSO.h#L25

rts_getThreadId (since that's also what Hashable ThreadId uses internally).

I didn't know that there was a Hashable instance for ThreadId! I came up with my solution by looking at GHC.Conc; In particular the Show instance of ThreadId.

I did some research on this and according to this discussion it should be Pracitically Impossible.

How so? The program in OP demonstrates the collision. Show instance uses rts_getThreadId.
The PR https://gitlab.haskell.org/ghc/ghc/-/merge_requests/1254 is a bit weird - long is not 64bit long e.g. on x86_32 (which is admittedly going away, but still).
It's also worth noting that the FFI import in both GHC and hashable is:
foreign import ccall unsafe "rts_getThreadId" getThreadId :: ThreadId# -> CInt

And CInt is a newtype for Int32. Looks to me like this wasn't properly fixed.

Well, the OP demonstrated the collision when the thread counter used to be 32bit. Note that https://gitlab.haskell.org/ghc/ghc/-/merge_requests/1254 was never merged into the mainline branch; Instead https://gitlab.haskell.org/ghc/ghc/-/commit/e57b7cc6d8b1222e0939d19c265b51d2c3c2b4c0 was merged. But maybe I didn't get your point, do I?

And CInt is a newtype for Int32. Looks to me like this wasn't properly fixed.

This is indeed something that should be fixed.

Oh, yes, your right! It is not what it said on the tin!

Not to mention that the behavior of the program from OP didn't change at all even with this patch, because FFI import still gets the id truncated to 32 bits, even on x86_64. It's a bit funny that it even works. I'm rusty with C calling conventions, but presumably the result of rts_getThreadId ends up in a 64 bit register, but GHC inspects the register that projects the lower 32 bits of said register.

Ah, nice, benchmarks! :+1:

Anyway, what matters for this PR is that Hashable still can produce collisions, so the dance I do in concUnliftIO is unfortunately necessary.

This is sad but now I understand at least why you went down this rabbit hole.
FYI: In the meantime I picked up the GHC issue. It seems the issue you described above is scattered all over the RTS.


invalid subcommand

calling kn results in an invalid sub-command error.
$ cd ~
$ kn .cargo
Value of arg `subcommand` is invalid.

I couldn't reproduce this behavior. Which shell are you using? Can I see your _kn init invocation (in your shell's startup file)? What's the output of this invocation? For example, if your startup file looks like this:
_kn init --shell fish | source

then what's the output of the command below?
$ _kn init --shell fish

I initialized kn and starship in on go. And that does not work so the init command was never executed. Separating them in just two separate commands works (order does not matter).
starship init fish | source
_kn init --shell fish | source

Sorry for the inconvenience.


Rename accountAddress to walletAddress in IHL

For clarity, in IHL we should use walletAddress instead of accountAddress, since we are switching to using EOA addresses to identify the user, and walletAddress makes it clearer that the address is for an EOA, not an MTW
Eng:

this will require db migrations
this entails breaking changes to the API

AC:

accountAddress replaced with walletAddress everywhere in IHL
consumers of IHL's API are updated (wallet repo, e2e tests)

starting to think this isn't worth the time. We already conflate the two terms in some places in the wallet repo already...