Long running ResponseBody async Task never gets canceled

I was able to write an event source text stream as a ResponseBody, but I have found that when the client disconnects, the task continues running indefinitely, even when the server tries to gracefully shut down.
I had assumed that the client disconnecting from the request would cause the body sending task to cancel. I tried diving through the code, but it got pretty deep into NIO, and well beyond my depth pretty quickly. Should the body async task become cancelled when the client disconnects?
Is there some other signal that such a long-running response might tap into for client disconnection, or graceful shutdown?
Here's a response that reproduces the issue:
Response(
    status: .ok,
    headers: [
        .cacheControl: "no-store",
        .connection: "keep-alive",
        .contentType: "text/event-stream",
    ],
    body: ResponseBody(asyncSequence: AsyncStream { stream in
        Task {
            stream.yield(ByteBuffer(string: "event: ping\ndata: \n\n"))
            while !Task.isCancelled {
                if case .terminated = stream.yield(ByteBuffer(string: "event: ping\ndata: \n\n")) {
                    break
                }
                try? await Task.sleep(for: .seconds(20))
            }
            // Note that more useful events would also be sent as applicable.
        }
    })
)

Note that I also tried something like the example below, in case my extra layer of Task was causing the issue. However, this also did not become canceled when the client disconnected, nor during graceful shutdown.
ResponseBody { writer in
    for await buffer in stream where !Task.isCancelled {
        try await writer.write(buffer)
    }
    try await writer.finish(nil)
}

There two issues here

Your unstructured task will cause issues as cancellation is not propagated to unstructured classes as they are not the child of another task.

There is currently no easy way to catch client input close and pass it onto the HTTP2 handler.  If the client completely closes the connection then the writes will fail and you will exit the write function. But if it does a clean close it only closes its side of the connection, the server is allowed to continue writing and there is currently no way to pass the event inputClosed to the HTTP handler. The way the NIO concurrency types are structured makes it hard to do this.

It is perfectly valid for a client to send a request and close its side of the connection before getting a response, so I can't cancel all requests based off the client closing input, but I need to be able to pass that information to users so they can make that call. Just haven't worked out exactly how to do that yet.
One possibility is to add an option to disable half closure and then writes should fail as soon as the input is closed.
I will speak with NIO team as well to see if they have any ideas.

Thanks for the quick reply! Yeah, this sounds like a tough one. For most normal requests I anticipate the body completes quickly, and actual task canceling may cause more trouble than it's worth. However, for these long-running event streams, it can become a memory leak. My unstructured tasks don't help either. I'll see if I can clean that up.
I did find the cancelOnGracefulShutdown helper from swift-service-lifecycle, that gives a signal for when the server is shutting down. I see they are using some @TaskLocal machinery for that. Perhaps a similar api could be made for inputClosed? If there was a cancelOnRequestDisconnect that would be enough for this case.

Thanks for the quick reply! Yeah, this sounds like a tough one. For most normal requests I anticipate the body completes quickly, and actual task canceling may cause more trouble than it's worth. However, for these long-running event streams, it can become a memory leak. My unstructured tasks don't help either. I'll see if I can clean that up.

Yeah I understand this is an issue and want to provide a solution. I've a few ideas to try out but it might require a special http1 handler for servers that have long running cancellable requests.

I did find the cancelOnGracefulShutdown helper from swift-service-lifecycle, that gives a signal for when the server is shutting down. I see they are using some @TaskLocal machinery for that. Perhaps a similar api could be made for inputClosed? If there was a cancelOnRequestDisconnect that would be enough for this case.

Yes a TaskLocal might be a solution for passing inputClosed state down the callstack.
On the subject of graceful shutdown. When the server receives a graceful shutdown message it stops accepting new requests and waits until all requests have been responded to before shutting down the server. In the case of your long running sse request it will never shutdown because you never return a finished response so you need to handle graceful shutdown within your response writer to end the sending of server sent events and finish the response.

I'd like to emphasize that this is not an urgent issue.
I was able to get my sse response shutting down gracefully with cancelOnGracefulShutdown. I don't have anything heading to production right now. I'm just learning & trying to understand server-side swift for future recommendations. For my toy app, once I got graceful shutdown handled, my responses do seem to terminate a few minutes after disconnects, probably because the other half finally closes & the writes fail.
In a future production app, I'm not sure how concerning it would be to have these streams hanging out longer than needed, but it definitely would give more confidence if there was a mechanism to clean them up immediately after disconnecting.

here is my "urgency += 1" to this (I am having the exact same issue)
I feel like SSE is having a bit of a renaissance, and when thinking about the "hypermedia" frontend bubble a la HTMX or datastar - where Swift can actually fit in quite nicely - a lot of the reactivity is driven by SSE streams.
whatever hummingbird can do to make this "work correctly by default" will go a long way I think.

See #631

This can now be resolved with RequestBody.consumeWithInboundCloseHandler. See server-sent-events example in hummingbird-examples repo for an example of usage https://github.com/hummingbird-project/hummingbird-examples/tree/main/server-sent-events.

awesome. Thank you!


Support for Google Cloud STT

I'll make another PR with documentation if you are interested in this patch.
I'm unsure on how to use the language code from the profile (e.g. "en") when Google APIs need a locale code (e.g. "en-US"). Do I have that information in the profile?

I've added a "locale" field to the profiles with values like fr_FR and en_US. Substituting the underscore should get you the correct value for Google STT.
Would you like to add that in before this is merged?

Would you like to add that in before this is merged?

Sure! I've modified the PR.

I've added documentation.

Looks great, thank you very much!

I realized I've made a mistake in the log message for the documentation commit: "TTS" instead of "STT". Sorry... you can fix it if you'd like.


[SF3] Some recipes not having arrow to flip between options

Skyfactory 3
Easiest to see this on is Chocolate Milk. It has option for Bucket of milk, fresh milk, and soy milk. Other recipes have the arrow to flip between the options to choose what you use but this (and many others) still flip quickly like in NEI/JEI so it's impossible to tell what it's using.

This is the handling for ore-dictionary items, which is different from multiple recipes resulting in the same (which is what the arrows are used for).
It's meant to switch way slower than it currently does (which was fixed in the latest version of the mod).
You can click the flipping item to lock it to that type, and also use the scroll wheel on it to pick one of the options.

oh cool! I didn't know that. Kinda wish there was an 'Info' tab in the book or something that had those instructions ingame as well as how to properly setup the kitchen. Still. I really REALLY love this mod!!!


Feature: backport modifiers

I previously shared this article https://davedelong.com/blog/2021/10/09/simplifying-backwards-compatibility-in-swift/
and while working on the new booking UI I started needing support for the .searchable modifier which
is only available in iOS 15.
I took some of the modifiers that I did and I included them in the backport type.
Check the individual commits for details.

I forgot to mention it.
This will crash during runtime if used in the App Store build on iOS 14 as the other conditional code.
but only if the code it's used. To begin with it will be only used in a development branch so it's not a problem


remove node 14 from travis yml

Description
we keep seeing errors for linux + node 14 installing aio app plugins since aio-lib-runtime node version got pinned to 10 or 12: https://github.com/adobe/aio-lib-runtime/blob/master/package.json#L56
failing test: https://travis-ci.com/github/adobe/asset-compute-integration-tests/jobs/403569170
Although it is only failing on linux, the easiest solution is just to remove node 14 from both windows & linux builds since its not supported by the aio cli.
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] I have signed the Adobe Open Source CLA.
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

How do we remember to add it back once aio adds support for it? We should have a ticket to track

@alexkli made a ticket to track it here: https://jira.corp.adobe.com/browse/NUI-907


Feature/list of pet photos

When deleting pet, now photo are deleted. Split updating pet, by update only pet, and update only photo. Added list of photos, and adding and deleting them. Some little fixes.

outdated, feature/user&shelter_pets implements better version.


Db return collection item implements \ArrayAccess interface

Why
I have this piece of code
$generate_data_by_user = function($user) {
  //validate ...
  return [
    'identity' => $user['id'],
    'name' => $user['real_name']
  ];
}

User::where('age', '>', 18)->get()->map($generate_data_by_user);   // pass

or

Db::table('code_user')->where('age', '>', 18)->get()->map($generate_data_by_user);  //fail

I don't understand what you're trying to say... Maybe I made a mistake it should be Db::table('code_user')->where('age', '>', 18)->get()->map($generate_data_by_user);  //fail
The db method is just a helper function I added for myself

I just want to say that a lot of times we get used to using $item[key], Because Model implements \ArrayAccess, but , Db return value not implements \ArrayAccess, This makes me need to determine the source of data is Model or Db

It's not clear - what is the exact issue you are describing here?

Db::table('code_user')->where('age', '>', 18)->get()->map($generate_data_by_user); //fail

This tails with your closure because your receive a stdClass object with properties hydrated from the DB result but you are trying to access it like an array.
This is exactly as https://github.com/laravel/framework/issues/25821#issuecomment-425611503 described: Eloquents user model implement \ArrayAccess that's why $user['id'] works on Eloquent results but not on DB results.
TL;DR: if you really have to use the same closure, use $user->id or even better, not mix them.

I just wanted to make a suggestion, Db return stdClass object implements ArrayAccess interface.

Thanks @HongjiangHuang
Unfortunately this GitHub area is not for ideas, suggestions etc. This is only for issues/bugs with the framework code itself. You are able to open a ticket at https://github.com/laravel/ideas
Alternatively you are able to open a PR using the Contributions guidelines: https://laravel.com/docs/5.7/contributions#which-branch and add the interface for Taylor to decide to accept or not.


Feature Request: RG351V Support

The stock firmware sucks, it would be great to get ArkOS on my RG351V!!!

I would recommend pinning one of these to the top to avoid more being made :)

Speaking of the vibration on a 351P when it boots up, is it possible to turn that off completely? I haven't found anything about that in the wiki.

Speaking of the vibration on a 351P when it boots up, is it possible to turn that off completely? I haven't found anything about that in the wiki.

Don't worry my guy it's done by the OS and not the device itself at boot.

Speaking of the vibration on a 351P when it boots up, is it possible to turn that off completely? I haven't found anything about that in the wiki.

Don't worry my guy it's done by the OS and not the device itself at boot.

Does that mean it can be turned off somehow?

We're waiting for Anbernic to release the sources for the bootloader to
tweak the vibration and other things. So, just sit tight for now. :)

I haven’t found a way to completely turn off vibration on boot yet.

Oh nice, sources are out.
Just so this can be here for others looking for it:
https://github.com/SummerSunGenius/RG351V_kernel
https://github.com/SummerSunGenius/RG351V_buildroot
https://github.com/SummerSunGenius/RG351V_uboot

Vibration is down to about 2 seconds on initial boot.  Shouldn’t be to much longer before ArkOS is out.

This is such great news, been waiting for this ever since I got my RG351V, ordered a brand new 16GB SanDisk SD card in preparation, can't wait!

Test release available via one of the links below.  Be sure to do an online update after install for the latest and greatest improvements.
Google Drive: https://drive.google.com/file/d/1AP5VFPN11LWnryvQS68Hpy3zkIn3jIpr/view?usp=sharing
Mega: https://mega.nz/file/SVoRRAzD#SJ5kS9B7nfPZQ4M_DgR0TjOozPrbdztqhDA-V5FpbBQ
FAQ: https://github.com/christianhaitian/arkos/wiki/Frequently-Asked-Questions---RG351V
Notes

For NDS (Drastic emulator) - Do not try to remap the stylus up, down, left and right buttons.  Doing so will cause the stylus movement to be erratic or not function at all.
Hotkey information: https://github.com/christianhaitian/arkos/wiki/Frequently-Asked-Questions---RG351V#q-what-are-the-global-event-keys-and-emulator-event-keys-in-ArkOS
See this FAQ link for information on how to make use of the slot 2 for your games and roms.  https://github.com/christianhaitian/arkos/wiki/Frequently-Asked-Questions---RG351V#q-how-do-i-make-use-of-the-second-sd-slot-for-roms-and-games

Is this image still the latest available? When I try to update it after flashing to the card and enabling remote services, it hangs in the middle of the update. About to test 351ELEC to see if it's just a bad SD card.

It's still available and being updated regularly.

Just got it up and running today. :) As of now all games (even GBA) are stretched to 4:3 and the aspect ratio setting in retroarch doesnt work. This is my Nr.1 complaint. If you could address this, everything else pretty much works quite well.
I've one more quarrel with the system as of now, and that is, that the displays colors in my case are just off my enough to bother me. Especially on DS games (tested Phoenix Wright) they have a brownish tint to them, and maybe look a tad desaturated. I understand that this most likely is the screen used itself, but just in case you might be able to look into it - I thought I'd mention it as well.
(Color accuracy on the first gen RK2020's was way better, btw. :) )
Oh, and thank you for allowing users to switch the led color in the middle of the device to red, its much more mute and less distracting. Thank you for the lateral thinking on this one. :)

The screen is 4:3 so no surprise with everything being 4:3.  GBA is probably the most impacted and there's not too much you can do to fix that but try different aspect ratios to see what works best for you.  You'll need to turn on Integer Scale then you can try the different Aspect Ratios depending on what you prefer.
I have no qualms or issues with the screen on my end.  The colors are very vivid and the resolution can't be beat for the options available for this chips.  Perhaps you unit has a fault with the screen as I haven't heard much of issues with the screen.
The RK2020 V1 has nothing on this unit other than being more pocketable but that's my opinion. :)

Thank you for the reply - even with integer scale on, at least in the mgba core - the core provided aspect ratio is off, 3:2 aspect ratio is off (provides a square'ish aspect ratio), and custom isnt fine grained enough you could set a default gba aspect ratio.
So - the entire aspect ratio behavior currently is off. (Normall aspect ratios also would take without having to turn on Integer scale. Also their behavior is not what would be expected in terms of actual aspect ratio behavior.
Probably a pain to fix - but please take another look if you can.
Also it might just be me, but the brownish tint seems to be fading... so maybe (fingers crossed) in my case it actually was a 'glue' layer in the screen assembly thats hardening out and getting transparent. Something is off regardless, contrast is lacking at the top end and gamma looks like it crushes darker colors (bad gamma curve?) - probably luck of the draw and the screen though.

I don't think this is possible.  Have a look at retrogamecorps video on the 351v about this.  It's what it is with this device.  To be sure. Try stock or 351elec and see if the results are any different for you there.  If not, you just need a 3:2 device for your gba emulation needs.  The rk2020 spoils you there.  :)

ArkOS Test Release 2.1 is now available
Google Drive: https://drive.google.com/file/d/1xRp8adMwWDCs_JqQJWd6sef9t5Bddzjd/view?usp=sharing
Mega: https://mega.nz/file/2Qo2kC5J#JosOCBh9CWRy4WEHVz2Cg8O5VkPoonSI-BgQiZ3dZWk
FAQ: https://github.com/christianhaitian/arkos/wiki/Frequently-Asked-Questions---RG351V
MD5: 51713AC3563FEBDAADABE09CAC141150
Important Notes

Follow the instructions from the wiki to properly loading this image: https://github.com/christianhaitian/arkos/wiki#instructions-for-loading
Don't use the original micro sd card that came with the unit.  Those are not good cards.  Use authentic Samsung or Sandisk micro sd cards for the best reliability and performance.
For NDS (Drastic emulator) - Do not try to remap the stylus up, down, left and right buttons.  Doing so will cause the stylus movement to be erratic or not function at all.
Hotkey information: https://github.com/christianhaitian/arkos/wiki/Frequently-Asked-Questions---RG351V#q-what-are-the-global-event-keys-and-emulator-event-keys-in-ArkOS
See this FAQ link for information on how to make use of the slot 2 for your games and roms.  https://github.com/christianhaitian/arkos/wiki/Frequently-Asked-Questions---RG351V#q-how-do-i-make-use-of-the-second-sd-slot-for-roms-and-games
Changelog: https://github.com/christianhaitian/arkos/blob/main/changelogs/rg351v-changelog

Make sure to Update online via the Options section for the latest fixes and features as they become available.

I don't think this is possible. Have a look at retrogamecorps video on the 351v about this. It's what it is with this device. To be sure. Try stock or 351elec and see if the results are any different for you there. If not, you just need a 3:2 device for your gba emulation needs. The rk2020 spoils you there. :)

I've now 'answers' on all my issues. :) So just for clarification: The Retrocorps video ( https://www.youtube.com/watch?v=ZzpjIdpZPRA ) shows 3:2 scaling being set correctly in retroarch on the default (emuelec based) firmware, without integer scaling having to be enabled. This solves the GBA aspect ratio issue. On ArkOS this is currently not possible (updated to the latest possible firmware), as all non-integerscaling based scaling is "broken" in the firmware. The workaround to use integer scaling with GBA games is not really workable - huge black bars on all sides and the image becomes too small.
What I perceived as a "gamma issue" actually was RGA scaling being set to enabled on a SNES game. Not recommended. It fills up the image content with extra white pixels at the edges, thereby causing a look as if the screens gamma setting was way off. Easy fix - turn RGA scaling off. (Maybe also by default for the RF351V. :) )
The brown screen tint is still an issue and sadly not imagined - the retrogamecorps shows this as well. This is probably the hardest to 'fix' as knowledge about color science, and driver/kernel level skills would be required, so nothing I really would hope for. But for the record, its not just me, its a general issue. :)
The screen also still has contrast issues (afair red/orange running out at the high end (high saturation colors)), and poor black level, and thats purely hardware - and nothing anyone could fix, short of replacing it. ;)
So in order of "severity" - retroarch scaling not "taking" in non-integerscaled mode is a big one, and not an issue in the official firmware. Probably hard to fix, but probably necessary for many people. Certainly for me. (Would be a cause for me to Switch to a different firmware longterm. GBA games are important. ;) )
Being able to counteract the brown tint would be a very nice plus - but at the same time entirely unterstandable if its beyond the scope of the project and not possible.
Also - something I noticed -- the flycast_rumble core on my RK2020 (newest model (colored buttons)) in retroarch32 performs far better than any DC emulator (and even flycast_rumble) itself on the RG351V. And actually outperforms all other DC cores, including the retrorun ones.
I've yet to test if this is due to SD card write speed, as part of the performance improvement Is due to the "Enable RTT buffer setting" in emulator, and my current main system SD card is a cheap chinese one, but if its not due to that, or additional system overhead on the RG351V, please take note of the emulation core (flycast_rumble in Retroarch32) that came bundled with the RK2020 1.6 release being far more performant, than any of the DC cores on the current release. (Same settings, everything else tested.) Didnt test replacing (swapping) cores themselves, but will do so in the future. :)

In terms of color tint - I found a way to compensate. :) Color performance on the RG351V gets better, if you turn the screen brighter, at the same time, doing that with RGA scaling enabled causes those strange shifts that look like shifts in gamma.
If you turn on RGA scaling, and enable the lcd3x filter (in the handheld folder if you aresearching in filters) the entire screen becomes darker, which allows you to push one brightness level higher (at the lower end of the brightness scale), leading to more accurate colors (greens and skintones), the "gamma shift" that RGA scaling produces is beneficial in this case. Probably the best looking image (at lower brightness levels) I got out of the device so far. There is one downside (apart from the LCD filter affecting your image - which you might like or not, but its subtle :) ) and that is raised black level.
Also make sure to put on the screen protector that ships with the device (tempered glass) imho - it lowers overall brightness by a bit (that helps black level ;) ), makes RGA scaling a little less harsh - anything that helps, really. ;)
__
Tried out the TheRA build for a while - which has non-integer scaling working (so perfect GBA image), which looked incredibly good on the RGA351Vs screen. The rest of the build is cluttered and overlayed with effects, and I switched back to ArkOS after about 3 minutes. ;) Couldn bare it.
__
flycast_rumble core, for some reason becomes less performant on 2.3 than on 1.5 - 1.6 ish builds on the device over time - so in the end there is a noticeable difference to the RK2020 on 1.6 - even accounted for SDcard speed - something reduces performance of the core on current test builds (2.3). Probably hard to trouble shoot, but I thought I'd mention it.
__
Duckstation works, probably even on 2x resolution scale on this build, It doesnt on 1.6 on the RK2020, but if you take the binary from the current RG351V build, and swap it with the on on the RK2020, that now can run both duckstation, and flycast_rumble at a speed where Soul Calibur becomes playable and doesnt suffer from frequent stutters. :) (Yay!)
__
Integer scaling "works" on GBA games if you use custom resolution, and then set resolutionscaling for height and with to 3x - then you get a "zoomed in" 3:2 image, with the left and right parts cut off (to 3:2). But the aspect ratio is correct. (This is probably what christianhaitian referred to in the first place - when guiding me towards integer scaling enabled).
__
Please look into getting non-integer based scaling to work, for the sake of being able to play GBA games at their optimal aspect ratio, wile filling the screen. A year from now - doesnt matter, but please give the implementation another shot, and therefore help to make this a true "alrounder". :)

Very odd as TheRA and ArkOS use the same exact version of Retroarch.  Same binaries as we compiled those and shared them together both slaminger and I.

Very odd as TheRA and ArkOS use the same exact version of Retroarch. Same binaries as we compiled those and shared them together both slaminger and I.

The Retroarch build TheRA is using is missing RGA scaling, and afair, wasnt 1.9.1 either (could be wrong on the second part). So there are differences.

it's 1.9.1.  Like I said, we're both using the same exact binaries.  Not sure on the RGA scaling issue.  I checked in the discord forum I normally hang out in, and no one seems bothered by GBA scaling in ArkOS so far.  They say it's fine and some are very picky about ratios so I don't know on this just yet.  As for flycast32_rumble core, for best dreamcast performance, you have to use the retrorun32 emulator.  The flycast32_rumble core is the exact same core used among all of the ArkOS distributions.  Exactly build.  The only difference is that this device is 640x480 natively while the others are 320x480 which may cause some slower performance on the V since it's the exact same chipset among all devices.  I compensated for that by setting the resolution to default to 320x240 for retrorun emulator use in dreamcast.  To enable 640x480, you have to hold down the a button while launching a dreamcast game using retrorun.

I also noticed when RGA scaling is turned off, I can change aspect ratios for GBA without turning on integer scale.  Perhaps this is what you're looking for?

I just released a new update today with Retroarch 1.9.2 with RGA scaling no longer available as an option.  It was causing too much confusion and didn't work right for the V anyway.  Perhaps this will solve the GBA scaling issues you had.

Hi, just checked, and yep it was RGA scaling turned ON that was causing the issues @notimp was experiencing.  I was able to replicate the issue and resolve it by turning off RGA scaling and then adjusting the aspect ratio.  In general, I would recommend RGA scaling off across the board -- it's not needed with 640x480 4:3 displays, as far as I can tell.  There is still some pixel distortion when scaling non-integer up to the full display on most systems, but RGA scaling causes a net larger distortion than that natural distortion (which can be fixed with shaders that have better clarity anyway).

I just woke up, finished a new sdcard setup (exfat partition got corrupted and I had to anyhow. :) ), noticed there was a 2.4 build, installed, and had the non-integer scaling working. :)
Now I miss RGA scaling.. ;) But retroarch guys took it out, so slave to the new default I guess.. - oh well... ;)
Thank you very much for looking into it, and yay for aspect ratio correct non integer gba game scaling. :)

Quick tip for RG351V owners, while trying to replicate the tip above (lcd3x filter) without RGA scaling being available on the latest builds, I came across the following scanline settings, that look decent.
Enable scanlines1280x720.cfg in the nesguy folder, and set overlay opacity to 0.30 (or 0.25 but 0.30 produces a darker image, which is beneficial to perceived black level), all under the On-Screen-Overlay setting.
Setting this as an overlay has the benefit of causing no performance penalty in game.
The goal still is to darken the overall image on screen, so you can use a one level higher brightness setting, to compensate for the horrible color accuracy of the screen on lower brightness settings.
If some of you alway notice a "beige" tint on screen - this combats it effectively. :) Of course you now have a scanline 'look' on your games, but even for people who dont like that, thats a small price to pay to get an overall better looking image out of the RG351V on lower brightness levels.
I've anyone knows where to get an overlay .cfg with scanlines for the native 640x480 resolution, I'd love to try that as well - but as is, retroarch only comes with scanline overlays for 1280x720 out of the box. There is no pixel distortion (apart from non integer scaling) but still, Id love to try this with a 640x480 scanline overlay as well.
Tested with SNES and Genesis games. Looks good. (Decent enough.) ;) From memory I'd say that lcd3x with RGA scaling lead to a more pleasant image but since RGA scaling is now gone as an option, this is a decent way to achieve a similar result in current builds. (Also the lcd3x filter had the downside, that if used with 480p (native) games, it sometimes would 'bug out' and only cause a very noticable green tint on screen. But for SNES or Genesis games, the combination with RGA scaling on, produced a nice result)

N64 seemingly isn't being picked up, if only .zip files are in the n64 folder. Please fix, if you can reproduce it.

.zip are not supported by the mupen64plus standalone emulator which is the best emulator to use for N64 games.  That's why .zip does not show up.  Just use the supported extensions as shown in the link here: https://github.com/christianhaitian/arkos/wiki/ArkOS-Emulators-and-Ports-information#nintendo-64

You can stop implementing/maintaining retrorun32 - imho.. ;)
PSA: MUCH(!) better DC and GBA performance using the OpenAL audio core
Make sure to save the config file before exiting.
FFVI (MGBA) works with almost no audio stutter during dialogs
SoulCalibur (flycast-rumble, using retroarch32) runs much faster with almost no audio stutter (Fast MicroSD (20MB write), VGA, region ntsc, Render to texture: on)
Tested on ArkOS.
With the default alsathread audio core, there are frequent very noticeable audio stutters throughout both games, any much more slowdowns during SoulCalibur gameplay.
Imho try switching to OpenAL for a few other systems as well, and see what results it produces. :)
On a different note, the current 351ELEC is based on Retroarch 1.9.2 but still has RGA scaling as an option, and with it enabled (main menu), still manages 3:2 scaling (non-integer) for GBA games.
This makes the resulting image on GBA games (without filters) much more pleasing. ("Health Warning Screen" Font appears to not have any double lines on very fine grained (one pixel) font.)
But you know I'm sticking with ArkOS. ;)

PPSSPP only outputs wrong aspect ratios. The image will be stretched on the y-achsis, no matter the setting.
See: https://i.imgur.com/Rj9U6Lo.jpeg
Please fix as soon as possible, this is most problematic high tier issue I've come across so far.

I couldn’t fix the stock one anyway as it came from the stock firmware.  My guess is it was stretched on stock as well.

Small request, subjective. When starting up the RG351V the rumble pulse lasts for about 3 seconds and is quite abrasive. 351ELEC pulses only for a second or so and after the 50th time or so, you start to appreciate it, imho. Just something about how you feel, turning the system on over time. As it is entirely subjective its hard for me to argue for it - but subjectively this being less abrasive 'feels' better over time. :)

Have you flashed the stable image or just upgraded from the test release?  The stable image buzzes a little less longer.
Sent from my iPad

On May 29, 2021, at 6:52 AM, notimp @.***> wrote:

Small request, subjective. When starting up the RG351V the rumble pulse lasts for about 3 seconds and is quite abrasive. 351ELEC pulses only for a second or so and after the 50th time or so, you start to appreciate it, imho. Just something about how you feel, turning the system on over time. As it is entirely subjective its hard for me to argue for it - but subjectively this being less abrasive 'feels' better over time. :)
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or unsubscribe.

Have you flashed the stable image or just upgraded from the test release? The stable image buzzes a little less longer.

Upgraded - thank you for the information. Will try. We just got another request for 'less annoing buzz please" on reddit (https://old.reddit.com/r/rg351v/comments/nqggqo/i_love_this_device_so_much_but_here_are_my/), not sure if the person is using ArkOS though.
Just saying.. ;)


google search "kong mesh rbac" returns 404 link

Where is the problem?
Not sure if this is the right place to report the issue, but the first link in google response for "kong mesh rbac" returns 404.

Link: https://docs.konghq.com/mesh/latest/production/secure-deployment/
What is your suggestion?
The actual link should be https://docs.konghq.com/mesh/latest/production/secure-deployment/rbac/
Code of Conduct and Community Expectations

[X] I agree to follow this project's Code of Conduct
[X] I agree to abide by the Community Expectations

@lobkovilya I believe that we fixed this, is this still an issue?

Yes, it's fixed now, thanks 👍


Feat: Implement the function in the ExchangeRouter contract.

Description
Implement the send_wnt function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
fn send_wnt(ref self: TContractState, receiver: ContractAddress, amount: u128);

Implement the send_wnt function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
fn send_tokens(ref self: TContractState, token: ContractAddress, receiver: ContractAddress, amount: u128)

Implement the create_deposit function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
fn create_deposit(ref self: TContractState, params: CreateDepositParams) -> felt252

Implement the cancel_deposit function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
fn cancel_deposit(ref self: TContractState, key: felt252);

Implement the create_withdrawal function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
fn create_withdrawal(ref self: TContractState, params: CreateWithdrawalParams) -> felt252;

Implement the cancel_withdrawal function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
fn cancel_withdrawal(ref self: TContractState, key: felt252);

Implement the create_order function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
fn create_order(ref self: TContractState, params: CreateOrderParams) -> felt252;

Implement the set_saved_callback_contract function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
fn set_saved_callback_contract(
        ref self: TContractState, market: ContractAddress, callback_contract: ContractAddress
    );

Implement the simulate_execute_deposit function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
    fn simulate_execute_deposit(
        ref self: TContractState, key: felt252, simulated_oracle_params: SimulatePricesParams
    );

Implement the simulate_execute_withdrawal function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
    fn simulate_execute_withdrawal(
        ref self: TContractState, key: felt252, simulated_oracle_params: SimulatePricesParams
    );

Implement the simulate_execute_order function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
    fn simulate_execute_order(
        ref self: TContractState, key: felt252, simulated_oracle_params: SimulatePricesParams
    );

Implement the update_order function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
    fn update_order(
        ref self: TContractState,
        key: felt252,
        size_delta_usd: u128,
        acceptable_price: u128,
        trigger_price: u128,
        min_output_amout: u128
    );

Implement the cancel_order function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
    fn cancel_order(ref self: TContractState, key: felt252);

Implement the claim_funding_fees function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
fn claim_funding_fees(
        ref self: TContractState,
        markets: Array<ContractAddress>,
        tokens: Array<ContractAddress>,
        receiver: ContractAddress
    ) -> Array<u128>;

Implement the claim_collateral function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
    fn claim_collateral(
        ref self: TContractState,
        markets: Array<ContractAddress>,
        tokens: Array<ContractAddress>,
        time_keys: Array<u128>,
        receiver: ContractAddress
    ) -> Array<u128>;

Implement the claim_affiliate_rewards function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
    fn claim_affiliate_rewards(
        ref self: TContractState,
        markets: Array<ContractAddress>,
        tokens: Array<ContractAddress>,
        receiver: ContractAddress
    ) -> Array<u128>;

Implement the set_ui_fee_factor function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
    fn set_ui_fee_factor(ref self: TContractState, ui_fee_factor: u128);

Implement the claim_ui_fees function in the ExchangeRouter contract.
File: src/router/exchange_router.cairo.
Function signature:
    fn claim_ui_fees(
        ref self: TContractState,
        markets: Array<ContractAddress>,
        tokens: Array<ContractAddress>,
        receiver: ContractAddress
    ) -> Array<u128>;

Acceptance criteria

Function is implemented and well commented.
Associated unit tests (nominal and failure) are implemented and passing.

Resources

Functions in GMX repo

Can i try picking this up ?

Assigned !

Hi @makluganteng ! How is it going ?


Add json encoder/decoder as keyword arguments in Redis PUBSUB methods

What do these changes do?
Provides the ability to specify json loads/dumps functions.
Are there changes in behavior for the user?
The change only adds additional functionality.
Related issue number
No
Checklist

[x] I think the code is well written
[ ] Unit tests for the changes exist
[ ] Documentation reflects the changes
[x] If you provide code modification, please add yourself to CONTRIBUTORS.txt

The format is <Name> <Surname>.
Please keep alphabetical order, the file is sorted by names.

[ ] Add a new news fragment into the CHANGES/ folder

name it <issue_id>.<type> (e.g. 588.bugfix)
if you don't have an issue_id change it to the pr id after creating the PR
ensure type is one of the following:

.feature: Signifying a new feature.
.bugfix: Signifying a bug fix.
.doc: Signifying a documentation improvement.
.removal: Signifying a deprecation or removal of public API.
.misc: A ticket has been closed, but it is not of interest to users.

Make sure to use full sentences with correct case and punctuation, for example:
Fix issue with non-ascii contents in doctest text files.

@7ckingBest -
Thanks for the contribution here! Maintenance for this library has recently resumed.
Would you be willing to rebase off of latest master and resolve your merge conflicts? After which you may be asked to sign a CLA, then I think we'll be good to merge this in!

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


ie8 purge() maybe have a bug when detached more than one Element?

function purge() {
  for (var
    node,
    i = 0,
    length = targets.length;
    i < length; i++
  ) {
    node = targets[i];
    if (!documentElement.contains(node)) {
      targets.splice(i, 1);
      verifyAndSetupAndAction(node, DETACHED);
    }
  }
}

if targets.length bigger than 1, when  targets.splice(i, 1);  targets.length is change,  then node = targets[i] will  be undefined ,   so documentElement.contains(node)  will throw a error

fix by:
function purge() {
  for (var
    node,
    restTargets = [],
    i = 0,
    length = targets.length;
    i < length; i++
  ) {
    node = targets[i];
    if (!documentElement.contains(node)) {
      verifyAndSetupAndAction(node, DETACHED);
    } else {
      restTargets.push(node);
    }
  }
  targets = restTargets;
}

or better?

There are tests in place. Does this pas all current tests? Can you write down an example where you explain what you are trying to fix and how your code fixes that?
I am having hard time following your issues ...

Also, and once again, IE8 is experimental. I'm not sure anything is going to be fixed 'cause this repository is already widely deployed in production. Not sure I'm going to risks these kind of untested changes to fix IE8, thanks for your understanding.

Thank u, IE8 is need to consider in China  ;(
i will fill my test case for this issue tomorrow

https://github.com/WebReflection/document-register-element/pull/46/files


is realtime event listening for firebase_admin.firestore possible ?

Environment

Operating System version: Ubuntu 16 LTS
Firebase SDK version:

firebase-admin=2.13.0
google-cloud-firestore=0.30.0

Library version: ? not sure
Firebase Product: firestore

Description:
Will listening for realtime events on collections be implemented for firestore python ?
Some documentation indicates that it's possible, yet other documentation and my own testing indicates else wise.
Details
The documentation for CollectionReference indicates that on_snapshot can be used to register event listeners on collections.
However, this firestore tutorial is saying that "Note: Realtime listeners are not supported in Python and PHP.".   So there are two contradictory sources
Furthermore, in my own tests I'm getting that on_snapshot is not an attribute of an CollectionReference, suggestion that this feature is not possible.
Can you confirm whether realtime listening in python firestore admin sdk is possible or not ?
Relevant Code:
The documentation for on_snapshot says this code should work
    from google.cloud import firestore
    db = firestore.Client()
    collection_ref = db.collection(u'users')
    def on_snapshot(collection_snapshot):
        for doc in collection_snapshot.documents:
            print(u'{} => {}'.format(doc.id, doc.to_dict()))
     collection_watch = collection_ref.on_snapshot(on_snapshot)

However it's giving me an error
AttributeError: 'CollectionReference' object has no attribute 'on_snapshot'

Duplicate of https://stackoverflow.com/questions/53511983/is-realtime-event-listening-for-firebase-admin-firestore-possible


Typo "Azure data factory"→"Azure Data Factory"

https://docs.microsoft.com/en-us/azure/data-factory/v1/data-factory-build-your-first-pipeline-using-rest-api
#PingMSFTDocs

@hyoshioka0128 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

The updated text appears to be referring generically to a data factory in Azure ("an ... data factory," "your ... data factory"), so it's correct as-is; we'll close the PR.
Internal style guidance is to refer to generic instances in lowercase, data factory, as opposed to referring to the Azure service name, Azure Data Factory.


when does result.x !== null && result.y !== null

result is instantiated on line 148 and checked on 156. in what case would this condition ever return true?

-> https://github.com/Turfjs/turf/issues/354


how to restart the sensor after accelerationObservable.stop()

@VincentSuperper Did you solve your issue? Would be great if you could add a line to the readme 👍  (sorry that I didn't respond quicker)

i just recreate it.


Prometheus exporter errors in production

Hey,
Does anybody know, why the following message pops up every 0.5 seconds in the logs?
Prometheus Exporter, failed to send message Cannot assign requested address - connect(2) for "localhost" port 9394
Best,
Tobias

I think this should be fixed by #364.

Fixed by #364.


Fixed index metrics serialization and tests

Fixed index metrics serialization and tests
Fixes #37426

API change check
API changes are not detected in this pull request.

/azp run java - cosmos - tests

/check-enforcer override


VScode cannot use command+click to jump to Vue3's custom component in MacOS

Describe the bug
For the same project, I can use ctrl+click to jump to the corresponding .vue file through the custom component name in Windows 10. However, I cannot use command+click to jump in MacOS. I have tried reinstalling VScode and Open other projects and the problem still exists.I use unplugin-vue-components to automatically import components.components.d.ts is generated.The project can run normally.
The display type is displayed when running under the windows version. However, the type of the custom component in mac displays any.TS or JS function methods and native tags such as div can be jumped, but custom components cannot be jumped, such as el-button.
Windows:

MacOS:

what i expected：
Use command + click on MacOS to jump to the .vue file of the corresponding custom component
System Info
- VScode version：1.84.0(arm)
- system version：MacOS Ventura 13.6
- unplugin-vue-components: 0.25.1
- Vue Language Features (Volar) 1.2.0
- TypeScript Vue Plugin (Volar) 1.8.22

Used Package Manager
pnpm

The problem is solved. It is a problem with the Vue Language Features (Volar) plug-in version. Version 1.8.22 has a BUG and cannot display the type of custom components. Just reduce the version to 1.8.11.


Feature Request:  Update window

I would really appreciate some sort of way to prevent a container from being upgraded during a specific window or to specifically allow only during a specific window.  I understand that this usually relies on docker rolling updates but I suspect I would still see downtime and I'd like to ensure that it happens when no one is using the service.

For the future, my plan is to use the upcoming job support in docker in conjunction with swarm-cronjob in order to run shepherd on a cron schedule. That will allow to run shepherd at exact times, which should fulfill your wish.

Thanks!


Is this package still maintained?

Hey,
I wonder if this package is still maintained, since the last update is from the 31.01.2017 but the status on pypi is still at 5 - Production/Stable.
I would appreciate your answer.

Sure, if you'd like to submit a pull request I'd take a look at it.

@mattfox Could you take a look at this one https://github.com/kti/python-netfilterqueue/pull/32 ?
It's been open for over a year and it's pretty trivial.

Yeah, my apologies. I will comment on the PR.


Fix ping service client

This PR fixes two issues:

Alluxio client can use ping service to query the health of Alluxio services. On client the authentication was incorrectly enabled by default causing it to fail to start when there is no client credentials given.

Another problem is that the client fails to retry because the gRPC UnavailableException is not converted to the AlluxioStatusException, effectively render the retry logic invalid.

@ggezer PTAL. Thanks!

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3491/
Test PASSed.

LGTM. Thanks for fixing it.

@gpang PTAL. Thanks!

alluxio-bot, merge this please


Fixed broken link under learn section

New link for creating new library is updated.

The committed file is generated from https://github.com/processing/p5.js-website/blob/6365eb301ca0e5af7d7de32185c6d580b957323a/src/templates/pages/learn/index.hbs#L109
@lmccart Is it still necessary for the deployment of the website to retain the contents of the dist folder? The reference files are now also meant to be copied into the src folder and when I test is a couple months ago, the whole of the dist folder's contents are already generated with the build step from scratch.

I suggest we create a new issue detailing that git-pull.php would be nice to rewrite to enable exlusion of the dist folder. I haven't looked at its source (where is it?) but did something similar in p5-types - it's not very difficult.

this has been resolved, thanks!


[automation] Update go release version 1.19.2

What
Bump go release version with the latest release.
Further details
See changelog for 1.19.2
Automatically generated from https://apm-ci.elastic.co/job/apm-shared/job/bump-go-release-version-pipeline/

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-10-10T04:06:04.539+0000

Duration: 6 min 37 sec

Test stats :test_tube:

Test
Results

Failed
0

Passed
19

Skipped
0

Total
19

:robot: GitHub comments
Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.


Chat application using Vue-Clerk failed in offline environment.

Hey, I am building a chat application in PWA.
I have a situation that user may check the message in offline environment.
When I switch the network off, the clerkjs couldn't initialize properly.

I came out a feature that user can set a fallback option, when he/she try to get access to the application with previous clerkjs data.
This feature may may use localStorage or IndexedDB to save the state in browser, and keep watching the network status.
Once the network is online, updating the state immediately.

Closing this for now. Looks more of a Clerk issue (which I dont think is an issue)


minor: Extract tests for EXTRACT AND date_part to their own file

Which issue does this PR close?

Related to https://github.com/apache/datafusion/pull/13372

Rationale for this change
I want to add additional test coverage as part of helping to avoid regressions with @jayzhan211  et al on  https://github.com/apache/datafusion/pull/13372
It turns out that the existing date_part tests are 800+ lines in the already massive  expr.slt file, which makes it hard / akward to work with
What changes are included in this PR?

This PR extracts tests for date_part and extract (which is syntactic sugar on date_part) into their own .slt files
Note it just moves the tests, it doesn't change or add anything (I will make a follow on PR to add new tests)

Are these changes tested?
By CI
I also tested manually to verify this new file is run:
(venv) andrewlamb@Andrews-MacBook-Pro-2:~/Software/datafusion$ cargo test --test sqllogictests | grep date_part
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.15s
     Running bin/sqllogictests.rs (target/debug/deps/sqllogictests-dee2dcb75b23ce67)
Executed "expr/date_part.slt". Took 161.646958ms

Are there any user-facing changes?
No, this is an internal only change

Thank you for the quick review @jayzhan211


Import Tumblr Data

http://import.jekyllrb.com/docs/tumblr/
$ ruby -rubygems -e 'require "jekyll-import";
JekyllImport::Importers::Tumblr.run({
"url"            => "http://myblog.tumblr.com",
"format"         => "html", # or "md"
"grab_images"    => false,  # whether to download images as well.
"add_highlights" => false,  # whether to wrap code blocks (indented 4 spaces) in a Liquid "highlight" tag
"rewrite_urls"   => false   # whether to write pages that redirect from the old Tumblr paths to the new Jekyll paths
})'

Issues with external resources such as links and images.

Posts are having issues displaying properly?


CASMTRIAGE-6188: add retries for tag retrieval

Description
Occasionally the latest tag retrieval for images may return an empty string, this PR adds retries for up to 60 seconds.

Checklist

[ ] If I added any command snippets, the steps they belong to follow the prompt conventions (see example).
[ ] If I added a new directory, I also updated .github/CODEOWNERS with the corresponding team in Cray-HPE.
[x] My commits or Pull-Request Title contain my JIRA information, or I do not have a JIRA.

/backport release/1.4, release/1.6


ES5 / ES6 dynamic require in index.js doesn't work with browserify

I end up with: Uncaught Error: Cannot find module './es5'
If I change the output from:
module.exports = require(useES6 ? './es6' :  './es5');
to:
module.exports = require('./es5');
Then everything works fine.

I have the same issue, please tell where exactly you've changed that? I'm new in typescript

Please, refer to this section in the documentation.
When using with browserify, webpack etc, you must import the library as:
import {AutoWired, Inject} from "typescript-ioc/es5";

or, if you are using es6 as output:
import {AutoWired, Inject} from "typescript-ioc/es6";


Jekyll inclusion of non-Jekyll yaml files

Mandy discovered an issue where the site rebuild was failing; thanks to David and Graham's analysis, it seems to be due to Jekyll trying to parse and include .yml files across the entire repository, whether those file are meant for Jekyll or not.
(In this case, those under open-metadata-resources/open-metadata-deployment/sample-data/coco-pharmaceuticals, which are actually for Ansible use.)
Currently investigating the best way to ensure these are excluded so the site builds again.

Another option (from: https://github.com/jekyll/jekyll/issues/948) sounds like it would be to explicitly add all yaml files to be excluded:
exclude:
  - "*.yml"


Can not use custom reporter for mocha

I am using command nyc --all node_modules/mocha/bin/_mocha -- test --reporter mocha-allure-reporter from command line. Please let me know if I am doing anything wrong.
Command works when I remove --reporter mocha-allure-reporter which works with mocha independently.

I suggest using a mocha.opts file.
Also, with nyc, you don't need to use the _mocha executable which exposes some coverage information, the plain mocha executable will be fine. See: https://github.com/saadtazi/until-promise/issues/3
There's also a discussion on this in the gitter channel.

@pjangam I think you can do something as simple as this:
nyc --all node_modules/mocha/bin/_mocha -- test --reporter mocha-allure-reporter
Let me know if you run into any issues.

Thank you @JaKXz and @bcoe for inputs. I have started using  mocha.opts file for now. I will surely see difference in mocha and _mocha.
@bcoe What you replied with is exactly my command and I get

cannot resolve path (or pattern) '--reporter' error

on it.
See error stack in attachment.
errorStack.txt

closing this as opts file solved my issue.


Allow searching on field choices display names

If a field has a set of choices this PR makes it possible to search using the display name rather than the stored database value.

Thanks!  I'll get this in sometime tonight probably.


a small need，can lvgl rotate all things 90 degree?

Hi，
I change the display degree, but can't change the touch erea... haha

I solved the problem,swap the input x and y.

Great!
I general you need to provide the coordinates according to the display's orientation.


Size of matrix mismatch error when using pre-trained model(transformer.wmt19.de-en)

Hello
I'm trying to finetune the provided pretrained model transformer.wmt19.de-en from the paper(Facebook FAIR’s WMT19 News Translation Task Submission).
However, I cannot find the correct architecture for this pre-trained model.
According to the paper, seems like 'transformer_vaswani_wmt_en_de_big' is used but it doesn't fit with the pre-trained model.
Furthermore, I tried out all sensible other architectures, such as transformer, wmt_en_de_big but it didn't work either.
I preprocessed the data usin this command ;
fairseq-preprocess --source-lang de --target-lang en \     --trainpref $TEXT/train   \     --validpref $TEXT/valid   \     --testpref $TEXT/test     \     --destdir data-bin/wmt19.tokenized.de-en \     --workers 20  \     --joined-dictionary --srcdict ../models/wmt19.de-en.joined-dict.ensemble/dict.de.txt  \
And afterwards finetune like this ;
fairseq-train \     data-bin/wmt19.tokenized.de-en \     --restore-file ../models/wmt19.de-en.joined-dict.ensemble/model1.pt \     --save-dir checkpoints/finetune_wmt_model1 \     --arch transformer_vaswani_wmt_en_de_big  --share-decoder-input-output-embed \     --optimizer adam --adam-betas '(0.9, 0.98)' --clip-norm 0.0 \     --lr 5e-4 --lr-scheduler inverse_sqrt --warmup-updates 4000 \     --dropout 0.3 --weight-decay 0.0001 \     --criterion label_smoothed_cross_entropy --label-smoothing 0.1 \     --eval-bleu \     --eval-bleu-args '{"beam": 5, "max_len_a": 1.2, "max_len_b": 10}' \     --eval-bleu-detok moses \     --eval-bleu-remove-bpe \     --eval-bleu-print-samples \     --best-checkpoint-metric bleu --maximize-best-checkpoint-metric   \     --max-sentences 100
Then it gives me this error ;

Traceback (most recent call last):
File "/data/s3475743/myver_fairseq/fairseq/fairseq/trainer.py", line 256, in load_checkpoint
self.get_model().load_state_dict(
File "/data/s3475743/myver_fairseq/fairseq/fairseq/models/fairseq_model.py", line 93, in load_state_dict
return super().load_state_dict(new_state_dict, strict)
File "/data/s3475743/myver_fairseq/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 846, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for TransformerModel:
size mismatch for encoder.layers.0.fc1.weight: copying a param with shape torch.Size([8192, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 1024]).
size mismatch for encoder.layers.0.fc1.bias: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current model is torch.Size([4096]).
size mismatch for encoder.layers.0.fc2.weight: copying a param with shape torch.Size([1024, 8192]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
size mismatch for encoder.layers.1.fc1.weight: copying a param with shape torch.Size([8192, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 1024]).
size mismatch for encoder.layers.1.fc1.bias: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current model is torch.Size([4096]).
size mismatch for encoder.layers.1.fc2.weight: copying a param with shape torch.Size([1024, 8192]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
size mismatch for encoder.layers.2.fc1.weight: copying a param with shape torch.Size([8192, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 1024]).
size mismatch for encoder.layers.2.fc1.bias: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current model is torch.Size([4096]).
size mismatch for encoder.layers.2.fc2.weight: copying a param with shape torch.Size([1024, 8192]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
size mismatch for encoder.layers.3.fc1.weight: copying a param with shape torch.Size([8192, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 1024]).
size mismatch for encoder.layers.3.fc1.bias: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current model is torch.Size([4096]).
size mismatch for encoder.layers.3.fc2.weight: copying a param with shape torch.Size([1024, 8192]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
size mismatch for encoder.layers.4.fc1.weight: copying a param with shape torch.Size([8192, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 1024]).
size mismatch for encoder.layers.4.fc1.bias: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current model is torch.Size([4096]).
size mismatch for encoder.layers.4.fc2.weight: copying a param with shape torch.Size([1024, 8192]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).
size mismatch for encoder.layers.5.fc1.weight: copying a param with shape torch.Size([8192, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 1024]).
size mismatch for encoder.layers.5.fc1.bias: copying a param with shape torch.Size([8192]) from checkpoint, the shape in current model is torch.Size([4096]).
size mismatch for encoder.layers.5.fc2.weight: copying a param with shape torch.Size([1024, 8192]) from checkpoint, the shape in current model is torch.Size([1024, 4096]).

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/data/s3475743/myver_fairseq/venv/bin/fairseq-train", line 33, in
sys.exit(load_entry_point('fairseq', 'console_scripts', 'fairseq-train')())
File "/data/s3475743/myver_fairseq/fairseq/fairseq_cli/train.py", line 352, in cli_main
distributed_utils.call_main(args, main)
File "/data/s3475743/myver_fairseq/fairseq/fairseq/distributed_utils.py", line 189, in call_main
main(args, **kwargs)
File "/data/s3475743/myver_fairseq/fairseq/fairseq_cli/train.py", line 106, in main
extra_state, epoch_itr = checkpoint_utils.load_checkpoint(args, trainer)
File "/data/s3475743/myver_fairseq/fairseq/fairseq/checkpoint_utils.py", line 134, in load_checkpoint
extra_state = trainer.load_checkpoint(
File "/data/s3475743/myver_fairseq/fairseq/fairseq/trainer.py", line 264, in load_checkpoint
raise Exception(
Exception: Cannot load model parameters from checkpoint ../models/wmt19.de-en.joined-dict.ensemble/model1.pt; please ensure that the architectures match.

Maybe the architecture was changed after saving the pretrained model or am I just doing things plain wrong?
I hope you can help me figure out how to load the pretrained models when finetuning because I kind of ran out of ideas whats going wrong.

You can determine the arguments/architecture by loading the model checkpoint and checking the 'args' attribute:
>>> import torch
>>> model = torch.load('wmt19.en-de.joined-dict.ensemble/model1.pt')
>>> model['args']
Namespace(adam_betas='(0.9, 0.98)', adam_eps=1e-08, adaptive_input=False, adaptive_softmax_cutoff=None, adaptive_softmax_dropout=0, arch='transformer_wmt_en_de_big', attention_dropout=0.1, bucket_cap_mb=25, clip_norm=0.0, cpu=False, criterion='label_smoothed_cross_entropy', data=['/private/home/edunov/wmt19/data/old/ende', '/private/home/edunov/wmt19/data/old/ende', '/private/home/edunov/wmt19/data/finetune/nc'], ddp_backend='c10d', decoder_attention_heads=16, decoder_embed_dim=1024, decoder_embed_path=None, decoder_ffn_embed_dim=4096, decoder_input_dim=1024, decoder_layers=6, decoder_learned_pos=False, decoder_normalize_before=False, decoder_output_dim=1024, device_id=0, distributed_backend='nccl', distributed_init_method='tcp://localhost:17406', distributed_port=-1, distributed_rank=0, distributed_world_size=2, dropout=0.2, encoder_attention_heads=16, encoder_embed_dim=1024, encoder_embed_path=None, encoder_ffn_embed_dim=8192, encoder_layers=6, encoder_learned_pos=False, encoder_normalize_before=False, extra_data='', fix_batches_to_gpus=False, fp16=True, fp16_init_scale=128, fp16_scale_tolerance=0.0, fp16_scale_window=None, keep_interval_updates=-1, keep_last_epochs=-1, label_smoothing=0.1, lazy_load=False, left_pad_source=True, left_pad_target=False, log_format='simple', log_interval=100, lr=[0.0007], lr_scheduler='inverse_sqrt', lr_shrink=0.1, max_epoch=0, max_sentences=None, max_sentences_valid=None, max_source_positions=1024, max_target_positions=1024, max_tokens=3584, max_update=201800, memory_efficient_fp16=False, min_loss_scale=0.0001, min_lr=1e-09, momentum=0.99, no_epoch_checkpoints=False, no_progress_bar=True, no_save=False, no_token_positional_embeddings=False, num_workers=0, optimizer='adam', optimizer_overrides='{}', raw_text=False, relu_dropout=0.0, reset_lr_scheduler=False, reset_optimizer=False, restore_file='checkpoint_last.pt', save_dir='/checkpoint/edunov/20190403/wmt19en2de.btsample5.ffn8192.transformer_wmt_en_de_big_bsz3584_lr0.0007_dr0.2_size_updates200000_seed20_lbsm0.1_size_sa1_upsample2//finetune1', save_interval=1, save_interval_updates=200, seed=2, sentence_avg=False, share_all_embeddings=True, share_decoder_input_output_embed=True, skip_invalid_size_inputs_valid_test=False, source_lang='en', target_lang='de', task='translation', tensorboard_logdir='', threshold_loss_scale=None, train_subset='train', update_freq=[1], upsample_primary=1, user_dir=None, valid_subset='valid', validate_interval=1, warmup_init_lr=1e-07, warmup_updates=4000, weight_decay=0.0)

While it started with transformer_vaswani_wmt_en_de_big there are some customizations to other parameters, e.g., --encoder-ffn-embed-dim=8192.


HWKAPM-767 | update version of all zipkin instrumentation libraries

zipkin isntrumentations update
framework dependencies update
Resteasy automatically decodes gzip encoded data therefore it is not
necessary to decode it manually. (Before it produced exception)

Should be reviewed and merged after this: https://github.com/smoketurner/dropwizard-zipkin/pull/18 is merged otherwise it would not be possible to start the server after the environment.
Review notes:
Build locally all touched images and run polyglot example.

@objectiser could you please review?

retest this please

retest this please

rebased. itest  should pass. I have added new test for gzipped spans.

@pavolloffay LGTM :+1:


Initial import of work in progress on the Common Terms Registry

This commit includes all of the code for the Common Terms Registry prototype, including the code required to import to the data, various maintenance scripts, and the user interface.

I am planning to close this one out and submit smaller pull requests for each of the remaining unique components of the PTD.  You will have seen most of the common code in the requests for the gpii.express and other common modules.


refactor: deprecate scrape method

Description
Following the CODEOWNERS the decision to deprecate scrape, this PR removes the unused method. A separate application layer handles scraping to manage the downloads for researchers. Please refer to the discussion thread regarding the metadata format the CLEAN project uses to archive and analyze police misconduct assets.
Summary of Changes

Removes def scrape methods in the existing codebase
Deletes unused imports
Updates docstrings

Related Issues

https://github.com/biglocalnews/clean-scraper/discussions/58

Tagging @stucka and @naumansharifwork for awareness regarding currently open pull requests. The scrape method was a helpful first step to a more robust pipeline in a separate application layer. Feel free to reach out in the GitHub discussion thread about metadata if there are concerns

tagging @zstumgoren for awareness 🥳


[SMALLFIX] Replace anonymous type with lambda

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/2532/
Test PASSed.

LGTM

alluxio-bot, merge this please


Ticking Player Error

I do not know if this is caused by Xaero, or not, but once I enabled Tetra-Pak, this keeps happening.
https://pastebin.com/DkUs7sF0

This seems to be an issue with the UtilBooster mixin I use to make the bubbler module work. Seems that it doesn't take kindly to Supplementaries not being installed. Will fix soon.

Should be fixed in 2455a32. I'll leave this issue open until you get a chance to test with a new version on CF.

Problem has been fixed. Thank you.


How to use HausdorffLoss?

It's a 2D, 2 class segmentation question, the HausdorffLoss input is what? I input the predict and target is wrong, self.idc = kwargs["idc"], how can use it

Closing as this is a duplicate to #54.


[FEAT]Calculate And Execute the corresponding actions

WHAT
I don't know if it is doable that calculate how many times the buttons has been pushed, and if it is doable.
Is it possible to execute the corresponding actions based on the number of times?
like:
name Daily Quote
type Text
times [1-n]
loop True
action: [1: action1;2: action2;3: action3;4: action4.....n: actionN;]

WHY I NEED AND HOW TO USE IT
It will be useful when I want to create different files in different day.
Or for building a text game.

So I'm getting closer with the new swap button. Check it out and let me know what you think!

@shabegom I will try it later~Thanks for your hard job.

@shabegom this issue can be closed. Thank you for implementing this interesting idea


Fix deprecations from header refactor

The header refactor in #4415 was done with a temporary "v2" package.  We need to clear out references to the old.
These are the files with warnings.  For new contibutors, headers will be the easiest.  There are patterns to follow in #4533 (recurring) and #4534 (singleton).
All this work needs to happen on series/0.22.  We'll be accepting volunteers at the LSUG event on Feburary 27 to kick this off.

[x] Accept-Charset.scala 3 core/src/main/scala/org/http4s/headers/ (Josef)
[x] Accept-Language.scala 3 core/src/main/scala/org/http4s/headers/ (Piotr)
[x] Accept-Patch.scala 3 core/src/main/scala/org/http4s/headers/ (TP)
[ ] Accept-Ranges.scala 3 core/src/main/scala/org/http4s/headers/ (Ross)
[x] Accept.scala 3 core/src/main/scala/org/http4s/headers/ (Josef)
[ ] Access-Control-Allow-Credentials.scala 2 core/src/main/scala/org/http4s/headers/
[ ] Access-Control-Allow-Headers.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Access-Control-Expose-Headers.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Allow.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Authorization.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Cache-Control.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Connection.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Content-Disposition.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Content-Encoding.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Content-Language.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Content-Length.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Content-Location.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Content-Range.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Content-Type.scala 3 core/src/main/scala/org/http4s/headers/
[ ] ContentLocationHeaderSpec.scala 6 tests/src/test/scala/org/http4s/parser/
[ ] Cookie.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Date.scala 3 core/src/main/scala/org/http4s/headers/
[ ] ETag.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Expires.scala 3 core/src/main/scala/org/http4s/headers/
[x] FollowRedirect.scala 1 client/src/main/scala/org/http4s/client/middleware/
[ ] Forwarded.scala 3 core/src/main/scala/org/http4s/headers/
[ ] HeaderEchoSuite.scala 2 server/src/test/scala/org/http4s/server/middleware/
[ ] HeaderLaws.scala 1 tests/src/test/scala/org/http4s/headers/
[ ] HeaderParserHelper.scala 1 tests/src/test/scala/org/http4s/parser/
[ ] HeaderParserSuite.scala 2 tests/src/test/scala/org/http4s/parser/
[ ] HeaderSuite.scala 6 tests/src/test/scala/org/http4s/
[ ] HeadersSpec.scala 20 tests/src/test/scala/org/http4s/
[ ] Host.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Http1ServerStageSpec.scala 8 blaze-server/src/test/scala/org/http4s/server/blaze/
[ ] HttpHeaderParser.scala 2 core/src/main/scala/org/http4s/parser/
[ ] HttpHeadersBench.scala 9 bench/src/main/scala/org/http4s/bench/
[ ] If-Match.scala 3 core/src/main/scala/org/http4s/headers/
[ ] If-Modified-Since.scala 3 core/src/main/scala/org/http4s/headers/
[ ] If-None-Match.scala 3 core/src/main/scala/org/http4s/headers/
[ ] If-Unmodified-Since.scala 3 core/src/main/scala/org/http4s/headers/
[x] JsonDebugErrorHandler.scala 1 circe/src/main/scala/org/http4s/circe/middleware/
[ ] Last-Event-Id.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Last-Modified.scala 3 core/src/main/scala/org/http4s/headers/
[ ] Link.scala 2 core/src/main/scala/org/http4s/headers/
[ ] LocationHeaderSuite.scala 6 tests/src/test/scala/org/http4s/parser/
[x] Logger.scala 4 client/src/main/scala/org/http4s/client/middleware/
[x] Logger.scala 7 server/src/main/scala/org/http4s/server/middleware/
[ ] MediaTypeSuite.scala 2 tests/src/test/scala/org/http4s/
[ ] OriginHeaderSuite.scala 10 tests/src/test/scala/org/http4s/parser/
[x] RequestLogger.scala 3 client/src/main/scala/org/http4s/client/middleware/
[x] RequestLogger.scala 5 server/src/main/scala/org/http4s/server/middleware/
[x] ResponseLogger.scala 3 client/src/main/scala/org/http4s/client/middleware/
[x] ResponseLogger.scala 5 server/src/main/scala/org/http4s/server/middleware/
[ ] ResponseParser.scala 7 blaze-core/src/test/scala/org/http4s/blazecore/
[x] Retry.scala 1 client/src/main/scala/org/http4s/client/middleware/
[ ] ServerTestRoutes.scala 4 blaze-server/src/test/scala/org/http4s/server/blaze/
[ ] SimpleHeadersSpec.scala 19 tests/src/test/scala/org/http4s/parser/
[ ] WebSocketBuilder.scala 3 server/src/main/scala/org/http4s/server/websocket/

Hi @vder @VlachJosef I gave a go for those headers below in #4547
i think there are already some duplicate but i'm happy to rebase later
addParser_(CIString("ACCESS-CONTROL-EXPOSE-HEADERS"), `Access-Control-Expose-Headers`.parse)
addParser_(CIString("AGE"), Age.parse)
addParser_(CIString("ALLOW"), Allow.parse)
addParser_(CIString("AUTHORIZATION"), Authorization.parse)
addParser_(CIString("CACHE-CONTROL"), `Cache-Control`.parse)
addParser_(CIString("CONNECTION"), Connection.parse)
addParser_(CIString("CONTENT-DISPOSITION"), `Content-Disposition`.parse)
addParser_(CIString("CONTENT-ENCODING"), `Content-Encoding`.parse)
addParser_(CIString("CONTENT-LANGUAGE"), `Content-Language`.parse)
addParser_(CIString("CONTENT-LENGTH"), `Content-Length`.parse)
addParser_(CIString("CONTENT-LOCATION"), `Content-Location`.parse)
addParser_(CIString("CONTENT-RANGE"), `Content-Range`.parse)
addParser_(CIString("CONTENT-TYPE"), `Content-Type`.parse)
addParser_(CIString("COOKIE"), Cookie.parse)
addParser_(CIString("ACCEPT-ENCODING"), `Accept-Encoding`.parse)
addParser_(CIString("ACCEPT-RANGES"), `Accept-Ranges`.parse)

Had a lengthy Gitter chat with @phongngtuan about standards for these.  To avoid the boilerplate, we think it makes sense to do as follows:

In the companion, define name, value, and parse only via the instance.  So:
object Date {
  /* `Date = HTTP-date` */
  private[http4s] val parser: Parser[`Date`] =
    HttpDate.parser.map(apply)

  implicit val headerInstance: v2.Header[Date, v2.Header.Single] =
    v2.Header.createRendered(
      CIString("Date"),
      _.date,
      ParseResult.fromParser(parser, "Invalid Date header")
    )

  // no `name` (formerly inherited through `Internal`)
  // no `parse` (it's the third argument to `createRendered`)
}

In the case class, don't worry about name, value, or renderString:
final case class Date(date: HttpDate) {
  // The old `value` is part of the Header instance
  // The old `renderString` can be derived from the Header instance

  // If you had a complicated `render` method, maybe comment it out, or use it in
  // your instance definition.  We're still benchmarking how that makes a difference
  // in backends.

  // Some of these might end up with no body at all:
}

.name, .value, and .renderString can be called from the typeclass, and we'll provide syntax via the Header typeclass in the usual way under org.http4s.syntax.  We'll want to get this in quickly, since it affects tests as we port the other headers, but I think @phongngtuan is already looking at this.

This will lead to maintainable, readable definitions.  It might make a few more users have to import syntax.all._, but those methods are mostly used in tests and backends, so I think it's a price we can bear.

I fell behind on the checklist above.  But in the meantime, if you're working on these, please leave a comment like @phongngtuan did, just to make sure we're not duplicating any work.  I'll try to cleanly separate out a list of headers tomorrow.

Hi @phongngtuan
here is a list of Recurring headers ported in #4545
Access-Control-Allow-Headers
Access-Control-Expose-Headers
Cache-Control
Content-Language
Cookie
Forwarded
Link
Proxy-Authenticate
WWW-Authenticate
X-Forwarded-For

Once the new syntax is in place I'll make use of it in these headers as well.

those are the the remaining headers (excluding what's in #4545 and #4547) up for grab :) might as well reserve a few at a time if you're working on them
  0 Access-Control-Allow-Credentials.scala
  1 Date.scala
  2 ETag.scala
  3 Expires.scala
  4 Host.scala
  5 If-Match.scala
  6 If-Modified-Since.scala
  7 If-None-Match.scala
  8 If-Unmodified-Since.scala
  9 Last-Event-Id.scala
 10 Last-Modified.scala
 11 Location.scala
 12 Max-Forwards.scala
 13 Origin.scala
 14 Proxy-Authorization.scala
 15 Range.scala
 16 Referer.scala
 17 Retry-After.scala
 18 Server.scala
 19 Set-Cookie.scala
 20 Strict-Transport-Security.scala
 21 Transfer-Encoding.scala
 22 User-Agent.scala
 23 X-B3-Flags.scala
 24 X-B3-ParentSpanId.scala
 25 X-B3-Sampled.scala
 26 X-B3-SpanId.scala
 27 X-B3-TraceId.scala

I'm booking the following numbers then:
10 Last-Modified.scala
11 Location.scala
12 Max-Forwards.scala
13 Origin.scala
14 Proxy-Authorization.scala
15 Range.scala
and I'll try to push PR later today

@phongngtuan thanks for the list, I'll grab the the ones on the end:
 20 Strict-Transport-Security.scala
 21 Transfer-Encoding.scala
 22 User-Agent.scala
 23 X-B3-Flags.scala
 24 X-B3-ParentSpanId.scala
 25 X-B3-Sampled.scala
 26 X-B3-SpanId.scala
 27 X-B3-TraceId.scala

hi @rossabaker I tried to bring the Select Renderer instance to Select but don't know you to use it without importing the implicit def itself. Do you have any advice?

also have some time so will do
  5 If-Match.scala
  6 If-Modified-Since.scala
  7 If-None-Match.scala
  8 If-Unmodified-Since.scala
  9 Last-Event-Id.scala

and:
  0 Access-Control-Allow-Credentials.scala
  1 Date.scala
  2 ETag.scala
  3 Expires.scala
  4 Host.scala

I'm tackling HeaderLaw in #4552. Once it merges we could change headerLaw(Expires) to headerLaw[Expires] (parentheses to square bracket)

Hi, I'll start migrating last cookies (I hope no one is doing it yet):
16 Referer.scala
17 Retry-After.scala
18 Server.scala
19 Set-Cookie.scala

Hi all, I'm tackling these 3...

ResponseParser.scala
Http1ServerStageSpec.scala
HeaderSuite.scala

I've noticed that headers:
Last-Modified.scala
Location.scala
Max-Forwards.scala
Origin.scala
Range.scala

has not been migrated yet. @vder are you still working on them?

Fantastic work, everyone.  #4576 removes the legacy headers.  If we remove the v2, I think we're done.  I'll tackle that tonight, unless someone claims it.

I've gone ahead tried the move in #4576, too.


Add dyn Trait annotations, remove unneeded lifetime annotations.

As of rust 1.37

dyn Trait refs need to be explicitly annotated
static variable decls don't need the 'static lifetime annotation.

We support Rust all the way back to 1.16.

We support Rust all the way back to 1.16.

Ah! My bad, I should have read the big print in the readme. Thanks.


Non-LazySeq SetVal method?

Hi @nathanmarz et al!
Playing around a little with this library, it appears that transform and setval will return a LazySeq and select will return a PersistentVector. Does Specter provide any way of controlling the return collection type?
Thanks for your time and for your work!
-Stephen

transform and setval maintain the type of the input, so I don't know what you're referring to. Can you provide an example? Also, are you using Clojure or ClojureScript?

Ahh, wonderful! Then, this is just a matter of debugging. For reference I'm using Specter 0.9.1 and Clojure 1.8.0 (not ClojureScript). Here are some of my tests (run from the REPL) and their output:
Select (PersistentList becomes PersistentVector):
(def select-test (com.rpl.specter/select [com.rpl.specter/ALL] '(1 2 3 4 5)))
=> #'clondike.core/select-test
(type select-test)
=> clojure.lang.PersistentVector
(pprint select-test)
[1 2 3 4 5]
Transform (PersistentList becomes LazySeq):
(def transform-test (com.rpl.specter/transform [com.rpl.specter/ALL] inc '(1 2 3 4 5)))
=> #'clondike.core/transform-test
(type transform-test)
=> clojure.lang.LazySeq
SetVal (PersistentList becomes LazySeq):
(def setval-test (com.rpl.specter/setval [com.rpl.specter/ALL] 1 '(1 2 3 4 5)))
=> #'clondike.core/setval-test
(type setval-test)
=> clojure.lang.LazySeq

select always returns a sequence of results and places no guarantees on the return type.
If you run transform/setval on a vector, set, or map, you will get back the exact same type. With lists, the best it can do is return a resolved LazySeq as the result – which is functionally equivalent to lists in all the situations I can think of (e.g. semantics with conj).

That makes perfect sense - it looks like I managed to hit edge cases then. I originally noticed the LazySeq results not with lists, but with PersistentQueue, which is what I'm actually using in my code.
At some point, I may look into adding PersistentQueues to the list of types that transform/setval leave in place - if I do and I manage to get it working, would you be interested in a pull request from me?
Thanks for the explanation!
-Stephen

Yea, that would be a good case to get working properly. The relevant code is here: https://github.com/nathanmarz/specter/blob/master/src/clj/com/rpl/specter/impl.cljx#L506
Lists are handled specially because doing a clojure.core.reducers/map into an empty list actually reverses the order.

Thanks - I'll check it out then!


Simpler report-card invocation: postgres-to-redshift

Context:  https://clever.atlassian.net/browse/INFRA-1577
Refactor so that report-card is invoked via a Publish block;
this lets Drone do the heavy-lifting instead of setting it up locally.
Expected changes:

ADDED

A single entry in the "publish" block report_card: {} (if this wasn't already present)

REMOVED

All "env" block entries related to report-card are removed
All "script" block entries related to report-card are removed

MODIFIED

YAML reformatting/linting (side-effect)

Please verify by looking at the build logs

Your build ran successfully (all tests and script steps completed)
Report card ran successfully (you can see it in the

If you see any errors in the above, please loop @nathanleiby (@n on Slack) into the PR.
Thanks!

Assigning to @schimmy as the current owner of the repo.


[WIP] NetCore remove TC.Active

WORK IN PROGRESS
I'm back after taking a few weeks off. I'm creating this Draft PR so I can fully review all of my changes and figure out what's next.
TODO

let users disable DiagnosticsTelemetryModule in TelemetryConfigurationOptionsSetup.cs
AzureInstanceMetadataTelemetryModule needs an alternate way to read TelemetryModules

Changes

remove TelemetryConfiguration.Active
remove TelemetryModules.Instance
ApplicationInsightsExtensions ??
Refactoring Heartbeats for NetCore scenarios previously dependent on TC.Active

Checklist

[ ] I ran Unit Tests locally.
[ ] CHANGELOG.md updated with one line description of the fix, and a link to the original issue if available.

For significant contributions please make sure you have completed the following items:

[ ] Design discussion issue #
[ ] Changes in public surface reviewed

The PR will trigger build, unit tests, and functional tests automatically. Please follow these instructions to build and test locally.
Notes for authors:

FxCop and other analyzers will fail the build. To see these errors yourself, compile localy using the Release configuration.

Notes for reviewers:

We support comment build triggers

/AzurePipelines run will queue all builds
/AzurePipelines run <pipeline-name> will queue a specific build

Lets try to do this in steps
1st remove .Active (with a config option to enable it for some customers who use it)
Add all modules including DiagModule via DI.
Fix heartbeat modules like AppServiceMetadata, AzureMetadata etc.

@cijothomas i don't think that will work because I think these things are too tightly coupled. Please respond if I'm misunderstanding...

First, supporting both .Active and not would be some additional work that I don't know if it's worth the effort.
Second, I can't support the "not .Active" config without first fixing the DiagModule and Heartbeat Modules.
Lastly, after I've fixed all the modules, I don't think there's any value in continuing to support .Active.

@cijothomas i don't think that will work because I think these things are too tightly coupled. Please respond if I'm misunderstanding...

First, supporting both .Active and not would be some additional work that I don't know if it's worth the effort.
Second, I can't support the "not .Active" config without first fixing the DiagModule and Heartbeat Modules.
Lastly, after I've fixed all the modules, I don't think there's any value in continuing to support .Active.

If there are users somehow tied to use .Active, then we want to have a way to support them. For majority of cases, we can ask to switch to DI model, but there are cases like some logging adapters for which there is no better alternate than TC.Active.

Hey @cijothomas, this test is failing at line 154:
I'm debugging it but I'm a little lost.
The server.Listener did not catch the TrackTrace from line 152.
Can you think of anything that I should investigate?
I confirmed that I can run this test locally on the develop branch, but I do get the same failure by replacing TC.Active with TC.CreateDefault().
https://github.com/microsoft/ApplicationInsights-dotnet/blob/d2e6e7b0ed975485a483f2b67f9be7b02186f9e5/NETCORE/test/FunctionalTests.WebApi.Tests/FunctionalTest/MultipleWebHostsTests.cs#L124-L161

/AzurePipelines run


Fix menu capitalization and typo

Fixed from "Preserve Setting to stop overide after Download" to "Preserve Setting To Stop Override After Download".

Thanks


Fixed closeAllNotifications() throwing error when there are no notifications

@treblereel
CC @tomasdavidorg
This fixes the error in the JSON test Completes serverless workflow from an empty file.
There is one more issue to be fixed in the YAML testCompletes serverless workflow with function and state autocompletion:
On my laptop, it seems that VSCode/Selenium is trying to close the editor while the test is running, showing the notification if the user wants to save the file. This notification is on top and hides the autocomplete menu.
I will investigate more, on that.

@fantonangeli cool, are you talking about ElementNotInteractableError: element not interactable ?

@fantonangeli cool, are you talking about ElementNotInteractableError: element not interactable ?

Exactly that one

Here is the video:
https://user-images.githubusercontent.com/4498639/229109616-51c34e66-b91a-412b-ae02-55c8f7e75a33.mp4


NTFS Extended attributes usage for DrvFs metadata incompatible with Windows API

Your Windows build number:  (Type ver at a Windows Command Prompt)
Microsoft Windows [Version 10.0.17063.1000]

What you're doing and what's happening:  (Copy&paste specific commands and their output, or include screen shots)
DrvFs with the metadata uses incorrect layout of FILE_FULL_EA_INFORMATION.
Below image is on-disk data.
The UID I use is 290848682 (==0x1155FFAA). And the GID is<PHONE_NUMBER> (==0x44880066).
WSL is reading position that has deviated by 1 byte.

The image below is read by ZwQueryEaFile().
Can not read data with 1 byte deviated using ZwQueryEaFile(). Windows will replace that with 0x00.

Looks like you are correct, unfortunately. We will fix this issue, which unfortunately means that any metadata created by this build will not show up correctly once the bug is fixed.
Sorry for the inconvenience!

Great find @0xbadfca11


Setup CI/CD

Setup github action and jacoco

Need setup with sonarcloud

Need setup jib to fully supports docker container


Popup open animation is quirky in Firefox

Short description
Very often when opening a w2ui popup window in Firefox, I observe one of two undesirable behaviors. Either the popup simply appears, without either the scale-up or fade-in transition apparently having run, or else, more commonly and more obtrusively, the scale-up transition appears to run at the wrong moment. What I mean is that the dialog seems to appear at full size, then the initial transform: scale(0.8) seems to be applied, and then the transition back up to full size seems to run.
What is current behavior
I may not have got the sequence precisely right in my description above, but the visually observable effect is that the dialog goes from full size, to smaller size, and then back to full size in quick succession, while it is in the process of opening. It's very evident, and very clearly not what's intended.
I have been testing in Chrome and in Firefox. I've never noticed this behavior in Chrome, but I almost always (though not absolutely consistently) notice it in Firefox. I am running Firefox 84.0.2 on 64-bit Windows.
What is desired behavior
The opacity and transform CSS transitions should run smoothly and in a coordinated way to give an appealing "appear" animation to the dialog.
Additional notes
Digging into the w2ui source code and experimenting, I happened to discover that if I increase the timeout interval, as specified on line 238 in w2popup.js, from 1ms to 100ms, I no longer observe the anomalous effect in Firefox.
This is the delay that's applied before setting the opacity and the scale factor back to 1 and starting the CSS transition on these two properties. That it appears to be a timing issue leads me to wonder if there is internally a delay in Firefox's applying the initial scale(0.8) transform (see line 198), such that it applies this after initially flashing the dialog on screen at full size.

FYI, a colleague reported observing this behavior in an earlier version of Chrome also. However, it doesn't seem to happen in the most up-to-date version of this browser.
Absent a fix, I will probably work around the oddness by suppressing the part of the popup animation that relies on the scale transform. This will leave me with a popup that simply fades in and out, which looks fine to me.
#w2ui-popup {     transform: none !important; }

FYI, a colleague reported observing this behavior in an earlier version of Chrome also. However, it doesn't seem to happen in the most up-to-date version of this browser.
Absent a fix, I will probably work around the oddness by suppressing the part of the popup animation that relies on the scale transform. This will leave me with a popup that simply fades in and out, which looks fine to me.
#w2ui-popup {     transform: none !important; }

Closing issue: PR #1935 has been merged


ENH+UX: raise error when setting a catalog super for an id+version that are not in the catalog

At the moment datalad catalog set-super will just set the details to whatever dataset is provided with the id and version flags, without knowing whether metadata content for this dataset is contained in the dataset. Ideally, this would require a check first, and a warning/error message to the user in case the dataset is not contained in the catalog.

Closed by https://github.com/datalad/datalad-catalog/pull/309


[Feature]-[3000]:Provide support for older MSSQL version that require TLS1.0

Is there an existing issue for this?

[X] I have searched the existing issues

Summary
Appsmith's backend server doesn't allow connecting to endpoints that require TLS1.0 and not TLS1.1 or TLS1.2
Why should this be worked on?
https://app.intercom.com/a/apps/y10e7138/inbox/inbox/all/conversations/164629100091357

MsSQL 6 month usage: 220
apply 25% requirement: 55
Stats

Stat
Values

Reach
55

Effort (months)
<>

@rohan-arthur the RICE score is infinity on this 🤯

@dilippitchika yes, still needs the effort to be added - have applied the label needs estimate to this issue so that the team knows to add the effort to get the final score

This isn't specific to MSSQL, but has been seen with MySQL and I think MongoDB as well. This needs to be fixed/changed at the JRE settings level.
Here's the past conversation on this with gory details: https://theappsmith.slack.com/archives/CPQNLFHTN/p1633705690184200.
The solution should be that TLSv1 will be disabled by default, and users can set an env variable, like APPSMITH_ENABLE_TLSV1, which, if set to true, should do the necessary changes from the entrypoint script before starting the backend server.

@sharat87 thank you for sharing more insight into this! I have updated the reach score.

Another user faced an issue due to this - https://discord.com/channels/725602949748752515/760761686549463060/1013390850811756545

Before we attempt to solve this, I'd encourage everyone to advise upgrading their systems to move away from TLS v1.0. It's been deprecated for over two years now, and is very-not-recommended. See https://en.wikipedia.org/wiki/Transport_Layer_Security#TLS_1.0.
I'm going to close this issue, in that interest. If there's someone who absolutely cannot upgrade from this vulnerable TLS version, we can reopen and reconsider.


useCustomResources infinitely runs

useCustomResources

Oddly, I don't see that hook being called anywhere in the current code base 🤔 I must be missing something ...

But if it indeed is that one anyway, a hunch based on just its shape: I'd guess the customResourceMatchers array comes in as a fresh new array every single time, leading the useCallback being triggered over and over etc

Oh it probably is useKubernetesObjects instead then? But that one doesn't seem to have any risky inputs 🤔 Maybe the entire component that uses the hook is mounted and unmounted repeatedly instead then, for some other reason?

I was using the useCustomResources hook externally and noticed it did this when I used it in the open source repo too, I'm gonna write a test to replicate then fix it so it doesn't regress.


Set minimum lenght on password field

Since using a shorter password results in an error creating the user and then you're left with an uncomplete database

I've added some validation. It goes red when lower than 10 characters


TST DOC Window docs, tests, etc.

This should fix https://github.com/scipy/scipy/issues/4234 and improve coverage of exceptions, etc. shown by codecov.
Combined several window functions into a generic sum-of-cosines function, since that's how most references describe the different functions anyway:

This kind of optimization leads to the definition of the so-called flat-top windows. If a
third order window is considered, this optimization yields the following coefficients: A0 = 0.2811, A1 = -0.5209, A2 = 0.1980.

So now you can reproduce this window with  _cos_win(M, [0.2811, 0.5209, 0.1980]).

Otherwise LGTM

Ping me when the last comment is addressed and the CIs are happy and I'll go ahead and merge.

@Eric89GXL Do you still have Matlab access?  Can you get the output of all of its window functions for odd- and even-length, both symmetric and periodic, so we can check consistency with it?  I think length 6 and 7 are enough to distinguish them without making the test file terribly long?  (5 is not: bartlett(5) == hann(5) == barthann(5))
I think something like this would work:
clear all; close all; clc
format long e
format compact

barthannwin(6)
barthannwin(7)
bartlett(6)
bartlett(7)
blackman(6, 'periodic')
blackman(7, 'periodic')
blackman(6, 'symmetric')
blackman(7, 'symmetric')
blackmanharris(6, 'periodic')
blackmanharris(7, 'periodic')
blackmanharris(6, 'symmetric')
blackmanharris(7, 'symmetric')
bohmanwin(6)
bohmanwin(7)
chebwin(6, 100)
chebwin(7, 100)
chebwin(6, 10)
chebwin(7, 10)
flattopwin(6, 'periodic')
flattopwin(7, 'periodic')
flattopwin(6, 'symmetric')
flattopwin(7, 'symmetric')
gausswin(6)
gausswin(7)
gausswin(6, 1)
gausswin(7, 1)
hamming(6, 'periodic')
hamming(7, 'periodic')
hamming(6, 'symmetric')
hamming(7, 'symmetric')
hann(6, 'periodic')
hann(7, 'periodic')
hann(6, 'symmetric')
hann(7, 'symmetric')
kaiser(6)
kaiser(7)
kaiser(6, 2.7)
kaiser(7, 2.7)
nuttallwin(6, 'periodic')
nuttallwin(7, 'periodic')
nuttallwin(6, 'symmetric')
nuttallwin(7, 'symmetric')
parzenwin(6)
parzenwin(7)
rectwin(6)
rectwin(7)
taylorwin(6)
taylorwin(7)
taylorwin(6, 5, -25)
taylorwin(7, 5, -25)
triang(6)
triang(7)
tukeywin(6, 0)
tukeywin(7, 0)
tukeywin(6, .25)
tukeywin(7, .25)
tukeywin(6)
tukeywin(7)
tukeywin(6, .75)
tukeywin(7, .75)
tukeywin(6, 1)
tukeywin(7, 1)

Here is the output of what you wrote:
https://gist.github.com/e-q/6cdaf9b098b76a1c2e428e6b583ec999
This looks like a great PR!

@e-q
Thanks!  What's different about https://gist.github.com/Eric89GXL/8239281 that it has so many more digits though? I thought if I copied the beginning lines it would do the same thing.

Hm, maybe my local MATLAB config messed with it. I tried again, and it looks like it came out right.

@e-q Matlab's triang(6 or 7) really goes above 1?  o_O

Not on my system, 2013a:
>> triang(6)
ans =
    0.1667
    0.5000
    0.8333
    0.8333
    0.5000
    0.1667
>> triang(7)
ans =
    0.2500
    0.5000
    0.7500
    1.0000
    0.7500
    0.5000
    0.2500

I don't think so. Maybe you miscounted the outputs and are looking at the taylorwin(6, 5, -25) and taylorwin(7, 5, -25) calls?

Oops, yes. 😬  Still weird that taylorwin isn't normalized to 1, but it's a frequency domain window and we don't have it yet, so I won't worry about it.

I think I will leave fixing https://github.com/scipy/scipy/issues/2190 for another PR in case it gets delayed, etc.

Great. LGTM +1 for merge. @e-q do you have time to have a look and merge if you're happy?

Added one more ref, squashed some stuff and rebased

Very nice @endolith, thanks!