Do not store context inside structs

See https://github.com/getAlby/nostr-wallet-connect-next/pull/92#discussion_r1515131003

Looking closer at this issue, I realized the current implementation should probably be left as is.
First of all, the rule of thumb in Go is to pass context as a parameter to methods instead of storing it in structs.
However, in NWC, the context is stored within Service. The intention, apparently, is to make it possible to abort currently running requests when the app is stopped and the parent context is cancelled. Since we have potentially long-running requests (i.e. requests going to the Lightning network), it does make sense. On the other hand, it only seems to be used in LND, because other clients call Rust code through FFI and Go contexts do not work there.

I think there are still some places where we can reduce the usage of the context stored in the service - by moving the signal.NotifyContext to the entrypoints (main_http and main_wails) and pass it as a parameter further down, rather than creating it in the service. There are some places in the api where we use svc.ctx instead of passing a context in (e.g. the HTTP service could pass the echo request context to the api functions such as ListChannels)
Would that sound like a good idea to you?

I think there are still some places where we can reduce the usage of the context stored in the service - by moving the signal.NotifyContext to the entrypoints (main_http and main_wails) and pass it as a parameter further down, rather than creating it in the service

Right, that makes sense. I moved it to main_http. However, I am not so sure about Wails — do we need to handle signals there? Doesn't Wails handle shutdown on its own?

There are some places in the api where we use svc.ctx instead of passing a context in (e.g. the HTTP service could pass the echo request context to the api functions such as ListChannels)

That could work, actually. First I thought it was impossible, since echo.Context is totally not the same as golang's Context. Then I realised that we still can get the request through echo.Context's Request() method, and then get the request's context, which should propagate cancellation when either a client drops connection or the server is shutdown.
I will implement that!

I have implemented the proposed changes.
I tried to remove the context from the service entirely, but that turned out to be much trickier. The service's context is used as the parent context when starting the API in the /api/start endpoint. We cannot use request context instead, because it is cancelled when the client connection is closed.


Add tests for JdkHttpOaiClient

Currently, Dataverse does not make use of the service provider part of this library. Leaving this for a later exercise.

Dataverse now does use this, as we saw with #120. Maybe we should think about this one again...?

This might be related to an implementation for #141, as we could use WireMock to try different OAI-PMH responses from out in the wild here.


Update yarn.lock with integrity data

Improves security of dependency libraries

this should work for everyone as the yarn version is dictated by the gradles plugin (unless you are calling yarn directly, instead of ./gradlew yarn)


Failed to complete quests

What is your Quests version / build number (do not say "latest")?
4.8.3
Describe the problem and how to replicate it.
Why was I completing a mission and when I logged out of the server and logged back in, I don't see what I have to continue doing? I get mission not found or something like that.
Any console errors to share?
No.

If applicable, can you provide an example quest from quests.yml?
No response

Without a console error or example quest, here is nothing for me to investigate. If you can at least provide the exact message you get, the Discord may be able to help you.


revert overdeleted c-a binding from #715

@crides any reason?
@Tastaturtaste was there a reason that ctrl-a was removed?

Yes, the reason was that I wanted to avoid specifying default behaviour for nushell in multiple places. Since the nushell defaults are already defined in config.nu I removed the default from reedline while I was already at it removing conflicting defaults. I figured that since the feature was still pretty fresh and not yet released in nushell there shouldn't be any breakage.

In nushell we prefer the reverse. If there is a default, it should be in rust code so that if you run nushell without a config file, you still get default keybindings as listed in default_config.nu. I don't see a conflicting ctrl+a. Is there one?
Also, not all nushell defaults are defined in the config.nu. Some of the VI keybindings aren't easily exposed and therefore are not available.

ctrl + char_a is already taken in default_config for move_to_line_start. Instead I added a binding for ctrl + shift + a for select_all.
And this confusion is the reason I would prefer to have all defaults in one place. But it's your call, of course.

ctrl+a is not conflicting with this PR

There is no conflict, but there is only human vigilance to keep those places in sync. I realize now this is known and very much intended, even though I don't agree.
When I deleted the ctrl + a binding I didn't realize it was already there before my PR was merged and I didn't want to introduce further duplication. The removal of existing duplication was not my intention.
For all I know this PR should be fine to merge to get the previous default even without a default_config back.

@crides any reason?

Restoring previous well-known defaults

ok, no worries at all @Tastaturtaste. I was just trying to understand your position. Thanks @crides.


Notification does not receive on Android

Push notification does not receive on android platform but working on iOS. getToken and onMessage function get fired but the token and payload is blank.
Can you please help in this.

Any help will be appreciated to solve this issue on Android platform.

Check the generated AndroidManifest.xml. If you are using<EMAIL_ADDRESS>then the problem might be in merging the configurations. by me
<service android:name="by.chemerisuk.cordova.firebase.FirebaseMessagingPluginService">
                <intent-filter>
                    <action android:name="com.google.firebase.MESSAGING_EVENT"/>
                </intent-filter>
            </service>

was missing from the manifest file.

hi @odai-alali, I am using 9.0.0<EMAIL_ADDRESS>the configuration is already available in manifest file, still not getting push notification. getToken and onMessage function get fired but the token and payload is blank.

Then you have another problem. Hope you find a solution

I'm having the same issue, in<EMAIL_ADDRESS>and<EMAIL_ADDRESS>and<EMAIL_ADDRESS>

I'm having the same issue, in<EMAIL_ADDRESS>and<EMAIL_ADDRESS>and<EMAIL_ADDRESS>
Did you find any solution
I have same configuration as yours

I'm having the same issue, in<EMAIL_ADDRESS>and<EMAIL_ADDRESS>and<EMAIL_ADDRESS>
Did you find any solution
I have same configuration as yours


changes in tensorboard

How can I change the tensorboard (in train.py/test.py) to show warped cloth overlayed on the person without making the warped cloth or the person transparent?

@solitarysandman  how did you generate warp-mask & warp-cloth? for training TOM

Hey @Arbi112
You can get warp-mask and warp-cloth by:

training the GMM module.
run a test on your entire data with a checkpoint of your choice and save all the warp-mask and warp-cloth generated.

Thank you so much for your quick reply.
1- Training the GMM Module
With following commend
python train.py --name gmm_train_new --stage GMM --workers 4 --save_count 5000 --shuffle
i run the test on my test data  as following. How can i save all wrap-mask and wrap-clothe

@solitarysandman Thank you so much for your quick reply.


Chat bugs

Latest client code on Windows. Bugs observed after receiving a normal chat message from another node.
Part 1

Part 2

I can't replicate these issues.

Fixed ages ago.


Is it possible to control the hosting environment in a MBB hybrid app?

There is some code in the BlazorWebHost class to change behaviour by environment, but I can't find any way to control the environment in which the application considers itself to be running. To be specific, I can't see any way to set HostingEnvironment.EnvironmentName.
As far as I can see, no environment variables are being added, for example - presumably because environment variables are not available in some hosts and/or not considered a sufficiently secure way to switch behaviours on a client-side app.
I can't see any way to customise this behaviour by specifying an environment for those people who consider it an acceptable risk. There doesn't seem to be any way to add a call to the ConfigureHostConfiguration method of the underlying HostBuilder instance being used prior to its CreateDefaultBuilder method having completed, and the setup of the environment being too late to affect most of the behaviours you would expect to be able to control.
Am I missing a hook that allows us to specify the environment in which a MBB host app is running somehow?
I think this is a limitation of Xamarin rather than anything else, but I'd appreciate people's thoughts on the security and other aspects associated with this feature, as well as ideas on how it might be managed.

Having thought about this some more, I'm wondering if a sensible approach might be to add hooks to allow it to be controlled through conditional compilation of the host application.
If you added just enough hooks into the BlazorWebHost and MobileBlazorBindingsHost classes to enable the environment to be set by conditional compilation (debug versus release, for example) then people would have a modicum of control for their development needs without exposing a significant security risk. Accompanying documentation would be needed to highlight the risk of controlling the environment by anything other than conditional compilation - unless Xamarin Forms developers already have a secure way of doing this somehow?

You can set the environment in App.cs like this:
using Microsoft.Extensions.Hosting;
...
            var hostBuilder = MobileBlazorBindingsHost.CreateDefaultBuilder()
                .UseEnvironment("staging123") // Set this to whatever you want
                .ConfigureServices((hostContext, services) =>
                { ... }

And then you can read the environment by getting the IHostEnvironment service from DI:
@inject Microsoft.Extensions.Hosting.IHostEnvironment Env

Current env: @Env.EnvironmentName

The default is always "Production" because that is the safe default setting. You can then set up your app to use different environments in different configurations either from code (as seen above), or using env variables (where supported), or any other configuration pattern.

Ah, that's great. Thanks. Sorry, I missed that completely.
I'll add some conditional compilation to enable me to switch by build configuration, so that moves me forward a bit.
However, I think I am right in saying that this needs to come after the CreateDefaultBuilder() method call, which means the environment gets changed after the CreateDefaultBuilder() method has completed in BlazorWebHost. As a result, things like loading environment-specific settings files will already have completed. It seems just slightly too late; I wonder if the method needs to be added to MobileBlazorBindingsHost as well, and then used to customise BlazorWebHost before the call to CreateDefaultBuilder.

Oh I think I see. You're saying that even though you can set the environment name with this API, it happens too late for some built-in features done by the BlazorWebHost that have already run?

BTW most of what's done in BlazorWebHost is "deferred" in the sense that it's defining method callbacks up front, but they don't run until "later." I'll have to test this to see if the order is already correct. (Just looking at the code can be deceptive!)

@Eilon yep, I think you've managed to struggle through my gibberish and get to the core of the problem: I think there's an issue with the order of execution.
I know exactly what you mean about deferred execution; anything to do with delegates can be, erm, lets say "cerebrally challenging." With delegates you are defining code that will run in the future. I understand anyone who complains that their head hurts when that is happening! However, in this case, I think the calls to the CreateDefaultBuilder() methods are regular method calls, and chained, and as a result they are not affected by the deferred execution of the delegates in this area of the code.
If your machine is set up to do so easily, setting a breakpoint on line 34 of BlazorWebHost.cs and then trying to hit it would tell us one way or another. I don't think it would be possible to execute that line of code, as the code is structured at the moment.
But there be delegates, and they be like (future) dragons, so I make no promises!


Failed to run program in core.py

changed path on file storage.py

Problem solved in last commit. Program working normally


Unable to Run Umami using Podman\Quadlet -  Error: getaddrinfo ENOTFOUND Umami

Describe the Bug
When start Umami with podman it appears that Umami is looking for the hostname Umami and cannot resolve it
Database
PostgreSQL
Relevant log output
__Setup__
RHEL 9.4
Podman 4.9.4

Containers are configured using quadlet

__log output__

yarn run v1.22.19
May 08 21:39:42 blog Umami[387610]: warning Skipping preferred cache folder "/app/.cache/yarn" because it is not writable.
May 08 21:39:42 blog Umami[387610]: warning Selected the next writable cache folder in the list, will be "/tmp/.yarn-cache-2018".
May 08 21:39:42 blog Umami[387610]: $ npm-run-all check-db update-tracker start-server
May 08 21:39:42 blog Umami[387610]: warning Cannot find a suitable global folder. Tried these: "/usr/local, /app/.yarn"
May 08 21:39:42 blog Umami[387610]: warning Skipping preferred cache folder "/app/.cache/yarn" because it is not writable.
May 08 21:39:42 blog Umami[387610]: warning Selected the next writable cache folder in the list, will be "/tmp/.yarn-cache-2018".
May 08 21:39:42 blog Umami[387610]: $ node scripts/check-db.js
May 08 21:39:42 blog Umami[387610]: warning Cannot find a suitable global folder. Tried these: "/usr/local, /app/.yarn"
May 08 21:39:42 blog Umami[387610]: ✓ DATABASE_URL is defined.
May 08 21:39:42 blog Umami[387610]: ✓ Database connection successful.
May 08 21:39:43 blog Umami[387610]: ✓ Database version check successful.
May 08 21:39:44 blog Umami[387610]: Prisma schema loaded from prisma/schema.prisma
May 08 21:39:44 blog Umami[387610]: Datasource "db": PostgreSQL database "umami", schema "public" at "Umami-Database:5432"
May 08 21:39:44 blog Umami[387610]:
May 08 21:39:44 blog Umami[387610]: 6 migrations found in prisma/migrations
May 08 21:39:44 blog Umami[387610]:
May 08 21:39:44 blog Umami[387610]:
May 08 21:39:44 blog Umami[387610]: No pending migrations to apply.
May 08 21:39:44 blog Umami[387610]:
May 08 21:39:44 blog Umami[387610]: ✓ Database is up to date.
May 08 21:39:44 blog Umami[387610]: warning Skipping preferred cache folder "/app/.cache/yarn" because it is not writable.
May 08 21:39:44 blog Umami[387610]: warning Selected the next writable cache folder in the list, will be "/tmp/.yarn-cache-2018".
May 08 21:39:44 blog Umami[387610]: $ node scripts/update-tracker.js
May 08 21:39:44 blog Umami[387610]: warning Cannot find a suitable global folder. Tried these: "/usr/local, /app/.yarn"
May 08 21:39:44 blog Umami[387610]: warning Skipping preferred cache folder "/app/.cache/yarn" because it is not writable.
May 08 21:39:44 blog Umami[387610]: warning Selected the next writable cache folder in the list, will be "/tmp/.yarn-cache-2018".
May 08 21:39:44 blog Umami[387610]: $ node server.js
May 08 21:39:44 blog Umami[387610]: warning Cannot find a suitable global folder. Tried these: "/usr/local, /app/.yarn"
May 08 21:39:45 blog Umami[387610]:  ⨯ Failed to start server
May 08 21:39:45 blog Umami[387610]: Error: getaddrinfo ENOTFOUND Umami
May 08 21:39:45 blog Umami[387610]:     at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:107:26) {
May 08 21:39:45 blog Umami[387610]:   errno: -3008,
May 08 21:39:45 blog Umami[387610]:   code: 'ENOTFOUND',
May 08 21:39:45 blog Umami[387610]:   syscall: 'getaddrinfo',
May 08 21:39:45 blog Umami[387610]:   hostname: 'Umami'
May 08 21:39:45 blog Umami[387610]: }
May 08 21:39:45 blog Umami[387610]: error Command failed with exit code 1.
May 08 21:39:45 blog Umami[387610]: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
May 08 21:39:45 blog Umami[387610]: ERROR: "start-server" exited with 1.
May 08 21:39:45 blog Umami[387610]: error Command failed with exit code 1.

Which Umami version are you using? (if relevant)
latest
Which browser are you using? (if relevant)
No response
How are you deploying your application? (if relevant)
quadlet

Solved the issue by using the option --hostname=unami - looks like the hostname is hardcoded somewhere


Retrieve All Job Titles

It would be pretty cool to be able to obtain all the job titles. Nifty feature in cases such as forms where users can pick their job titles.

Job titles are randomly generated.

Addressed in #540


Composable docker images with Nix

So I wanted a Nix function to create docker images. That's easy, just make a tarball of the closure of a derivation, and it's ready to be imported.
The problem is: how do you add more content to an existing image, by using layers?
So here's an attempt to provide a solution, even if it's very ugly.
Let's start with a simple bash container:
$ nix-build -E 'with import <nixpkgs> {}; dockerImage { drv = bashInteractive; drvCommand = "${bashInteractive}/bin/bash"; }'
# cat result/image.tar.gz|docker import - nixbash
# docker run -t -i nixbash /bin/bash
bash-4.3#
# docker history nixbash
IMAGE               CREATED             CREATED BY          SIZE
4055d6dc456e        30 minutes ago                          186.4 MB

Now let's create another image with python only:
$ nix-build -E 'with import <nixpkgs> {}; dockerImage { drv = python; drvCommand = "${python}/bin/python"; }'
# cat result/image.tar.gz|docker import - nixpython
# docker run -t -i nixpython /bin/python
Python 2.7.10 (default, Jan 01 1970, 00:00:01)
...
# docker history nixpython
IMAGE               CREATED             CREATED BY          SIZE
4ed68027aae5        10 seconds ago                          223.5 MB

Now it comes the magic. Let's layer that python image on top of bash:
$ nix-build -E 'with import <nixpkgs> {}; dockerImage { drv = python; drvCommand = "${python}/bin/python"; baseImage = "nixbash"; }'
# cd result
# ./docker-build.sh -t nixpython2 --no-cache .
# docker history nixpython2
IMAGE               CREATED             CREATED BY                                      SIZE
51b3911260fc        17 seconds ago      /bin/sh -c hostip=$(/nix/store/wa3fimm2m26fwh   46.57 MB
4055d6dc456e        24 minutes ago                                                      186.4 MB

Note: the baseImage is there only for convenience. One might as well generate a Dockerfile on the fly, and use the same image.tar.gz. I haven't done that, but it's easy to do.
In other words, we're able to take arbitrary nix containers and layer multiple of them with minimal overhead.
Now that 184mb is because for some reason gcc is pulled in with a lot of other stuff, most of which are useless for doing the layer merge. The python closure is only 36mb, so... there's a lot of opportunity to optimize.
Just think a debian docker is around 150mb, so we aren't that huge either.
cc @offlinehacker @datakurre @domenkozar for discussion :)

This is neat. Do you think it would be possible someday to have the layering work within the nix expressions? Could you have baseImage = bashImage and bashImage = dockerImage { ... } so that Nixpkgs could provide a generic baseImage attribute that lives in the nix store that other specialty images could layer on top of?
Also it sounds like with some tuning (figure out why gcc is pulled in) Nix could create the leanest container images of any OS since exactly the minimal set of dependencies for any given program would be pulled in. That would be almost micro-kernel like.

@trishume I'm thinking about that and I feel that it may be possible. docker save would be the derivation output of an image. At this point inside nix, run the docker daemon with custom paths, docker load, docker build and finally docker save. It all depends if the docker daemon may run as a user and with enough privileges for creating his filesystems inside nix.
And yes, docker images with nix, if packages were properly optimized, would deliver the smallest images.

@lethalman That 184 MB for Python sounds strange. I've been getting 36 MB tar.gz and 108.9 MB unpacked "virtual size" already. Since I've still learning Nix as language, I've simply:
nix-build python.nix
tar cvz `nix-store -qR result ` result > python.tar.gz

Also good to know: If there's no explicit PATH env defined, Docker run seems to add the normal /usr/local/bin:/usr/bin:/bin/etc... so that docker run -t -i nixpython python should be enough for run.
Then there's that issue of many software expecting /bin/sh or /bin/bash and writable /tmp to exist. And usually one need to add many derivations to PATH. I went far enough to map expression build result into image root with:
mkdir tmp
tar cvz --transform="s|^result/||" tmp `nix-store -qR result` result/* > image.tar.gx

And with buildEnv that results with /bin et al. having all the expected paths. Probably there are also downsides.

@datakurre that's not only python, but also tools for extending the image later with layers.

@trishume so it seems possible to do with runInLinuxVM, however there's a drawback of docker save/load. The of the image cannot be chosen when loading, but it can be renamed afterwards. So indeed we can generate layered images. Will do a complete PR.

@lethalman That explains. You figured out quite a way to get data into image without through the filter without ADD or COPY :)
Still, could there be a way to create delta.tar.gz outside Docker (e.g. getting the paths from the base image with docker save) and use a Dockerfile with ADD delta.tar.gz / instead? According to the docs https://docs.docker.com/engine/reference/builder/#add ADD has bevior of tar -x
the result is the union of:

Whatever existed at the destination path and
The contents of the source tree, with conflicts resolved in favor of “2.” on a file-by-file basis.

@datakurre with a delta it could be possible yes, but it's still another step to do from the outside :(

@lethalman Another option could be to optionally manage image "version" with Nix profiles. Could that make any sense? You would not want that for one one-shot images, but if you'd maintain a long-living production app it might make sense.

@datakurre can you explain better? I didn't understand the image version with Nix profiles...

@lethalman Sorry, I missed your update on deltas. Is there any option, which would work within Docker? (To make it possible to build a Linux Docker image on a Mac with boot2docker.)
About the profiles. I was thinking, if it would make sense to manage named dockered apps using Nix profiles so that it would be possible to create a delta tarball by diffing paths from the previous profile version to the current version. But I didn't think this too much. Probably something, where a bash script works better than a nix expression. And very impure.

@datakurre the problem is docker build , so you still need impurity, hence no reason to put other metadata somewhere else... because you need impurity anyway.
I didn't understand you first question...

@lethalman Agreed. I understand this now.
Also the first question was probably a misunderstanding from me. So, would nix-docker-merge result an image.tar.gz, Dockerfile and the script to create the delta for the base image? So that to build x86 containers, I could call dockerImage at Nix container in VM, copy the results into darwin host and run the script on darwin host?

nix-docker-merge creates the Dockerfile, the delta and outputs the saved image. If the VM has x86 nix, yes. Note that runInLinuxVM bind mounts the host /nix, also I don't know if it works on darwin.
So if you have an x86 VM, and you create two docker images from there, you can merge the result of course regardless of the architecture. Basically that nix-docker-merge has nothing to do with nix, it only uses nix-hash :)

Alright :D We have it...
with import <nixpkgs> {};

let
  image1 = dockerTools.createImage { drv = bashInteractive; };
  image2 = dockerTools.createImage { drv = python; };
in
  dockerTools.mergeImages { name = "bash-python"; inherit image1 image2; }

$ nix-build
/nix/store/br5hqbzcqcgv7fy85pp7h91smpia32nb-bash-python.tar.gz
# docker load -i result
# docker history 240c630f46d8
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
240c630f46d8        5 minutes ago       /bin/sh -c #(nop) ADD dir:f576a907f86b772f9b7   68.29 MB
df1dafed18ab        6 minutes ago                                                       50.16 MB            Imported from -

There are two problems:

When doing docker build I've passed -t bash-python, but then docker load ignore the name and imported it anonymously.
But the real problem is that docker load does not print the image id :( hence mergeImages will not work with layered images................... that's really something to report to docker.

@lethalman I fell in love with you :laughing:
That's so great! I have to test it

@geerds don't scare him away :warning:

Just reported the docker load problem upstream: https://github.com/docker/docker/issues/18117

:+1:

Alright. For now I've workarounded that issue, and we can only deal with layered images that have a name, because we cannot know easily the image id of the loaded image.
So we can stack a many layers as we want from independent images:
with import <nixpkgs> {};

rec {
  image1 = dockerTools.createImage { drv = bash; };
  image2 = dockerTools.createImage { drv = python; };
  image3 = dockerTools.createImage { drv = perl; };
  bash-python = dockerTools.mergeImages { name = "bash-python"; baseImage = image1; image = image2; };
  bash-python-perl = dockerTools.mergeImages { name = "bash-python-perl"; baseImage = bash-python; image = image3; };
}

Do nix-build -A bash-python-perl et voila!
Going to add also some other useful tools, like converting a flattened image to an image with one layer, and viceversa.

@edolstra do you feel we can merge this? It's just tools, no harm for the build farm.

Docker is taking up so much disk space (like over 2gb...) that I'm thinking of implementing the packing format of the images by myself. It's quite simple, and there will be no need of runInLinuxVM, completely pure.
https://github.com/docker/docker/blob/master/image/spec/v1.md
And they even reimplemented nix-hash: https://github.com/docker/docker/blob/master/pkg/tarsum/tarsum_spec.md

There we go, cleaned up of all the docker impure and huge evilness. It's all pure nix, we generate compatible docker images.
with import ./. {};

rec {
  image1 = dockerTools.mkImage { drv = bash; };
  layeredImage1 = dockerTools.toLayeredImage { name = "bash"; image = image1; };
  image2 = dockerTools.mkImage { drv = python; };
  image3 = dockerTools.mkImage { drv = perl; };
  bash-python = dockerTools.mergeImages { name = "bash-python"; baseImage = layeredImage1; image = image2; };
  bash-python-perl = dockerTools.mergeImages { name = "bash-python-perl"; baseImage = bash-python; image = image3; };
}

# docker load -i result
# docker history bash-python-perl
IMAGE               CREATED               CREATED BY          SIZE
2fbfd365736b        45.916921 years ago                       145.2 MB
c23bc184169f        45.916921 years ago                       68.29 MB
8ed4ee623119        45.916921 years ago                       40.8 MB
# docker run -t -i bash-python-perl /nix/store/rfcsnszaw2jlkqdgg65h3phss8xa0mlp-python-2.7.10/bin/python
Python 2.7.10 (default, Jan 01 1970, 00:00:01)
[GCC 4.9.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

New syntax:
with import <nixpkgs> {};

rec {
  image1 = dockerTools.mkTarball { drv = bash; };
  layeredImage1 = dockerTools.build { name = "bash"; addTarball = image1; };
  image2 = dockerTools.mkTarball { drv = python; };
  image3 = dockerTools.mkTarball { drv = perl; };
  bash-python = dockerTools.build { name = "bash-python"; fromImage = layeredImage1; addTarball = image2; };
  bash-python-perl = dockerTools.build { name = "bash-python-perl"; fromImage = bash-python; addTarball = image3; };

  just-python = dockerTools.build { name = "just-python"; config = { Cmd = [ "${python}/bin/${python.executable}" ]; }; };
}

But now we also support arbitrary config and its dependencies! Look just-python:
$ nix-build -A just-python
# docker load -i result
# docker run -t -i just-python
Python 2.7.10 (default, Jan 01 1970, 00:00:01)
[GCC 4.9.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

We basically have a partial docker build working. Other stuff like handling multiple images ecc. are not working of course.

I'm going to merge this in a few days, unless somebody is contrary.

@lethalman would be really awesome to have this in nixpkgs docs

@domenkozar of course, I still have some ideas to make this interface even more pleasant... once finished I will write docs.

:cake:

Holy crap, composable docker builds reimplemented with Nix in <100 lines of Nix expressions. They look really easy to use. This is really cool work @lethalman!
I wonder how well hydra could do as a container build infrastructure. I worked at a company that was scaling up a containerized deployment process and their container build system was built from scratch. It later became a bottleneck since it wasn't distributed. Using Hydra's distributed builds could have solved that problem and they wouldn't have even had to write the build infrastructure that they did.
Might even be able to plug a deployment framework up to Hydra to ship the built containers to a cluster.

@trishume and the advantages:

No need to worry about the "context"
No need to chain RUN && && && && to the death, nix only does one layer per build.
It takes care of making an actual diff with the base image.
As usual nix, you don't need to sort out dependencies manually.
No need to mess with your system docker.
Smaller, even smaller when packages will have proper multi outputs.

There is a problem with this approach however... it's not possible change special permissions to files or change ownership.
I think for that case runInDockerVM must be used :(

Can this approach control exposed ports and control environment variables like a Dockerfile? If the answer is "not yet but possible" that can definitely be a future addition, I don't want to make it sound like I don't think this should be merged yet.

This is defintelly interesting and +1 for merging this, but in our deployments we are still planning to take another approach with distributed and shared nix store, where you don't even need any image, docker registry and other components, you just run software you have in nix store and that's it.

@trishume exposed ports can be always controller with docker run -p ...

@trishume this is already possible, see the config variable.
@offlinehacker of course, as long as you don't have to deal with foreign cloud environments that's the perfect solution ;)

This will be a nice solution for those (me included), who have already docker deployments set up, but would like to try transition to nix based builds. Once all builds are nix'd, moving away from docker is easier.

Ok so this is a new example:
buildImage { name = "python-link";
                     run = ''
                       mkdir bin
                       ln -s ${python}/bin/${python.executable} bin/python
                     '';
                     config = { Cmd = [ "/bin/python" ]; }; };

The run script will be converted as derivation, its deps will be included (like for the config) and the contents of the run script will be inside the image. Just convenience.
So it should be complete. Accepting reviews before merging, like missing use cases ecc..

Alright... I've finally managed to get something from the docker library hub written in nix. I started from something simple, redis.
So this is the dockerfile: https://github.com/docker-library/redis/blob/8929846148513a1e35e4212003965758112f8b55/3.0/Dockerfile
And this is our nix expression: https://github.com/NixOS/nixpkgs/blob/f5412cfc12d93c4bf9a8750bba90e4fcd7f3552f/pkgs/top-level/docker-library.nix#L29
Also it's possible to redis.override { fromImage = utils; };, which makes it easy to create containers for production or for development.
That opens up to pkgs/top-level/docker-library.nix, hence pkgs.dockerLibrary. However we probably won't enable those for hydra.
Note the runAsRoot runs in a linux vm, so it requires kvm, and uses overlayfs, which is the only simple "union" fs available by default in our kernel. It's still not complete as the instrumentation doesn't take in account deleted files, but that's easy to do.
TODO:
[ ] - Short term: use tarsum instead of our nix-hash. Our nix-hash works on directory but we need to sum the layer.tar.
[ ] - Short term: flatten an image to a single tarball, i.e. docker export -> flattenImage.
[ ] - Long term: add instrumentation to manipulate tarballs. Most of docker impure commands are useradd, creating root files, ecc., which with some tooling they may be done by tweaking directly the layer.tar instead of the filesystem, thus avoiding even the kvm, making it fully pure.
That opens up to pkgs/top-level/docker-library.nix, hence pkgs.dockerLibrary. However we probably won't enable those for hydra.

After some thinking, I believe it's better to avoid adding a docker library in our nixpkgs. If people is interested, it can be done in a separate repository. Time to write docs I guess...

@lethalman I'm very interested, I've wanted exactly this for a long time and you've done an amazing job :fireworks:
I think it would be great in nixpkgs but will follow along wherever it is.

So I added a fetchFromRegistry, too bad docker hub returns non-deterministic json metadata: https://github.com/docker/hub-feedback/issues/488
Hope they will tell me that only happens with empty tars so we may add some workaround...... :(

Done documenting the API. There are some hidden features but are not important to document for now.

The docs are awesome!
A couple questions I have that I couldn't answer from the docs:

If I use Nix to build a 3GB image and then 3 independent 10mb layers on top of that image, will it use more like 3GB + 3x10mb or 4x3GB + 3x10mb. I.E is the base image copied in the nix store when you add a layer to it?
Is there any reason I would want to layer on top of a Docker Pull or is it more for whole containers. If I take a debian image and then add Python from Nix as a layer, won't none of the Debian stuff get used since the Nix Python will include and use its dependency closure? I definitely see the utility for things like pre-built redis containers though.

These questions may well be answered, but I might not have read hard enough.

It will use space in the store once for the base image, then temp space for unpacking, the new layer in the store and the result in the store. All this is probably still less than what's required by docker. All can be garbage collected after you get the result.
For example you want a debian as base and nix software, because you may want debian tools and environment or some apt packages. Or some base fancy openstack image, wordpress, rails, anything that is not in nixpkgs or hard to package, then layer your app on top of it. Because your sysadm knows debian and provides base debian images... Whatever.

Oh I see, so you could use a base container like a fancy apache java servlet whatever with monitoring and log forwarding and all sorts of stuff. Then you can build your java app with nix and layer that on top, then just use a shell script to connect your Java app built with Nix up to whatever servlet thingy comes with the container.
The key being that the Nix layer won't interact with anything in the base layer on its own, but base layer services can still run, and you can use a build script to connect up your nix build to things present in the base image.
It might be helpful to have an example use case in the docs. For the purpose of marketing how cool and useful this stuff is.

I would also like to see this land in the main nixpkgs repo.

also, is it still possible to create multiple layers? It at least seems like mergeLayers doesn't exist any more, and your last syntax example is only a single layer.

@anderspapitto every buildImage adds one layer. Hence 5 buildImage, 5 layers. While docker adds a layer for each Dockerfile command.

Very nice work :cake:
I'm also :+1: to add this to nixpkgs directly.

one extra thought - would it be possible to avoid the expensive filesystem diffing in some cases, for example if the final layer only changed the 'config' attribute, and didn't introduce any new deps? currently it's a little painful to play around with, since changing an exposed port or a command line flag is a minute+ operation

@anderspapitto note that config may as well introduce dependencies (forgot to mention in the docs). However if the dependencies tarball is empty, yes we can avoid the diff only in that case.

Is there  proper way to add file to image? I want to create container with Nginx with my nginx.conf:
with import /home/andrey/development/nixpkgs {};
with dockerTools;

let
    nginxConf = writeText "nginx.conf" (builtins.readFile ./nginx.conf);
in

buildImage {
  name = "nginx-container";
  contents = pkgs.nginx;

  config = {
    Cmd = [ "nginx -c ${nginxConf}" ];
    ExposedPorts = {
      "80/tcp" = {};
    };
  };
}

Image is successfully created, but when I run it, I got an error:
docker run -t -i nginx-container
exec: "nginx -c /nix/store/31hslif7vdy2hmc43wagc3nnhh6lzicj-nginx.conf": stat nginx -c /nix/store/31hslif7vdy2hmc43wagc3nnhh6lzicj-nginx.conf: no such file or directory

try contents =  [pkgs.nginx nginxConf ];

@couchemar that seems like a bug... will investigate thanks.

@couchemar ah no it's correct, Cmd is an array of arguments... split it

@lethalman Thank you! This works now.

It seems that runAsRoot does not knew about export:
with import ../. {};
with dockerTools;

buildImage {
    name = "shadow-basic";

    runAsRoot = ''
      ${shadowSetup}
    '';
}

$ nix-build
these derivations will be built:
 /nix/store/ka6jpv3n41lxrp1lbc7qaai12211z3xs-docker-layer.drv
 /nix/store/d670hzfpy5kq9pmw0i60s1y2m0flxfgi-runtime-deps.drv
 /nix/store/l4ndcghxf6g76lxyyd6awyay9mppw3mx-shadow-basic-deps.tar.gz.drv
 /nix/store/khnrm6gw570lvcp19c01xlbxp0lshl86-shadow-basic.tar.gz.drv
building path(s) ‘/nix/store/6nmabpmblgj0r0hsd1ii9bxff2pk2pgn-docker-layer’
Formatting '/nix/store/6nmabpmblgj0r0hsd1ii9bxff2pk2pgn-docker-layer/disk-image.qcow2', fmt=qcow2 size=1073741824 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
loading kernel modules...
mounting Nix store...
mounting host's temporary directory...
starting stage 2 (/nix/store/yfl34z6w8s3c54b9994564alsfggqxwp-vm-run-stage2)
mke2fs 1.42.13 (17-May-2015)
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: 38e0b905-6667-43dd-a9cb-dc63d1dd1e27
Superblock backups stored on blocks:
   32768, 98304, 163840, 229376

Allocating group tables: done
Writing inode tables: done
Writing superblocks and filesystem accounting information: done

chroot: failed to run command 'export': No such file or directory
[    6.434359] reboot: Power down
builder for ‘/nix/store/ka6jpv3n41lxrp1lbc7qaai12211z3xs-docker-layer.drv’ failed with exit code 127
cannot build derivation ‘/nix/store/khnrm6gw570lvcp19c01xlbxp0lshl86-shadow-basic.tar.gz.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/khnrm6gw570lvcp19c01xlbxp0lshl86-shadow-basic.tar.gz.drv’ failed

@couchemar put #!/bin/sh on top...

@lethalman still error...
$ cat default.nix
with import ../. {};
with dockerTools;

buildImage {
    name = "shadow-basic";

    runAsRoot = ''
      #!/bin/sh
      ${shadowSetup}
    '';
}

$ nix-build
these derivations will be built:
  /nix/store/2s97hmi0x61mdl4mv1kh1kckxv4p5rqx-docker-layer.drv
  /nix/store/mfakz29ymvdhcklpzdsb6rqx3c68wvj4-runtime-deps.drv
  /nix/store/s14lds43pa7gvvm32pbbz03x1bc0nzxq-shadow-basic-deps.tar.gz.drv
  /nix/store/jx7bs22w4fnamm71nb7bcw27g9g3ampm-shadow-basic.tar.gz.drv
building path(s) ‘/nix/store/2kkfz3nczbsrqayp11l58k1jsjy47jk1-docker-layer’
Formatting '/nix/store/2kkfz3nczbsrqayp11l58k1jsjy47jk1-docker-layer/disk-image.qcow2', fmt=qcow2 size=1073741824 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
loading kernel modules...
mounting Nix store...
mounting host's temporary directory...
starting stage 2 (/nix/store/yfl34z6w8s3c54b9994564alsfggqxwp-vm-run-stage2)
mke2fs 1.42.13 (17-May-2015)
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: ccb2d9d4-9711-44a5-af94-be074370afff
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376

Allocating group tables: done
Writing inode tables: done
Writing superblocks and filesystem accounting information: done

bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
bash-4.3# exit
/nix/store/h84kc62s87mknjrln5rd6s332zfxfrk5-stdenv/setup: line 867: /etc/pam.d/other: No such file or directory
[    6.408975] reboot: Power down
builder for ‘/nix/store/2s97hmi0x61mdl4mv1kh1kckxv4p5rqx-docker-layer.drv’ failed with exit code 1
cannot build derivation ‘/nix/store/jx7bs22w4fnamm71nb7bcw27g9g3ampm-shadow-basic.tar.gz.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/jx7bs22w4fnamm71nb7bcw27g9g3ampm-shadow-basic.tar.gz.drv’ failed

ah ok that's different @couchemar will fix it

fixed and rebased on current master

@rvl ok the example was misleading because runAsRoot required directly an executable. Now it can be a string and your code should work.

@anderspapitto optimized the diffing part, but note the most expensive operations are due to unpacking and repacking the whole image.

Thanks @lethalman, it works well. Is there anything we can do to help getting this merged?
I reviewed the code (or tried to). The only questions I can come up with are:

how long will the docker hub registry /v1 continue to work?
is there any way of calculating a timestamp for a derivation which isn't 1970?
is the result let-binding needed in buildImage? (seems to work without it)
is an end-to-end test possible? ... e.g. build an image, load into docker, check that it runs.

@rvl thanks for testing.

Well, they don't have a v2 so... :)
Derivations have to be reproducible. You have to think they are addressed by hash, not by time. That's for any derivation in nix, not only docker images. I understand that's ugly, it's as ugly as running uname on NixOS.
It's handy just in case we need to tweak the result further. If in the future you have to re-indent, that would break the blame.
Yes, it's possible with runInLinuxVM but also like with any other nixos test. There's even a docker test already for nixos.

As for the last bits todo:

Allow contents to be a list.
Currently in some code paths, when copying contents, there's a chance it fails due read-only files of different nixbld users. It should suffice to do a chown -R a+w after copying stuff.

I should have addressed the last bits too, and the docs about the default values for pullImage.
I've rebased on top of current master. Asking for some other testing now, then I'll merge.

Great, it still works for me. A list of derivations in contents also works for me. Can I suggest rewording the sentence in doc/functions.xml:405 to "contents is one or more derivations that will be copied in the new layer of the resulting image."

@rvl thanks for testing. How should it be reworded?

Instead of "contents is a derivation that will be copied in the new layer of the resulting image", try "contents is one or more derivations that will be copied in the new layer of the resulting image." I think that would suggest using a list if necessary.

What's the status of this?

@jgillich mergeable. I'm going to rebase, retest and merge right now.

Awesome! :dancers:

Tested pullImage, sha is almost always wrong. I think docker changed again something :(

Nvm it's now somewhat consistent. I guess there's still some non-determinism somewhere in pullImage, but we'll see with time.

Merged. No reason to keep this unmerged, it can be improved over time and already does a very good job I think.

:1:

Wow, this is pretty awesome looking!

@lethalman Would you know, is something special required for hydra on nixos to be able to build image with runAsRoot (requiring kvm)? Images without buildAsRoot build fine. Images with runAsRoot build fine as a user, but not by hydra (on the same machine).

@lethalman Sorry for bothering. It seems that supportedFeatures = [ "kvm" ]; for nix.buildMachine did it.


Update Function for Production Deployment

This PR updates the lambda for deployment to the production environment. Changes include:

Use hard coded directory names and changed success_folder and error_folder to save_succeeded and save_failed respectively. These values now represent booleans to determine if the input should be saved.
Remove checking object name in lambda as it will be handled by the lambda's trigger instead.
Reduce document validation down to the existence of two required fields instead of checking all fields against a list of allowed fields.
Streamline database insertion into one method call.
Track the number of documents actually processed.

We went over this in person on Wednesday so it was an easy review. Thank you, Nick.


If the order has an error with the blockOrder flag, the checkout confirm button remains enabled and clickable

PHP Version
8.0
Shopware Version
6.4.14
Expected behaviour
If the order has an error with the blockOrder flag, the checkout confirm button must be disabled.
Actual behaviour
Button remains enabled and clickable
How to reproduce

set phone number as mandatory field in admin
select the billing address without phone in checkout
flash error is displayed, but confirm button remains enabled

see https://github.com/shopware/platform/commit/bec6ac21a78ab84c7181dac2347e3e670f92033c#commitcomment-81208122

See here, please vote there. https://issues.shopware.com/issues/NEXT-22205

I will close this as its duplicate to https://github.com/shopware/platform/issues/2542


Theme UI / Research Project

@crystaljuarez3's research methods cards
due to updates in Node, Reakit's grid styles weren't being built
implemented Theme UI
replaced all Reakit components (most were Grid) with Theme UI's Grid component
updated a bunch of packages

I definitely got carried away — have been wanting to add Theme UI to the site for a long time, just haven't prioritized it.
Ran into some unsolvable issues after upgrading Node, I think it broke Reakit, seemed like the styles were getting removed at the build phase and wasn't able to figure out why.
So I added Theme UI, which required rewriting everything that used Reakit's Grid component...which was a lot. But it makes future development way, way easier, faster, and more consistent.


101: More details for SoA vs AoS

Hey,
I read the comments in 101, and I am having trouble understanding the very last bit:
https://github.com/ratfactor/ziglings/blob/9d6f1e2f3656b10cb09fef23c165f9def0a5cd4c/exercises/101_for5.zig#L117-L121
I feel like I am missing an example for the last bit.
I understand the Array-of-structs part, as it should be something like []Character with:
const Character = struct {
 gold: u32 = 0
 health: u8 = 100,
 experience: u32 = 0,,
}

from exercise 51.
And, as mentioned in the comments, it is something that feels natural to me.
But now, how would the struct of arrays be implemented or rather used?
I can see in the code above, what is meant by having multiple arrays for each data type, but I am unsure about how this would be actually used.
I also find this:

versus a single RPG character struct containing three arrays of one data type each

confusing. Shouldn't it be a single RPG characters struct ?
As far as I understood it, the idea is to use a single struct for all RPG characters and use arrays for each type and imply that a single character just means that the length of the arrays is 1, or put differently, that the single character case is a special case of the Characters struct (if this makes any sense :D )
IMHO, it would be nice if the comments were more explicit and show the Character struct + array of the Character struct again for an explicit example of the AoS case and an example for the SoA case.
Maybe even with a function that prints the latter?
I think with a calling/function example, I would have an easier time understanding how data-oriented design would be implemented.
Sorry for the long issue, I am just trying to convey what I (an absolute zig n00b) have trouble with and hope that it might help others in the future. :)

I think it's very difficult to explain the principles of Zig's "data-driven design" in an exercise, but luckily there's a good video by Andrew Kelley that I can recommend: https://vimeo.com/649009599
In the case of the exercise, it is faster and uses less memory to use the arrays this way because the indices of the arrays are paired instead of address pointers for each value. But honestly, I don't know how we can describe it better. Maybe after watching the video you will have a good idea. :wink:

Since then, I have also watched the same video :D
Yes, it helped quite a bit, wrapping my head around it.
But maybe it would be nice to add the link to the exercise?
Though I can also understand if this seems "out of place".
Either way, thank you for responding :)

But maybe it would be nice to add the link to the exercise?

Good idea I will see how I can implement it.

I opened a pull-request on how I would shortly introduce it :)
Feel free to close it if you would like to phrase it differently.


Make :* result in kissing_heart instead of kiss

Currently, :* shows up as 💋 . However, experts believe it should show up as 😘 . This fixes that discrepancy.

maybe the most important change in our lifetimes


Configuration page WSOD when Tomcat is down

If Tomcat is down when visiting the general configuration page '/admin/config' will WSOD and report to the log GuzzleHttp\Exception\ConnectException: cURL error 7: Failed connect to <IP_ADDRESS>:8080; Connection refused (see http://curl.haxx.se/libcurl/c/libcurl-errors.html) in GuzzleHttp\Handler\CurlFactory::createRejection() (line 185 of /var/www/html/drupal/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php)..
To be honest, I don't know why the general configuration page crashes when Tomcat is down, but it does.
To reproduce:

Spin up a box
Visit `http://localhost:8000/admin/config' which should load fine.
vagrant ssh
sudo systemctl stop tomcat8
Visit `http://localhost:8000/admin/config' which will WSOD.
Visit the recent log messages page (http://localhost:8000/admin/reports/dblog) and see the error listed.

Resolved with https://github.com/Islandora/islandora/commit/2a1024c19a4b495c64fb88d4260e8d7ebea1f9b5


feature: 支持预cache secgroup到云上

这个 PR 实现什么功能/修复什么问题:

支持预创建安全组，或单个删除云上安全组
优化cache，支持云上删除的安全组，同步时自动清理cache
优化同步安全组功能，避免因使用颗粒度高的同步函数
支持删除账号时同时删除账号对应的安全组缓存
安全组删除时，状态跟随变化

是否需要 backport 到之前的 release 分支:

release/2.12

/area region
/cc @swordqiu @yousong @tb365

Codecov Report

Merging #3281 into master will decrease coverage by <.01%.
The diff coverage is 0.62%.

@@            Coverage Diff            @@
##           master   #3281      +/-   ##
=========================================
- Coverage    7.05%   7.04%   -0.01%
=========================================
  Files         482     482
  Lines       78513   78604      +91
=========================================
+ Hits         5538    5540       +2
- Misses      72513   72603      +90
+ Partials      462     461       -1

Flag
Coverage Δ

#aFlag
7.04% <0.62%> (-0.01%)
:arrow_down:

Impacted Files
Coverage Δ

pkg/compute/models/regiondrivers.go
20% <ø> (ø)
:arrow_up:

pkg/compute/guestdrivers/azure.go
8.19% <ø> (+0.62%)
:arrow_up:

pkg/compute/models/guestdrivers.go
20% <ø> (ø)
:arrow_up:

pkg/compute/guestdrivers/base.go
0% <ø> (ø)
:arrow_up:

pkg/compute/guestdrivers/openstack.go
7.46% <ø> (+0.21%)
:arrow_up:

pkg/compute/models/guests.go
0.63% <ø> (ø)
:arrow_up:

pkg/compute/guestdrivers/qcloud.go
3.87% <ø> (+0.05%)
:arrow_up:

pkg/compute/models/secgroups.go
2.18% <0%> (-0.16%)
:arrow_down:

pkg/cloudprovider/fakeregion.go
0% <0%> (ø)
:arrow_up:

pkg/compute/guestdrivers/managedvirtual.go
0% <0%> (ø)
:arrow_up:

... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8e04c66...7e923d9. Read the comment docs.

/lgtm

/approve


fix: close chrome gracefully with Browser.close

browser.close() now closes the browser, regardless of whether or not it was created with puppeteer.launch() or puppeteer.connect()
#989
browser.disconnect() is a new method which disconnects Puppeteer from the browser, but does not close the Chromium process.
Should fix #921 and #918

@vsemozhetbyt @elisherer does this work with your Windows setups? Mine is pretty bizarre.

Very cool. Thanks for the quick fix on this one!

This seems to fix https://github.com/GoogleChrome/puppeteer/issues/918, but Ctrl+C seems not to work:

'use strict';

const puppeteer = require('puppeteer');

(async function main() {
  try {
    const browser = await puppeteer.launch({
      headless: true,
      userDataDir: 'test-profile-dir',
    });
    // await browser.close();
  } catch (err) {
    console.error(err);
  }
})();

If I type Ctrl+C once, nothing happens. If I type Ctrl+C second time, node exits, but Chromium remains.

This DOES NOT fix https://github.com/GoogleChrome/puppeteer/issues/921: headless Chromium still does not preserve authorization (does not send cookies?).

to fix #918:  in the lib/Launcher.js find this spawn crome line:
const chromeProcess = childProcess.spawn(chromeExecutable, chromeArguments, { detached: false });
set detached: false

#921:
In the headless mode, chromium can't save storage and cookie. But in the normal mode, it can.
I think it's chromium bug.

It appears to be working with localStorage but not with cookies. I'll look into it further.

I will wait this one, can we expect Browser.close to return boolean? Currently i check it just return empty object.

@brutalcrozt Browser.close tries to close the browser, but it doesn't make any guarantees about when or if that will happen.

I subscribe to this PR, i was thinking that i will get notified when it got merged. But it wasn't :confused:

ok I'll subscribe also


setup return url setup

use return url from configs

Thank you @stojankukrika - merged. Will create a new release shortly.

Released :)


Convert deploy into a floating button/Modal

Description
With the current design of the deploy flow, we had 2 issues:

Needing to move back to the Dashboard page to deploy was a pain point for users
No information provided on how we deploy and what the deployed sit url is

This PR fixes these issues by creating a floating button for deploy that is available on all pages (that affect the deployed site). Upon clicking the user sees a Modal with more information. I have also combined the Login with gitHub and Deploy with GitHub button into just one, reducing the number of clicks required.
Also included in this a minor fix:

When deploying site with no aboutUs description, the deployed code will fail, I've made it so that the attribute is defaulted to an empty string.

Type of change

[ ] Bug fix
[X] New feature
[X] Refactoring
[ ] Documentation
[ ] Improvement

Tests
All unit tests pass?
Yes
Builds successfully?
Yes
Do all SA checks pass?
Yes
Attach screenshot of the expected result (e.g. UI of the website/ postman api call result/ database changes)
Deploy Modal:

Floating button:

Provide steps in detail for reviewers to reproduce the expected result of the code changes.

Login and try deploying the site from different pages.
Try using the url shown on the Deploy Modal.

Risk
medium, may impact deployment functionality

I love the fact that deploying is more streamlined! But I think there's a few places where the logic can be improved:

It looks like you're doing the access token exchange every time the user clicks on deploy - that is, we get a new access token each time and you can see the new tab that takes you to the GH login page. We should be caching/storing the initial access token on the frontend (like we were doing in localStorage) because:

It would save us an extra call to GH API for the access token every time we want to deploy (also the pop up is annoying)
We're lucky that GH doesn't have a TPS limit for the exchange flow, but it's good practice to cache anyway for situations when you encounter something that does have a TPS limit

Once the above is done, we can think about only showing the deploy modal the first time (which will prompt the login) and replace the hovering button with the spinner? This can be a later improvement though.

Also, the deploy button is unclickable on the homepage editor, possibly because it overlaps with the Twitter box.


[BUG]: OpenApiSpecGeneratorPlugin generated docs don't appear valid

Description
The description document created using the OpenAPI spec generator plugin doesn't appear to create valid documents as it's using an invalid data type. The generated document doesn't pass multiple validators (such as this one and this one).
Expected behaviour

Update the Dev Proxy config to add support for the OpenApiSpecGeneratorPlugin.
Record a request using the following URL: https://api.nasa.gov/mars-photos/api/v1/rovers/curiosity/photos?api_key=DEMO_KEY&sol=1000&camera=MAST
Load the generated JSON file generated by the Dev Proxy into one of the validators - it should pass.
Using the VSCode Teams Toolkit, create a new project using the following options:

Message Extension > Custom Search Results > Start with an OpenAPI Description Document
When prompted, select the JSON file generated by the Dev Proxy.
Select the only path in the description document & let the TTK wizard complete the project creation process.

Actual behaviour
When repeating the process above...

when validating the JSON file generated by the Dev Proxy, it fails
when selecting the JSON file generated by the Dev Proxy during the process of creating a new project using the TTK, it fails and displays errors in the terminal:

Summary:
1 failed.
(×) Error: No supported API found in the OpenAPI document.
For more information visit: "https://aka.ms/build-api-based-message-extension".
Reasons for API incompatibility are listed below:
GET /mars-photos/api/v1/rovers/curiosity/photos: response json is empty

Steps to reproduce
Noted above in Expected behavior process
Dev Proxy Version
0.19.0
Operating system (environment)
macOS
Shell
zsh
Configuration file
{
  "$schema": "https://raw.githubusercontent.com/microsoft/dev-proxy/main/schemas/v0.19.0/rc.schema.json",
  "plugins": [
    {
      "name": "RetryAfterPlugin",
      "enabled": true,
      "pluginPath": "~appFolder/plugins/dev-proxy-plugins.dll"
    },
    {
      "name": "GenericRandomErrorPlugin",
      "enabled": true,
      "pluginPath": "~appFolder/plugins/dev-proxy-plugins.dll",
      "configSection": "genericRandomErrorPlugin"
    },
    {
      "name": "OpenApiSpecGeneratorPlugin",
      "enabled": true,
      "pluginPath": "~appFolder/plugins/dev-proxy-plugins.dll"
    }
  ],
  "urlsToWatch": [
    "https://jsonplaceholder.typicode.com/*",
    "https://api.nasa.gov/*"
  ],
  "genericRandomErrorPlugin": {
    "errorsFile": "devproxy-errors.json"
  },
  "rate": 0,
  "logLevel": "information",
  "newVersionNotification": "stable"
}

Additional Info
No response

Thanks for letting us know. I'll check it out asap

I just tried reproing the issue you mentioned. The generated spec validates fine in https://apitools.dev/swagger-parser/online/:

But validation fails in https://www.panoptica.app/tools/api-spec. Unfortunately, beyond saying that the spec file is invalid, it doesn't tell what's wrong so it's hard to know what to fix.
I can repro the issue with TTK. I'll reach out to the team to see why it's rejecting the spec and will share an update as soon as I know more.

Got a short update: it seems that TTK is looking for application/json while the generated spec includes encoding application/json; charset=utf-8. I'm going to find out more if it's something that should be adjusted in Dev Proxy or in TTK. Stay tuned

Bug fixed in v0.19.1 available on Homebrew


Responsive flex layout works abnormally

https://codesandbox.io/s/react-overflow-list-flex-error-u2b26?file=/src/App.tsx
When the current overflow container has no fixed size, the element will change the width of other containers

I have some dropmenu, overflow will affect them, is there any other way to solve it?

https://codesandbox.io/s/react-overflow-list-flex-error-u2b26?file=/src/App.tsx
The complete code of my project about the layout


What's the license type of dog-tunnel?

What's the license type of dog-tunnel?
Is it opensource project?
GPL ? MIT ? New BSD ? Apache?
If you have no idea, MIT license will be fine.

The project using MIT license, so this issue can be closed.


Clean up test boilerplate

This MR reduces the boilerplate code in the unit tests with some test utilities.

Approved by Kenny L.


Fixed form comments not converting to free form comments

VSCODE version: 1.70.0
vscode-rpglefree version:  0.0.23
Perhaps this warrants some discussion.
I have been trying out VSCODE in local development and building with BOB, mainly as a learning exercise. As part of the process I'm taking older RPGLE programs and

Creating a local copy in a VSCODE with BOB project
Converting them to free form (select all with ctl+a, right click, convert to free form)
Adding **FREE at the top
Reformatting based on linting rules.

One issue I have run across repeatedly is fixed form style comments will break the compile. These aren't touched by the converter, and seem to confuse the linter. My manual work around is to add '// ' at the beginning of each line - manually changing them to free form style. This allows the linter to work and it will then compile successfully.
Here is an example:

Were these comments converted with other code or just on their own?
Could you provide an example of the code before the conversion?

the comments at the top I manually converted by typing '// ' at the beginning of each line.  Sorry, should have noted that.
This is how the code originally looked (notice another bug where first line comment messes up source coloring for the rest of the document):

Same code in SEU (after hitting F19):

I'm not sure why in this source member that first string of asterisks start in column 1 instead of column 7, but the compiler doesn't mind it that way.  That's not related to this issue though.  NONE of the comments were converted by the converter to free form when I copied the source to a local file, ctl + A, then convert to free form on the right click context menu.

Doing a bit of testing I think the problem here is that, in the screenshots, the comments are the only lines which are not already free-form.
I've tried taking a fully free form program and swapping some of the comment lines back to fixed form, nothing happens.
If I add in a couple of fixed format lines and then convert the extension will convert the lines to free form comments.
Example that won't work:
       ctl-opt DATEDIT(*DMY) DFTACTGRP(*NO);
      ***-----------------------------------------------------------------
      ***
      ***   Program     :   Test program
      ***   Description :   Testing free format conversion
      ***  Author      :   David shears
      ***
If I ask it to convert this, the comments will remain untouched. If I then add the below (part of my testing for the C-spec pull request.)
     C                   MOVE      *OFF          *IN98
     C                   MOVE      *ON           *IN98
Then select all code again, the comments will be converted.
       ctl-opt DATEDIT(*DMY) DFTACTGRP(*NO);
        //**-----------------------------------------------------------------
        //**
        //**   Program     :   Test program
        //**   Description :   Testing free format conversion
        //**  Author      :   David shears
        //**
       *IN98 = *OFF;
       *IN98 = *ON;


Send window size initially

SetSize should only be done for interactive mode, pipes don't support the ioctl.
Also, please update rest-api.md

Doh, ignore that last one, I can't read apparently :)

It's unclear which of the two ioctl failed, whether it's the server when setting it or the client when getting it, but neither should be done in non-interactive mode really.


Migration script to separate credits from expenditures

https://folio-org.atlassian.net/browse/MODFISTO-482
Approach
For each

sum credit transactions in the budget fiscal fiscalYear
add to credits
subtract from expenditures

For each encumbrance transaction:

recalculate encumbrance.amountExpended and encumbrance.amountCredited by summing related payment and credit amounts (using transaction.paymentEncumbranceId to link the payments/credits to the encumbrances).

Query Result Execution time:
Initial Data:

Created data 15000 payment transaction 10000 credits transactoin

Setting wrong data 25000 expended amount for encumbrance and budget

Run budget update query

Result

Run encumbrance fix query

Result


Build long prefix package using conda-build 2.x

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Thoughts on this @conda-forge/dbus ?

+1 from me since it's building ;-)


ADDED "#331"

KINDLY CHECK @Kritika30032002

have u checked ?

Thanks for your Valuable contribution.
Don't forget to star the repo.

Yeah sure @Kritika30032002 😊


New bug reporting process

On Github?

Bug because the current button links to the old repo.
We talked about this in a meeting that there should be a link to "create issue" on GitHub on the repo dhcr-main.
So the new process will be that normal users have to create a GitHub issue.
The change of the link is also addressed here; https://github.com/acdh-oeaw/dhcr-main/issues/42#issuecomment-1465211487
Moved to mar23 milestone, since this work can be done together.

The possibility to create an issue on Github is (currently) only available after login to Github.

waiting for decision in meeting

Moved to Mai milestone to have a realistic estimation of when this will be implemented.

in case a user reports a bug, we can assume that he/she has a github account and makes use of github to report.
--> implement a link to DHCR github repo - where the user can write down the bug
alternative solution: implement 'bug-report' form on website - the message will be sent to<EMAIL_ADDRESS>we can discuss this in the next tech meeting

Talked about in the meeting today:
The link on the button "report bug" will be updated to create an issue in this repo.  https://github.com/acdh-oeaw/dhcr-main/issues/new
Explanation line will be added that users need to create a GitHub account, etc.
@vronk Can you add the explanation you've suggested here?

Blocked: Waiting for text that should be added as explanation

“Please use your GitHub account in order to file a bug report. In case you haven't created one, please create an account here (link). Alternatively, you can contact us via dhcr-helpdesk /at/ clarin-dariah.eu.”

Changed description to match the process:
" Click the button below to create a new issue and file a bug report. You'll need a GitHub account to do this. In case you don't have a GitHub account, you can create one. Alternatively, you can contact us via dhcr-helpdesk /at/ clarin-dariah.eu. "
Development finished.
Implemented in 2023-05.

@patrickakk Suggestion for the text:
Click the button below to create a bug report on GitHub. In case you don't have a GitHub account, you can create one here. Alternatively, you can contact us via dhcr-helpdesk /at/ clarin-dariah.eu.
No need to wait for Anna.

@IvdL22 The text has been updated, see https://dev-dhcr.clarin-dariah.eu/info#contact
Implemented in 2023-05.


⚠ Bofrost.at has degraded performance

In 53711c7, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 2203 ms

Resolved: Bofrost.at performance has improved in aa28726.


Completion pollution between switch cases

switch {
case true:
	foo: int
case false:
	fo // foo is in completions, even though it's undefined here
}

Fixed.


removed nested loop structure in favor of grep -f

The commit comment says it all.

I have been trying to do this without a temporary file, but could not find an elegant solution.

You know what?  I might have submitted this to the wrong branch.  I forgot that you had a feature branch set up for this.  Can you fix it without trouble or should I submit a new pull request?

I was just looking into what is wrong. I will fix it.


Emacs cleanup

I ended up doing quite a lot of changes, see commits for more details.
Setting up racer-mode should be more straightforward now:
(add-hook 'rust-mode-hook #'racer-mode)
(add-hook 'racer-mode-hook #'eldoc-mode)
(setq company-tooltip-align-annotations t) ; looks better

I also removed the dependency of company since completion-at-point is supported out of the box via company-capf.
Fixes #339.
Cheers :)

Hi @fbergroth. Thanks for this patch, really appreciated.
I haven't had a proper chance to test yet, but am merging so it will get picked up by melba and will test then.
Cheers, Phil


Does not work with PurePerms still.

When I do /rankup i recieve this message:
[Server thread/CRITICAL]: Unhandled exception executing command 'rankup' in rankup: strtolower() expects parameter 1 to be string, object given
[17:55:32] [Server thread/WARNING]: InvalidArgumentException: "strtolower() expects parameter 1 to be string, object given" (E_WARNING) in "/RankUp_v0.3.phar/src/rankup/rank/RankStore" at line 25

You aren't using the latest development version. The fix will be made available in 0.3.1.

So I should download the latest developmental version of pocket mine and it should work?

So I should download the latest developmental version of pocket mine and it should work?

@Nader6 I am referring to the latest version of this plugin, in which @64FF00 has fixed this issue.

I will download and test right now and give you my feedback @Falkirks.

Oh I am sorry, has it not been released yet?

I downloaded, but I do not know how to use it as a .zip file. I tried the ZipPluginLoader plugin but it didn't work. Sorry for all the trouble this is my first time being introduced to this.


Support "DROP DATABASE IF EXISTS"

This useful command would allow clients to idempotently drop a database. If IF EXISTS is present, no error would be returned if the database does not exist.
https://github.com/influxdb/influxdb/pull/3892 introduced CREATE DATABASE IF NOT EXISTS, so using that PR as a guide, along with investigating the current implementation of DROP DATABASE should make it relatively easy to implement this feature.

Hi @otoolep , can I have a try for this issue? I have create a pull request for it.
If someone is already working on it then just ignore mine ya

@ch33hau - email me your address so we can send you a tshirt. philip at influxdb

@otoolep Thanks for your Monday early good news! =)
I have sent my address to your email.


Fix assessment crashing on load

I believe this was caused by various updates in componentWillMount and componentWillUpdate. As such, they have been removed to use componentDidMount (as recommended by React). Also removed one instance of unnecessary currying (inconsequential)
Fixes #206.

Pull Request Test Coverage Report for Build 318

1 of 2 (50.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 26.043%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/components/academy/grading/GradingWorkspace.tsx
0
1
0.0%

Totals

Change from base Build 310:
0.03%

Covered Lines:
715

Relevant Lines:
2328

💛  - Coveralls


Fixes incorrect label colors under certain circumstances.

Fixes incorrect tint color occurring if label has a non-white setColor tint applied and label’s computePrefSize method called while label is in non-wrapping state or has ellipses set.
Problem with improper label colors occurring on non-white labels in a scrollpane when scrollbar appears has been traced to here. We shouldn’t ignore resetting color to white upon compute size, otherwise the cache font’s current color is used which, if not white, will not provide a proper base for tinting when drawn.

How can I reproduce the problem that this PR fixes?

Thank you for your interest. I am attempting to re-replicate the issue (my project has left that state behind) to determine if this is still an issue since recent changes in label and bitfonts may have solved it elsewhere. I'll respond here as soon as I have info.

This appears to have been fixed by you when you changed computePrefSize() to use Label.prefSizeLayout as its working reference instead of the main layout field. Nice move, and thanks.


python3Packages.georss-client: disable failing XML tests

Description of changes
Fix build (https://hydra.nixos.org/build/169483372)

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Not needed will be fixed by the expat update which is in staging.


security: Update information for CVE-2023-4262

This CVE was rejected after further analysis.

@henrikbrixandersen @kartben @nordicjm can you please take a look again ?


Basic peer discovery

This adds a basic peer discovery mechanism similar to Bitcoin's. When a node connects somewhere, it tries to load addresses from remote peers and also advertises its own discovered address. Such advertised addresses are relayed to random peers (two peers) and spread across the network this way.

File p2p/src/peer_manager/global_ip.rs is based on a source file from libp2p, that's why the copyright line is different there. BTW, I said earlier that Rust's stdlib has is_global method for IP addresses but actually it's behind unstable feature flag (and probably won't be stabilized any time soon). That's why we have global_ip.rs file.


Not Working parsing multipart file

I'm parsing the file multipart.eml in the sample folder and getContents() method resturn null.

For some reason I'm also getting null body of the message with IBM Notes emails. There is no text/plain parts in those messages at all and only text/html:
Content-type: multipart/mixed;
	Boundary="0__=A12E5F1C43DD85F28f9e8a93df938690918cA12E5F1C43DD85F2"
Content-Disposition: inline

--0__=A12E5F1C43DD85F28f9e8a93df938690918cA12E5F1C43DD85F2
Content-type: text/html; charset=UTF-8
Content-Disposition: inline
Content-transfer-encoding: base64

Stale.


Admin List Users

Hi!
Is it possible to list users?
With Administration we can create, delete, modify users ... but cannot see how to get a list of users

This is now documented: https://github.com/delight-im/PHP-Auth/issues/201


No release to download from

No release to download from, to be able to manually add mod to game.

Just saw that it can be downloaded from https://nasb.thunderstore.io/package/Bobbie/AltSkins/


Adds a check to ignore pods which are marked for deletion

What type of PR is this?
/kind bug
What does this PR do / why we need it:
It adds a check to ignore pods which are marked for deletion.
Which issue(s) this PR fixes:
Fixes #4655
PR acceptance criteria:

[X] Unit test

[ ] Integration test

[ ] Documentation

[ ] I have read the test guidelines

How to test changes / Special notes to the reviewer:

Create a storage and push the changes to the component odo storage create && odo push
Delete the component and then list the storage odo delete -f && odo storage list.

Upon trying this, I see that the storage attached to the component is showing up as Not Pushed in odo storage list. The issue talks about not showing the storage at all, but Not Pushed seems like the right behaviour to me.

Yes Not Pushed is the right behavior because the devfile.yaml and the env.yaml still exists for the component.

However, the unit test added here doesn't test the behaviour of odo storage list command that we're trying to address here. Can we please add an integration test for the same?

Sure but there is no guarantee that the integration test would be able to test this properly as the pod needs to be marked for deletion but should exist on the cluster. If the pod is deleted quickly on the cluster, the test won't be able to catch it.

Sure but there is no guarantee that the integration test would be able to test this properly as the pod needs to be marked for deletion but should exist on the cluster. If the pod is deleted quickly on the cluster, the test won't be able to catch it.

I was thinking more along the lines of checking odo storage list output. Kinda covering what's mentioned in the original issue.

I was thinking more along the lines of checking odo storage list output. Kinda covering what's mentioned in the original issue.

I meant even if I check the list output, there is no guarantee that the integration test would be able to test this properly.

I meant even if I check the list output, there is no guarantee that the integration test would be able to test this properly.

Really? Why? It's a genuine question because when I played with the functionality locally, it worked just fine.
If you think it'll take a few seconds for the output to reflect correctly, we could use helper.WaitForCmdOut.
Just to be clear, I don't mean that I am not going to LGTM without an integration test. I'm just trying to understand why it would be unreliable.

Really? Why? It's a genuine question because when I played with the functionality locally, it worked just fine.
If you think it'll take a few seconds for the output to reflect correctly, we could use helper.WaitForCmdOut.

If the pod is deleted quickly before we execute list, the output would be Not Pushed even without this fix and helper.WaitForCmdOut would exit after the first check. We need the pod to be marked for deletion but existing on the cluster.
But since it worked for you locally, I guess we can ignore the above and add a integration test. Let me find a good place to keep the check.

We need the pod to be marked for deletion but existing on the cluster.

I think this part is where my understanding is different from yours. In the integration test, I was only considering checking output of odo storage list without being concerned about the state of the pod. Pod is indeed getting deleted fast.

But since it worked for you locally, I guess we can ignore the above and add a integration test.

I used springboot example - odo create java-springboot --starter. It showed the pod to be in Terminating state for some time.

@dharmit I have added the integration test. Please have a look.

/retest


Ensure that Until nodes expire as expected for Credit Default Swaps

Seems that Until nodes do not get pruned when the condition has been met. This is the case for both Time based (TimeGte) and Observation based (Lte) Until nodes.
In the contingent claims repo I noticed a function expire that seems to do this:
https://github.com/digital-asset/contingent-claims/blob/master/daml/ContingentClaims/Lifecycle.daml
I cannot find this being used in Daml Finance at the moment.
We should think about the what process we want for (automatically?) expiring Until nodes when the condition has been met.
The order of the processing is also important, especially in this kind of claims statement:
creditEventClaims = until creditEvent $ when creditEvent $ payoff
in order to achieve the expected result (if a credit event happens: first pay the payoff and then expire the product)

https://github.com/digital-asset/contingent-claims/pull/87 is the ticket that implements this

I will close this. The expiry item is tracked in https://github.com/digital-asset/daml-finance/issues/231


App static layout structure (plus...)

Addresses ticket 175
🚧 Changes
✨ Components:
- Add client\middleware.ts for request routing management
- Add client\middlewares\ * files for task specific functions applied to a request (TBD)
- Add client\app\auth\signin\page.tsx for login route (TBD)
- Add client\app\components\theme* for MUI custom theme with Next.js 13 app router
- Modify client\app\layout.tsx to use ThemeRegistry
- Modify client\next.config.js for MUI imports
- Add client\app\components\layout\Header for static app layout
- Add client\app\components\layout\Footer for static app layout
🧹 Cleanup:
- Remove Tailwindcss configurations files
- Modify package.json to resolve dependancy warnings
- Remove client\app\styles\global.css
- Remove client\app\styles\layout.module.css
- Rename client\app\utils\actions\operationSubmitHandler.ts to utils\actions.ts, as per Next.js convention
- Add client.env\API_URL variable for API requests URL
- Add client\app\utils\api.ts fetch utility function to handle API requests and error catching
- Recode fetch calls to hardcoded API to use utils\api.ts fetch utility function, as per DRY convention
🏗 Redesign:
- Delete client\app\components\loading.tsx for suspend loading effect
- Add client\app\components\loading\SkeletonGrid.tsx for suspend loading effect
- Add client\app\components\loading\SkeletonForm.tsx for suspend loading effect
- Add client\app\components\routes\operations\Operations.tsx (contains async components from client\app\operations\page.tsx)
- Recode client\app\operations\page.tsx to wrap asynchronous client\app\components\operations\Operations.tsx using
- Add client\app\components\routes\operations\form\Operation.tsx (contains async components from client\app\operations\create\page.tsx)
- Recode client\app\operations\create\page.tsx to wrap asynchronous client\app\components\operations\form\Operation.tsx using
🚀 Impact:

Material UI configured with the Next.js App Router
Initial static layout structure (rudimentary and non-functional- refinement TBD)
Middleware request routing architecture in place for future requirements of request rewriting, redirecting, or modifying before a response
Re-architecture of app router files for wrapping asynchronous components in page routes to improved user interaction.
Ex:
Route http://localhost:3000/operations displays page.tsx which displays none dynamic elements and uses  with a loading skeleton to indicate waiting for the async data components

📝  Notes:

Added a temp latency simulation on async data to emphasize the user interaction with  skeleton loading

🔬 Testing:

Open terminal in cas-registration\bc_obps directory
Follow the developer environment setup to start the API
Open a terminal in cas-registration\client folder
Install changes to the packgae.jsonyarn

Start the development server:yarn dev

In your web browser, navigate to http://localhost:3000/
You should see "navigation" page with MUI header and footer

Click link "Operations"
You should see "Operations List" page with loading skeleton

Once loading has completed you should see list of operations
Click button "Add Operation"
You should see "Create a New Operation" page with loading skeleton

Once loading is complete you should see an empty form
Complete and submit form
A success message should display
Click the back button
New record should display
Click the "View Details" on a row
19.. You should see "Edit an Operation" page with loading skeleton

Once loading is complete you should see an form with data from the selected row.
Edit submit form
A success message should display
Click the back button
Edits should display

Deleted auth demo folder with error.jsx page as SonarCloud Code Analysis failed with:
Do not use "Error" to declare a function - use another name.
export default function Error() {
Need to determine how to handle Netx.js error.jsx with SonarCloud Code Analysis

Also, just thought of it, are you planning to take out the delay?


Align tool: wrap / nowrap / reverse / forward

Would be great to reverse and wrap. curious what hotkeys we could do or visual buttons we could show? thoughts?

Hi @argyleink
I would like to contribute to this project. Can I take this issue?
My thoughts on hotkeys:
We can make ctrl + left or right to cycle through properties or we can make individual hotkeys for each property, e.g. shift + w for wrap etc..
What do you think?

Hi! you can definitely take this issue 👍
Here's the current hotkey set to help us ideate:

we have 4 actions to map, split into 2 groups of similar behavior (wrap/order)
could we map them to cmd/ctrl + shift, then have up/down be order and left/right be wrap?

cmd+shift+up: nowrap
cmd+shift+down: wrap
cmd+shift+left: reverse
cmd+shift+right: forwards

up/down change wrapping on the container
left/right change order
thoughts?

Sounds good, I like the idea of splitting them.

Rad 🤘
Here's the steps I'd take (please ask clarifying questions if you need):

add new ${metaKey}+shift listeners to the command_events for the arrows you want to get events for
use this switch area to identify and direct the event
write changeOrder and changeWrap functions
follow patterns made by the changeDirection function
update the instruction card for the flex tool
update the flex keyboard trainer component to display the results of the new keyboard commands
QA QA QA
Make a PR
append tests for your feature? (optional but super welcome)

Thanks for the guidance :blush: Since I'm newbie I'll probably have lots of questions :sweat_smile: Anyway I'll start working on it and see how it goes.

fixed in #394


TimeRate's getTicks() method incorrect behaviour

In order to return number of seconds, division must be used instead of multiplication.
Currently new TimeRate(0.5, TimeRate::PER_SECOND) is not equal to new TimeRate(30, TimeRate::PER_MINUTE).

Fixed by #44


Heavy performance drops after setting up the integration

Version of the custom_component
0.6.5 (and tried main)
Describe the bug
Directly after I set up the integration, Home Assistant experienced significant performance drops. I restarted, but the restart was no longer possible (the core did not start anymore) and I had to force shutdown the machine.
Unfortunately, I don't have any logs because of this.
After removing the integration, everything ran normally again with the usual performance.
I have 8 BT proxies (ESP) and 1 BT adapter (ASUS). None of them are assigned to areas.
Is this a known issue or has anyone had similar experiences?

I'm really sorry to hear that!
There have been some changes in recent HA versions (around 2024.5 I believe) that appear to have made it less stable when integrations do non-threadsafe things, leading to (effectively) lock-ups. The changes were noted as adding stability but I've only seen it cause more frequent problems - which leads to fixes ultimately, but a lot of user frustration along the way!
Do you know what version of main you tried? I have been pushing a few changes up of late, so it's possible the one you grabbed might have been before a fix. Anyway, I have just pushed another potential fix, and bundled everything up into a release, so if you're able to see if v0.6.6 works for you that would be awesome.
I know it's a big ask, but if v0.6.6 is still a problem, would you be able to do some testing for me? The problem is that I have been unable to replicate some of these issues on my systems, which makes finding a fix so much harder.
From  https://community.home-assistant.io/t/2024-5-tracking-down-instability-issues-caused-by-integrations/724441
if you could add this to your configuration.yaml and see  if it gives you any helpful output:
homeassistant:
  debug: true

You might need to be able to SSH into your box to be able to get the logs, so I understand if this is a bit much to ask, especially if your system is actively running stuff in your home. I really hope I can get these things sorted out quickly, it's just tricky when I can't get my own systems to show the error. Thanks for your patience, though, and I really appreciate you taking the time to report your experience!

I had a similar experience after installing the integration (v0.6.6) 3 weeks ago:

Installed the integration
Worked some minutes and then slowed down the system (Raspberry Pi 4 with HA 2024.5.5)
System got unresponsive so that a restart was not possible
Unplugged power to restart the system

I checked the logs and found a warning from the integration that there is a device (Shelly with enabled bluetooth proxy) which has no assigned area. Unfortunately I haven't the exact log anymore.
Since I added an area to the device the problem is gone.
I'm not sure if the restart or the assignment of the area fixed the problem.
In the near future I'll integrate some more Shelly devices which gives me the possibility to do some tests and maybe the problem could be reproduced.
Thanks for the great project, exactly what I looked for.

Running into similar here. Running HAOS 2024.6.3 on Pi 4B 4gb w/ ssd for storage. No addons to speak of, fair few integrations, and usually it zips right along, but something with this integration seems to be slowing things down. As soon as I disable it things are quick as normal, but with it enabled everything is noticeably slower, particularly booting up.
Now, I'm gonna caveat all of that and say that this is a fantastic tool and absolutely killer integration, so please don't think I'm displeased or anything. Just wanting to add that it seems to be happening to me too, and willing to provide what support I can in terms of logs and whatnot if you want them to have a look at.

Oh, interesting!
So it seems there is a general performance issue rather than the complete wedging / spinning issues that came with the async changes, then.
@jack3308 do you have any proxies not assigned to areas? (there will be regular log entries about it if that's the case)
The slow boot-up is a bit perplexing. A gradual decrease in performance over time could point to a memory leak or possibly extra tasks being scheduled that shouldn't be, but slow bootup is odd, because unlike many integrations we are only working with local data so no remote APIs to contact etc, and startup is usually lightning fast as a result.
If you can enable debug logging for a minute or so, then disable it and send the resulting file that might shed some light. Note that the file will contain MAC addresses and possibly IPs etc, so redact anything you're not comfortable with sharing, or email directly to me at<EMAIL_ADDRESS>if you'd rather not post publicly.
Also if you could let me know how many devices and entities you have enabled (just the numbers, as shown in the screenshot below) that will be helpful, too.

At the moment, only tracking 2 devices (my phone and my partners) using iBeacon from HASS companion app on android. I worked my way through the beacons I have and I don't think there are any not assigned to an area.
Thank you so much your work on this, it's truly the best option by miles for room presence and the methodology and aspirations are miles ahead of other options. Not that you need to hear it from me, I'm sure, but it's seriously night and day how much more clever and useful this is, so thanks for all of your effort. I'll email the logs directly to you, I'm in a bit over my head when it comes to debugging an integration so if I've done something wrong please let me know.

Uhh, it wirked OK with 1 tracker and 1 device to track. I hoped to replace my clunky custom solution with this one. (Though i remember that ESPHome BLE tracker is buggy crap, that's why i came out with my own one at the first place.)
But no miracle - after updating 5 other trackers with BLE tracker, HA just went unresponsive, and after i deleted that BLE code and updated ESPs, HA rebooted and started working again. Before this happened, i saw that my only device tracker was going to unavailable and back to home once per second.
Deleting. It should've been the best attempt on presence, wit device trackers inbuilt - but unfortunately it's not. :)

@formatBCE I am having trouble understanding what you mean.
You mention "BLE tracker", are you referring to the esp32_ble_tracker config in your esphome devices? If so, and turning that on and off causes HA to lock up, can you confirm that doing so behaves differently when Bermuda is installed vs when it's not?
If your esphome device was also dropping in and out (going unavailable and back every second) does this also happen when Bermuda isn't running but the esphome is configured as a proxy?
A full copy of your esphome yaml might be helpful to diagnose this. Also, are you in a fairly "dense" bluetooth environment, like an appartment building or other area where there are "many" bluetooth devices around? I am wondering if perhaps the volume of BT traffic might be triggering an issue, either on the esphome side or perhaps in Bermuda.
I've had my system previously running with a cache of tens of thousands of addresses (most were transient or rotated random macs that were no longer present, of course) and it didn't cause any issues other than a slightly longer processing loop (each second, when in debug mode, HA will tell you how long Bermuda spent processing its data). It got up to about 0.1 seconds or so at worst. More recent releases have record pruning so you wouldn't see anywhere near that number of addresses any more, but if many active devices are around it could possibly increase the processing load.
I'd want to confirm that the esphome setup with bluetooth_proxy turned on and Bermuda turned off was stable first, though - that could be an indicator of something else going on.

@agittins sorry, I won't be helpful here. I reverted everything back, and can't afford reproducing that behaviour again... Will use my old stuff.
Yes, I referred to esp32_ble_tracker. Couple years ago I tried to play with ESPHome BLE tracking, but had only bad experience with inbuilt mechanism... So I can suspect that it takes at least some part in that hanging.

I am going to close this issue since I think the original problem @collateral87 experienced was probably some race conditions that were exposed after changes in recent versions of HA. As far as I know these are bedded down since v0.6.7.
@srempfer and @jack3308 your issues seem to be performance related rather than the complete wedging that the async issues caused. Since you're both on Raspberry Pi's it's possible that memory usage or log/database churn might be contributing factors (Jack I note you're using an SSD which should be pretty solid, not sure Sean if you are on SD card or SSD). SD cards in particular are prone to I/O issues which Bermuda is likely to make worse, but hard to say for sure.
I have just written up a wiki page on how to manage database IO if that's a possible cause: https://github.com/agittins/bermuda/wiki/Logs,-Recorder-and-Database-size
@jack3308 did you end up emailing me any logs? I haven't found anything but it might be in spam (I get a lot of that, given I keep putting my email address in public places!). If you have sent it let me know something I can search on to check my mailbox, eg if jack is in your address that would be enough for me to find it, I think. If you have the time, could you open a new issue for your performance problem? I think it's probably quite different from what the OP had, so would rather track it separately.
@formatBCE I'm not sure what hardware you are running on, but your experience is definitely not typical. If you do decide to give it another go please feel free to raise another issue if you run into trouble again. My home production server is tracking 22 devices over 418 entities (I have a lot of the extra distance sensors enabled) and its totally stable, with cpu usage around 10% to 20% typically, most of which is from other integrations. Database output is significant though, and I run a separate postgres db with over 200GB of history.
The once-per-second flipping from unavailable is weird, and might indicate the integration was crashing and reloading, or just that one of the timeout settings was set too low. Anyway, if you do decide to give it another crack I'm happy to help track down the issue.
For now though, I'll close this issue and invite y'all to pop open a new one if comes up again.

@agittins email should start with "jk@". Sorry, custom domain so this happens sometimes.

I running my HA on an SSD.
Last weekend I integrated some more Shelly devices which gave me the possibility to do some tests but I couldn't reproduce the problem. Neither with HA 2024.5.5 nor HA 2024.7.1

For anyone following this issue, it seems from @jack3308's logs that Bermuda is having a bad interaction with another integration, which seems to be generating tens of thousands of bluetooth device entries.
If anyone is experiencing performance issues and has some time to try a thing or two, it would be great if you could:

update to the main version via HACS
Check how many scanners and devices Bermuda reports on the first page of its "Configure" dialog
If it's "a lot", then select the "Download Diagnostics" option in the Bermuda menu and share the file on #234

Obviously you won't be able to do this if your system completely wedges up, so I totally understand that not everyone will be able (or willing!) to give this a shot.
Device and scanner counts (only in main, currently):

Download Diagnostics (only in main, currently):