[FEATURE] Add support for downloading YouTube videos

Describe the solution you'd like
Currently, the application shows HTTP proxy error while downloading an YouTube video. So, I want to have the application support for downloading YouTube videos. Anyone can contribute, just leave a comment and I will assign you with the task. Please link this issue to your PR.
Additional context
Refer to the FileDownloader.java file to see how it downloads a file and find how you can add support for downloading YouTube videos in that file.

Anyone interested to contribute and solve the issue should at first comment then I would assign them with the task. Multiple assignees can be there.

Hey, can i take this up?

Please link this issue to your PR.

@aswinikalyan30 how much is the work done?

@aswinikalyan30 please respond

Sorry, please assign to someone else.

Ok @aswinikalyan30 .
If anyone wants to work on this issue, please comment below and I would assign you with the task.

@SaptarshiSarkar12 Hi! Can I take this up?

@poshi1865 Yes, assigning you. Please link this issue to your PR.

Yes will do, thank you!

Hi, unfortunately something came up and I will be very busy for the next few days. I might get some time to work on this, but feel free to unassign me.

Ok @poshi1865 . You may continue your work.
If anyone wishes to contribute to this project, feel free to comment below and I would assign you with the task.

If anyone wishes to contribute to this project, feel free to comment below and I would assign you with the task. Multiple assignees are allowed.

Hey can you assign this to me?

Sure @Ishika-jain . Please link this issue with your PR .

@poshi1865 @Ishika-jain How much work has been done?

Hi! Sorry I have been very busy, will get on this as soon as time allows

Hi! Sorry I have been very busy, will get on this as soon as time allows

Ok @poshi1865 .
What about @Ishika-jain? How much has been completed?

hey sorry, I'm sorry, I am having difficulty doing it, feel free to unassign me.

hey sorry, I'm sorry, I am having difficulty doing it, feel free to unassign me.

Ok @Ishika-jain

@poshi1865  How much work has been done ?

Hi! I just got free today, so I am on it. Will update you soon.

Hi! I just got free today, so I am on it. Will update you soon.

Great @poshi1865 😄! Waiting for the updates.

@SaptarshiSarkar12 so downloading youtube videos is proving to be more difficult that I thought. However, there is a program called youtube-dl which can be used to perform the download. Is it okay if we add youtube-dl as a dependency?

@SaptarshiSarkar12 so downloading youtube videos is proving to be more difficult that I thought. However, there is a program called youtube-dl which can be used to perform the download. Is it okay if we add youtube-dl as a dependency?

ok then

issue resolved?

issue resolved?

@thexahmed00 Not yet. @poshi1865 is working on this.

@SaptarshiSarkar12 Hey! So I am using https://github.com/yt-dlp/yt-dlp, and drifty can now download youtube videos on my ubuntu machine. For drifty to be able to do that, the yt-dlp executable should be in the PATH variable. It is easy to do that with an install on a linux based machine throught the package manager. One can simply run apt install yt-dlp. In windows, one will have to download yt-dlp and then add it to the path environment variable.

Also as a side note, the automatic download path detection is broken (atleast for me, I run ubuntu).

@SaptarshiSarkar12 Hey! So I am using https://github.com/yt-dlp/yt-dlp, and drifty can now download youtube videos on my ubuntu machine. For drifty to be able to do that, the yt-dlp executable should be in the PATH variable. It is easy to do that with an install on a linux based machine throught the package manager. One can simply run apt install yt-dlp. In windows, one will have to download yt-dlp and then add it to the path environment variable.

@poshi1865 is there any way to automatically download yt-dlp and automatically add it to the path?

@SaptarshiSarkar12 Hey! So I am using https://github.com/yt-dlp/yt-dlp, and drifty can now download youtube videos on my ubuntu machine. For drifty to be able to do that, the yt-dlp executable should be in the PATH variable. It is easy to do that with an install on a linux based machine throught the package manager. One can simply run apt install yt-dlp. In windows, one will have to download yt-dlp and then add it to the path environment variable.

@poshi1865 is there any way to automatically download yt-dlp and automatically add it to the path?

@poshi1865 Please see if you can find any fix for this. Then, I shall review #110.

Right now, when a user downloads drifty, the exe is downloaded directly. Instead, what we can do is write a powershell script (for windows) and a bash script (for linux), which the user will download and run. That script will download and install both Drifty and yt-dlp automatically, and add them to the path. This will also ensure that Drifty is able to run from any directory.

Right now, when a user downloads drifty, the exe is downloaded directly. Instead, what we can do is write a powershell script (for windows) and a bash script (for linux), which the user will download and run. That script will download and install both Drifty and yt-dlp automatically, and add them to the path. This will also ensure that Drifty is able to run from any directory.

What if Drifty runs the script through its code? Is it possible?

@SaptarshiSarkar12 Hmmm yes I think that should be possible, I will find out how to do that and update you

@SaptarshiSarkar12 Hmmm yes I think that should be possible, I will find out how to do that and update you

Ok @poshi1865

Is there a reason why you are using an exe file instead of a jar file for windows downloads?

Is there a reason why you are using an exe file instead of a jar file for windows downloads?

@poshi1865 Because the user does not need to run commands. I have thought of creating an executable for macOS and guidelines for running the application in Linux.

Okay got it. As far as yt-dlp is concerned, we will need to download and store the exe somewhere. The code for that can be put in Drifty but then the yt-dlp executable will be downloaded and put in the same location as the Drifty exe, or somewhere else on the system. I think that will be very messy.

The idea I had earlier was to make an installer for Drifty. This will install drifty and all its dependencies (like yt-dlp) in the user's preferred location and then create a shortcut for them on the desktop. The user will still be able to run Drifty without any commands and this is a much cleaner way to install.

This is how I think most apps on windows are installed.

An installer is downloaded.
Installer is run to install the app in the user's preferred location.
A shortcut is created so that the user can run it in the desktop.

We can have the same flow for Drifty as well.

Also if you are on a windows machine, can you please install yt-dlp linked here and test pr #110 once? I have to know whether it is working on windows or not. You can place the yt-dlp executable in the same directory as Drifty.exe, it should work.

The idea I had earlier was to make an installer for Drifty. This will install drifty and all its dependencies (like yt-dlp) in the user's preferred location and then create a shortcut for them on the desktop. The user will still be able to run Drifty without any commands and this is a much cleaner way to install.

And we will not have to add anything to the path as well.

The idea I had earlier was to make an installer for Drifty. This will install drifty and all its dependencies (like yt-dlp) in the user's preferred location and then create a shortcut for them on the desktop. The user will still be able to run Drifty without any commands and this is a much cleaner way to install.

And we will not have to add anything to the path as well.

Great idea! You can work on this if you want.

Also if you are on a windows machine, can you please install yt-dlp linked here and test pr #110 once? I have to know whether it is working on windows or not. You can place the yt-dlp executable in the same directory as Drifty.exe, it should work.

Ok @poshi1865 Let me check once.

Also if you are on a windows machine, can you please install yt-dlp linked here and test pr #110 once? I have to know whether it is working on windows or not. You can place the yt-dlp executable in the same directory as Drifty.exe, it should work.

Yeah @poshi1865 , it's working! But the details of yt-dlp should not be printed in the console as the users won't understand any. Instead, it should be silently downloaded and a progress bar should be present.
Here are the print statements displayed -

====================================================================
  _____   _____   _____  ______  _______ __     __
 |  __ \ |  __ \ |_   _||  ____||__   __|\ \   / /
 | |  | || |__) |  | |  | |__      | |    \ \_/ /
 | |  | ||  _  /   | |  |  __|     | |     \   /
 | |__| || | \ \  _| |_ | |        | |      | |
 |_____/ |_|  \_\|_____||_|        |_|      |_|
====================================================================
Enter the link to the file : https://www.youtube.com/watch?v=BQLOfws52Rs
YOUTUBE URL DETECTED
Do you want to download the file in your default downloads folder? (Enter Y for yes and N for no) : y
Trying to auto-detect default Downloads folder...
Default download folder detected : D:\Downloads\
Downloading from youtube....
Directory: D:\Downloads\
[youtube] BQLOfws52Rs: Downloading webpage
[youtube] BQLOfws52Rs: Downloading android player API JSON
[youtube] BQLOfws52Rs: Downloading player 7a062b77
[youtube] BQLOfws52Rs: Downloading MPD manifest
[youtube] BQLOfws52Rs: Downloading MPD manifest
[info] BQLOfws52Rs: Downloading 1 format(s): 22
[download] Destination: D:\Downloads\The Ultimate GitHub Campus Expert AMA - Out of Syllabus [BQLOfws52Rs].mp4
[download]   0.0% of  217.98MiB at   24.86KiB/s ETA 02:29:39
[download]   0.0% of  217.98MiB at   39.54KiB/s ETA 01:34:05
[download]   0.0% of  217.98MiB at   92.25KiB/s ETA 40:19
[download]   0.0% of  217.98MiB at  167.37KiB/s ETA 22:13
[download]   0.0% of  217.98MiB at  282.29KiB/s ETA 13:10
[download]   0.0% of  217.98MiB at  573.69KiB/s ETA 06:28
[download]   0.1% of  217.98MiB at    1.13MiB/s ETA 03:30
[download]   0.1% of  217.98MiB at    2.08MiB/s ETA 01:44
[download]   0.2% of  217.98MiB at    2.84MiB/s ETA 01:16
[download]   0.5% of  217.98MiB at    3.84MiB/s ETA 00:56
[download]   0.9% of  217.98MiB at    1.93MiB/s ETA 01:51
[download]   1.5% of  217.98MiB at    1.90MiB/s ETA 01:52
[download]   2.4% of  217.98MiB at    2.61MiB/s ETA 01:21
[download]   4.1% of  217.98MiB at    2.74MiB/s ETA 01:16
[download]   5.4% of  217.98MiB at    2.68MiB/s ETA 01:17
[download]   6.6% of  217.98MiB at    2.75MiB/s ETA 01:14
[download]   8.0% of  217.98MiB at    2.24MiB/s ETA 01:29
[download]   8.7% of  217.98MiB at    2.37MiB/s ETA 01:23
[download]  10.2% of  217.98MiB at    2.42MiB/s ETA 01:20
[download]  11.5% of  217.98MiB at    2.40MiB/s ETA 01:20
[download]  12.5% of  217.98MiB at    2.31MiB/s ETA 01:22
[download]  13.2% of  217.98MiB at    2.41MiB/s ETA 01:18
[download]  14.7% of  217.98MiB at    2.43MiB/s ETA 01:16
[download]  16.0% of  217.98MiB at    2.50MiB/s ETA 01:13
[download]  17.7% of  217.98MiB at    2.59MiB/s ETA 01:09
[download]  19.5% of  217.98MiB at    2.67MiB/s ETA 01:05
[download]  21.2% of  217.98MiB at    2.71MiB/s ETA 01:03
[download]  22.7% of  217.98MiB at    2.79MiB/s ETA 01:00
[download]  24.6% of  217.98MiB at    2.68MiB/s ETA 01:01
[download]  25.5% of  217.98MiB at    2.71MiB/s ETA 00:59
[download]  27.3% of  217.98MiB at    2.77MiB/s ETA 00:57
[download]  29.2% of  217.98MiB at    2.80MiB/s ETA 00:55
[download]  30.7% of  217.98MiB at    2.85MiB/s ETA 00:53
[download]  32.5% of  217.98MiB at    2.89MiB/s ETA 00:50
[download]  34.3% of  217.98MiB at    2.91MiB/s ETA 00:49
[download]  35.8% of  217.98MiB at    2.87MiB/s ETA 00:48
[download]  36.9% of  217.98MiB at    2.80MiB/s ETA 00:49
[download]  37.6% of  217.98MiB at    2.83MiB/s ETA 00:48
[download]  39.0% of  217.98MiB at    2.81MiB/s ETA 00:47
[download]  40.0% of  217.98MiB at    2.83MiB/s ETA 00:46
[download]  41.9% of  217.98MiB at    2.84MiB/s ETA 00:44
[download]  43.3% of  217.98MiB at    2.78MiB/s ETA 00:44
[download]  44.0% of  217.98MiB at    2.81MiB/s ETA 00:43
[download]  45.5% of  217.98MiB at    2.81MiB/s ETA 00:42
[download]  46.8% of  217.98MiB at    2.83MiB/s ETA 00:41
[download]  48.4% of  217.98MiB at    2.75MiB/s ETA 00:40
[download]  49.3% of  217.98MiB at    2.70MiB/s ETA 00:40
[download]  49.8% of  217.98MiB at    2.72MiB/s ETA 00:40
[download]  51.0% of  217.98MiB at    2.74MiB/s ETA 00:38
[download]  52.9% of  217.98MiB at    2.77MiB/s ETA 00:37
[download]  54.7% of  217.98MiB at    2.80MiB/s ETA 00:35
[download]  56.5% of  217.98MiB at    2.81MiB/s ETA 00:33
[download]  57.9% of  217.98MiB at    2.84MiB/s ETA 00:32
[download]  59.8% of  217.98MiB at    2.86MiB/s ETA 00:30
[download]  61.4% of  217.98MiB at    2.87MiB/s ETA 00:29
[download]  63.1% of  217.98MiB at    2.88MiB/s ETA 00:27
[download]  64.5% of  217.98MiB at    2.90MiB/s ETA 00:26
[download]  66.3% of  217.98MiB at    2.92MiB/s ETA 00:25
[download]  68.1% of  217.98MiB at    2.87MiB/s ETA 00:24
[download]  69.0% of  217.98MiB at    2.88MiB/s ETA 00:23
[download]  70.8% of  217.98MiB at    2.90MiB/s ETA 00:21
[download]  72.4% of  217.98MiB at    2.85MiB/s ETA 00:21
[download]  73.3% of  217.98MiB at    2.86MiB/s ETA 00:20
[download]  74.9% of  217.98MiB at    2.87MiB/s ETA 00:19
[download]  76.5% of  217.98MiB at    2.88MiB/s ETA 00:17
[download]  78.0% of  217.98MiB at    2.91MiB/s ETA 00:16
[download]  79.8% of  217.98MiB at    2.93MiB/s ETA 00:15
[download]  81.6% of  217.98MiB at    2.94MiB/s ETA 00:13
[download]  83.3% of  217.98MiB at    2.85MiB/s ETA 00:12
[download]  84.1% of  217.98MiB at    2.86MiB/s ETA 00:12
[download]  85.8% of  217.98MiB at    2.81MiB/s ETA 00:11
[download]  86.6% of  217.98MiB at    2.83MiB/s ETA 00:10
[download]  88.3% of  217.98MiB at    2.82MiB/s ETA 00:09
[download]  89.4% of  217.98MiB at    2.81MiB/s ETA 00:08
[download]  90.4% of  217.98MiB at    2.82MiB/s ETA 00:07
[download]  92.1% of  217.98MiB at    2.81MiB/s ETA 00:06
[download]  93.4% of  217.98MiB at    2.83MiB/s ETA 00:05
[download]  95.2% of  217.98MiB at    2.85MiB/s ETA 00:03
[download]  96.9% of  217.98MiB at    2.85MiB/s ETA 00:02
[download]  98.3% of  217.98MiB at    2.86MiB/s ETA 00:01
[download] 100.0% of  217.98MiB at    2.88MiB/s ETA 00:00
[download] 100% of  217.98MiB in 00:01:16 at 2.83MiB/s
Press Q to Quit Or Press any Key to Continue
q

Okay, I will fix that.

Can this be a separate pr? This will be a significant task, as the progress bar shows the completion of the download and I will have to find some way to feed that from yt-dlp to the ProgressBarThread class.

Yes @poshi1865

Also, can you merge #110 now? After that I can work on figuring out the installation part. Until then contributors and users will have to do a one time download of yt-dlp. If you want I can put instructions for that in the README.md file.

Ok.


Describing hack how to clean up local branches

Hey guys, maybe I needed to dig deeper for this feature, but this hack helped me a lot to clean up my local repo.

Thank you for this. Worked as described for me. I mainly wanted to use git-sweep to clean up my local branches. My remote and local branches are now all cleaned up!

@zane-techempower great to hear it helped :) really hope they can merge it, even if they don't maintain repo anymore.
@robmadole can you please take 5 second look on this and merge this PR? It's just a README update.

@Sapphire64 I no longer have commit access to this repo or I would. Sorry!

@Sapphire64 Merging!


'Backend' error in command prompt

c:>python run_server.py --port 9090 --backend faster_whisper
Traceback (most recent call last):
File "c:\run_server.py", line 31, in
server.run(
TypeError: TranscriptionServer.run() got an unexpected keyword argument 'backend'
Any solutions?

@Farooq87 That shouldn't happen if you are on the latest version. Can you try upgrading whisper-live and test if it resolves the issue. Because, I see that we have a backend command-line arg in run_server.py:
https://github.com/collabora/WhisperLive/blob/a9683319e05bb881f560b713260d0727edf69cae/run_server.py#L34

@Farooq87 That shouldn't happen if you are on the latest version. Can you try upgrading whisper-live and test if it resolves the issue. Because, I see that we have a backend command-line arg in run_server.py:
https://github.com/collabora/WhisperLive/blob/a9683319e05bb881f560b713260d0727edf69cae/run_server.py#L34

I updated Faster Whisper, but still have the same error

@Farooq87 Instead of faster-whisper, you need to check that you run latest version of whisper-live.
pip install --upgrade whisper-live

@Farooq87 just tested with latest version of whisper-live:
$ python run_server.py --port 9090 --backend faster_whisper
INFO:websockets.server:connection open
INFO:root:New client connected
INFO:root:Running faster_whisper backend.
INFO:faster_whisper:Processing audio with duration 00:01.024
INFO:faster_whisper:VAD filter removed 00:00.000 of audio
.
.


feat: [M3-7028] - Add AGLB Edit Route Drawer

Description 📝
Implements the AGLB edit route drawer.
Preview 📷
https://github.com/linode/manager/assets/125309814/e0b40211-998f-472c-bcc1-9a37082e7c2b
How to test 🧪
Prerequisites

Checkout this PR
Turn on the MSW (bottom left dev tools)

Reproduction steps

Go to http://localhost:3000/loadbalancers/0/routes
Click on the action menu ... next to a route
Click the Edit button

Verification steps

Observe drawer opens with:

Title containing specific route label
Input field containing current route label
Current protocol selected

Observe when click Edit Route network payload should contain your new options
Run cypress tests yarn cy:run -s cypress/e2e/core/loadBalancers/load-balancer-routes.spec.ts

As an Author I have considered 🤔
Check all that apply

[x] 👀 Doing a self review
[x] ❔ Our contribution guidelines
[x] 🤏 Splitting feature into small PRs
[x] ➕ Adding a changeset
[x] 🧪 Providing/Improving test coverage
[x] 🔐 Removing all sensitive information from the code and PR description
[x] 🚩 Using a feature flag to protect the release
[x] 👣 Providing comprehensive reproduction steps
[x] 📑 Providing or updating our documentation
[x] 🕛 Scheduling a pair reviewing session
[x] 📱 Providing mobile support
[x] ♿  Providing accessibility support

I have noticed that other places of edit scenarios I feel like language for the primary button is inconsistent. "Save Changes" make sense to me.

I've also noticed inconsistency of whether we include data next to the title or not (ie: Edit Route: <route_label> ... in addition, some designs have a dash instead of a colon 🤔 ). @bnussman-akamai did you want me to reach out to UX or did you want to make a judgment call?

I think we should do what is most common in Cloud Manager. I think that would be Edit Route <route label>. So without the : @jaalah-akamai

I think we should do what is most common in Cloud Manager. I think that would be Edit Route . So without the : @jaalah-akamai

Looking good now 👍


NixOS modules: Add mkSuperOptionAlias and mkForEachSubModule.

Previously, getting values from submodules to modify the super-module options,
required more complex filtering logic which is dependent on the option type
wrapping the submodule, as well as making sure that the types are coherent.
This change makes it possible to easily set super-module options from submodule
aliases. This change adds 2 major functions: mkSuperOptionAlias and
mkForEachSubModule.
mkSuperOptionAlias is made to create an option within a submodule, which
inherit the type of a parent module option, if it exists as well as preventing
any uses of its computed output. Previously, an easy mistake when implementing
such aliases was to use the computed value of the option. Using the computed
value is a problem as the type, which provide a merge and apply function is
not garanteed to be idempotent. The work-around is to expose the options with
the submoduleWith argument named exportOptionsUnderConfig. This flag, set to
false to avoid regressions, when enabled, it will add the options attribute
under the configuration produced by each submodule instance. This is mandatorry
to extract option definitions using mkAliasDefinitions.
mkForEachSubModule is made to abstract over the wrapping type of submodules,
and forward definitions with mkAliasDefinitions or similar functions, to skip
over complexity induced by the merge and apply function logic. This works by
adding a new attribute named extractSubValues to each option type. This
attribute is a function which is responsible from unpacking any wrapping types
which are wrapping submodules, such as list or attribute sets, and return an
unordered list of submodule instances. Each submodule instance is then iterated
over and given as argument to the first argument, which is in charge of
producing a valid module or option definition. This is best achieved with
mkAliasDefinitions followed with the access to the option exported by the
submodule, using exportOptionsUnderConfig flag of the submoduleWith type.
The following example illustrates how these functions can be used to aggregate
all definitions defined within multiple submodules, into the super-module which
collect all these definitions.
{ config, options, lib, ... }:

let superOptions = options; in
{
  # Suppose that the following option exists:
  #   options.aggregate = lib.mkOption { .. };

  # `namedAggregate` is a tiny submodule which exposes `super.aggregate`, and let
  # the use set parts of the aggregated content under a specific name, such as:
  #
  #   namedAggregate.setupEtc.super.aggregate = " .. ";
  #   namedAggregate.setupBin.super.aggregate = " .. ";
  #
  # All of which would be used as a definitions of the `aggregate` option.
  options.namedAggregates = mkOption {
    type = with lib.types; attrsOf submoduleWith {
      # Expose the eval.options attribute used as argument of `mkAliasDefinitions`
      exportOptionsUnderConfig = true;

      modules = [
        # Create a sub-module which aliases the super-module option, if it exists.
        #
        # To explicit the fact that this option is used only by the super-module,
        # it is added under `super.` attribute set, which might be a convention.
        #
        # Using (.. or null) as argument is useful for submodules which can be
        # embedded in various super-modules. As, this will also work if the
        # option does not exists.
        ({ lib, ... }: {
          options.super.aggregate =
            lib.mkSuperOptionAlias (superOptions.aggregate or null);
        })
      ];
    };
  };

  config = {
    # Collect all options definitions from submodules, if they are defined.
    # `mkForEachSubModule` iterate over all submodule instances. Each instance
    # `eval` has an attribute named `options` exposed by `exportOptionsUnderConfig`
    # flag. This `options` attribute is the same as the `options` argument of
    # submodules and contains the hierachy of options declarations and definitions.
    # `mkAliasDefinitions` is then used to extract options definitions if the
    # option is set in the submodule.
    aggregate = lib.mkForEachSubModule
      (eval: lib.mkAliasDefinitions eval.options.super.aggregate)
      options.namedAggregates;
  };
}

Things done
Tested locally using:
  cd <local-nixpkgs>/lib/tests
  NIX_PATH=<local-nixpkgs> ./modules.sh

All changes made here are new functions, or should be backward compatible.

I'm not sure about exportOptionsUnderConfig. It doesn't seem right, to return options there.
What does a module return?
a. Since RFC42, we've cleaned up the return "config" so that it is (usually) ready to be used for config file generation. exportOptionsUnderConfig seems like a step in the other direction.
b. In existing submodules that have logic in them, the returned value is (conceptually) limited to one or two option values.
c. With this PR, we'll be "returning" definitions to merge elsewhere; not really a well-defined value, but more like a side effect, so to speak.
In one design, we could give submodules complete control over their return value. This allows a submodule to be more explicit in case of (b), although the resulting encapsulation makes troubleshooting in a repl a bit harder. The same facility could implement (c), but I don't like that the module author has to deal with a new "definition set" type that can only be used in one way anyway.
In another design, we could make "supermodules" an integrated feature that seems to fully capture the use case. From a module author perspective, it could look like:
acme.nix
{
  options.security.acme.certs = mkOption {
    type = attrsOf (submodule ({ config, name, ... }: {
      options.webroot = .....;
      super = { # supermodules section. Fixed field like `options`, attrset of configs
        system = { # config to apply at the system level
          services.systemd.services."acme-${name}" = {
            # ...
          };
        };
      };
    }));
  };
}

nixos.nix/nix-darwin.nix
{
  applySupermodules = "system"; # applies all `super.system` supermodules.
  options = .....;
  config = .....;
}

This is achievable with the following changes:

two new fields for the module syntax
keep track of config._module out-of-band instead of throwing it away when converting config to the submodule value. We have to do this for built-in assertions and warnings anyway (https://github.com/NixOS/nixpkgs/pull/97023#issuecomment-999639778); may as well do it for supermodules
a solution for restricting the "write scope" of supermodules

The latter is important for laziness. Without it, the module system will recurse infinitely, as it tries to determine which submodules exist. To determine the set of all system-level defs, you'd have to evaluate all submodules, but in order to know which submodules exist, you'd need to know all system-level defs. This recursion needs to be restricted. One way to achieve this is by requiring the module author to specify which options can be set via supermodules, as you've shown in your example (config.aggregate = /*...*/). This gets the job done, but it would be nicer if the module author didn't have to specify this. We could take this information from the imports graph, paired with some logic that can also be used for imports hermeticity checking, which was identified to be useful for https://github.com/NixOS/rfcs/pull/78.
So this PR seems to provide an intermediate solution for part of the problem that will not be part of the final solution, unless I'm wrong about the feasibility of the design I've written about here.

I'm not sure about exportOptionsUnderConfig. It doesn't seem right, to return options there.
What does a module return?
a. Since RFC42, we've cleaned up the return "config" so that it is (usually) ready to be used for config file generation. exportOptionsUnderConfig seems like a step in the other direction.
b. In existing submodules that have logic in them, the returned value is (conceptually) limited to one or two option values.
c. With this PR, we'll be "returning" definitions to merge elsewhere; not really a well-defined value, but more like a side effect, so to speak.
In one design, we could give submodules complete control over their return value. This allows a submodule to be more explicit in case of (b), although the resulting encapsulation makes troubleshooting in a repl a bit harder. The same facility could implement (c), but I don't like that the module author has to deal with a new "definition set" type that can only be used in one way anyway.
In another design, we could make "supermodules" an integrated feature that seems to fully capture the use case. From a module author perspective, it could look like:
acme.nix
{
  options.security.acme.certs = mkOption {
    type = attrsOf (submodule ({ config, name, ... }: {
      options.webroot = .....;
      super = { # supermodules section. Fixed field like `options`, attrset of configs
        system = { # config to apply at the system level
          services.systemd.services."acme-${name}" = {
            # ...
          };
        };
      };
    }));
  };
}

nixos.nix/nix-darwin.nix
{
  applySupermodules = "system"; # applies all `super.system` supermodules.
  options = .....;
  config = .....;
}

This is achievable with the following changes:

two new fields for the module syntax
keep track of config._module out-of-band instead of throwing it away when converting config to the submodule value. We have to do this for built-in assertions and warnings anyway (https://github.com/NixOS/nixpkgs/pull/97023#issuecomment-999639778); may as well do it for supermodules
a solution for restricting the "write scope" of supermodules

The latter is important for laziness. Without it, the module system will recurse infinitely, as it tries to determine which submodules exist. To determine the set of all system-level defs, you'd have to evaluate all submodules, but in order to know which submodules exist, you'd need to know all system-level defs. This recursion needs to be restricted. One way to achieve this is by requiring the module author to specify which options can be set via supermodules, as you've shown in your example (config.aggregate = /*...*/). This gets the job done, but it would be nicer if the module author didn't have to specify this. We could take this information from the imports graph, paired with some logic that can also be used for imports hermeticity checking, which was identified to be useful for https://github.com/NixOS/rfcs/pull/78.
So this PR seems to provide an intermediate solution for part of the problem that will not be part of the final solution, unless I'm wrong about the feasibility of the design I've written about here.

I'm not sure about exportOptionsUnderConfig. It doesn't seem right, to return options there.

I agree, the problem is that I wanted to avoid have 2 evaluations of the submodules, one for getting the config, and one for getting the options set.
The exportOptionsUnderConfig flag is a hack in the sense that it does not fit nicely.
Maybe we should find a way to extract the options set of each submodule instance as part of the value computation, and make it bubble up the value computation of any wrapping type, such that we can expose the list of options set instances on the options of the super module. While removing it from the config value, exposed as the returned value of the submodule.
Thus, not polluting the configuration, not requiring the exportOptionsUnderConfig flag, while keeping the same usage.
I think this is doable. This will require some extra logic around the merge functions.
I will try to do it ...

[...] but I don't like that the module author has to deal with a new "definition set" type that can only be used in one way anyway.

What do you mean? The new mkForEachSubModule is made to explicitly handle values which are set in a submodule, as well as avoiding the lack of idempotent guarantee on the merge / apply functions.

In another design, we could make "supermodules" an integrated feature that seems to fully capture the use case. From a module author perspective, it could look like:
acme.nix
{
  options.security.acme.certs = mkOption {
    type = attrsOf (submodule ({ config, name, ... }: {
      options.webroot = .....;
      super = { # supermodules section. Fixed field like `options`, attrset of configs
        system = { # config to apply at the system level
          services.systemd.services."acme-${name}" = {
            # ...
          };
        };
      };
    }));
  };
}

nixos.nix/nix-darwin.nix
{
  applySupermodules = "system"; # applies all `super.system` supermodules.
  options = .....;
  config = .....;
}

(From what I understand) doing so implies that resolving the set of configurations for the system attribute set would lookup every submodule which exists, and seek for super.system inside these submodule. This sounds highly confusing to me:

The system might itself have submodules, should they be allowed to participate in the merging part? If a submodule wants to setup a web service, should we state applySupermodules = "services";?
A single module might allow any unrelated submodule to participate in the system configuration.
There is no option definition which explicit the relation. Having implicit relations sounds more dangerous, from my point of view, than having explicit declarations and definitions.

[...] One way to achieve this is by requiring the module author to specify which options can be set via supermodules, as you've shown in your example (config.aggregate = /*...*/). This gets the job done, but it would be nicer if the module author didn't have to specify this. We could take this information from the imports graph, paired with some logic that can also be used for imports hermeticity checking, which was identified to be useful for NixOS/rfcs#78.

My understanding between what you suggest and the current patch is that the current patch is making everything explicit, at the cost of having 10's of options to export within submodules, and to alias in supermodules.
While your approach is to take an holistic approach to cover an attribute set of options, and alias these implicitly between submodules and supermodules.
I think the implicit  / holistic approach to be dangerous and a huge performance hazard. Dangerous as a submodule which was not meant to participate can be taken into account, and a performance hazard as all options declarations would have to be scanned to filter all submodules which might provide extra definitions. This holistic approach will also skip mkIf properties, such as enable flags.

So this PR seems to provide an intermediate solution for part of the problem that will not be part of the final solution, unless I'm wrong about the feasibility of the design I've written about here.

Well, ... I do not think many options would have to be aliased, and if this is the case, maybe we can come-up with a short notation for it, similar to mkRenamedOptionModule. And if this short notation is too much of a burden as well, then we can think of aliasing set of options too.
I do not think that adding these short notation should block this change, nor that it would be too hard to add these notations in follow-up changes, if needed.

Superoptions or supermodules
I've found another way to write supermodules that avoids the early merge problem. It is very similar to what you're introducing in this PR. Here's what I did:
In the submodule:
      options.systemConfig = lib.mkOption {
        internal = true;
        type = types.unspecified; # A function from module arguments to config.
       };

      config.systemConfig = { config, ... }: {
           systemd.services."hercules-ci-agent${suffix}" = {
           # ...
       }

In the parent module ("system" module):
systemArgs@{ config, lib, ... }:
let
  mergeSub =
    f: lib.mkMerge (map (sub: f (sub.systemConfig systemArgs)) (lib.attrValues config.services.hercules-ci-agents));

    config = {
      nix = mergeSub (c: c.nix);
      systemd = mergeSub (c: c.systemd);
      users = mergeSub (c: c.users);
    }

This sidesteps the early/double merging problem by returning a function in a type that doesn't merge.
The other problems are still solved "manually", but can be generalized.
The main difference comes from trying to implement supermodules as opposed to superoptions -- keeping the system-level options together, as in a module.
Like a normal system-level module, the supermodule can also access the system-level config, pkgs, etc.
Some improvements are still necessary to turn this into a generalizable solution:

Have a better type than unspecified.
Generate the config value. This is still necessary for laziness and efficiency and seems unavoidable. Thank you for clarifying the performance hazard.

Writing it manually is error prone (and slightly repetitive). Specifically, if the module author makes a change that sets an option that's not below nix, systemd, users (in this example), the new value is silently ignored. This problem can be detected by generating an assertions value that checks whether the set of options paths is complete. assertions can be evaluated last, so shouldn't create a strictness problem.
This does depend on assertions, which is a NixOS module, not a module system built-in. We'll want to implement this first.
The projection c: c.systemd is a little simplistic, because it doesn't consider mkIf.

Also perhaps some other improvements are still possible, such as putting the supermodules in _module, accessing all submodule values through options instead.
Full example: https://github.com/hercules-ci/hercules-ci-agent/pull/359/files#diff-508731c212bbfef59eff41255fdb91a1e89bb82e9eadd6e7ddb4630f9ac30d14

IMHO it shouldn't be possible for submodules to modify the super-configuration at all. I.e. it should not be possible for security.acme.certs to modify services.systemd.services.
The big problem with the NixOS module system as currently used is that it breaks POLA, i.e. any top-level module can modify anything. For instance, the postgresql module can change my window manager settings. So we should move to submodules that have a much more limited scope (e.g. a "system service" submodule type that can declare a systemd unit but can't mess with the global system configuration). But obviously that depends on submodules not being able to modify the parent configuration.

@edolstra, except for my unrealistic proposal where "supermodules" would be integrated into the system automatically, they don't actually allow the submodule itself to modify the system config.
The regular, non-submodule, "system-level" module is still responsible for setting system level configuration. What we're discussing is providing means to make this easier, as having access to both module and submodule scope simultaneously is convenient, especially when writing modules that allow multiple instances of a service.
Regarding POLA, this can be a useful guide, but not a necessary one, as we do not need to treat those modules as adversarial. Services, as they exist in NixOS and its siblings, are already fully allowed to set any system configuration. This is what makes NixOS convenient and powerful, with cross-service functionality like ACME as a prime example.

For instance, the postgresql module can change my window manager settings.

Yet, it doesn't. We can trust the postgresql module to do the right thing.
I'd be interested in a different configuration model where POLA applies within the configuration manager, but this would be a research project, not an incremental improvement to a tangential PR.

(e.g. a "system service" submodule type that can declare a systemd unit but can't mess with the global system configuration). But obviously that depends on submodules not being able to modify the parent configuration.

Submodules are all of the same type, as far as the module system is concerned. I think you mean a set of new options like restricted.services.<name>.x, I can see that work. We can easily have imports at the submodule level too (esp with submoduleWith). If this is what you're thinking of, please create an issue where this can be discussed separately.
Again, this is possible both before and after this PR. Applying "supermodule" configuration automatically is definitively off the table. This PR can continue with the goal of making the already common module+submodule logic a little easier to write.

IMHO it shouldn't be possible for submodules to modify the super-configuration at all. I.e. it should not be possible for security.acme.certs to modify services.systemd.services.
The big problem with the NixOS module system as currently used is that it breaks POLA, i.e. any top-level module can modify anything. For instance, the postgresql module can change my window manager settings. So we should move to submodules that have a much more limited scope (e.g. a "system service" submodule type that can declare a systemd unit but can't mess with the global system configuration). But obviously that depends on submodules not being able to modify the parent configuration.

Yes, NixOS breaks the Principle of Least Priviledges.
However, I completely disagree on the fact, that removing this feature is beneficial for end users.
The fact that a security flag can harden all services.systemd.services sounds to me like an appealing feature. If we were to use PoLP, we would limit our-self to only providing self-contained abstractions where users would have to manually forward the each configuration option.
Breaking PoLP gives us the ability to add abstractions and integrations. And today, these 2 are lacking:

Integration is completely left behind in NixOS. One will provide a module, but not plug it into other components, the minimal set you will find is that it will register a systemd service. Great⸮ This is what every over distribution provide out there! While for example, each module could provide an nginx/apache backend for all services which are requiring a website to work. No …, let's document that on a website a redo the same mistakes which motivated me to create the module system in the first place … what a shame!
Abstraction is also left out of NixOS, the only level of abstractions we have today is to provide installation devices. Great⸮ These already existed a decade ago! In the mean time the community is creating additional forks of NixOS, to provide curated version of it such as Musnix, Simple NixOS Mailserver or SelfPrivacy.

Unfortunately, this seems to be a general trend with Nix and NixOS these days, which is to make it good for DevOps but aweful for the rest of the world. Just because the few companies who are paying for making a mess in it only use it for DevOps should not imply that we degrade the experience of everybody else.
So, yes, we could add PoLP and make sure everybody learns about all the settings of everything, and that NixOS become THE super humans distribution who have absolute knowledge of everything, because we are too shy to make integration/abstraction choices.
I understand why DevOps prefer to have PoLP than trusting other developers. One reason is a flaw in how people operate today, which is that they want the latest versions of everything including bugs, but they do not want these bugs to mess-up other things.

Not having PoLP implies that we trust each others, we trust that developers are able to curate good settings, we trust that the integrations is well selected.
Not having PoLP means that we reduce the cognitive load of our users, with a risk induced by the trust we put in developers of NixOS.
Having PoLP implies that we validate every other changes that people are making, and selectively integrate each change at our discretion, even if this means adding more bugs in the process, but these would be our bugs and not others bugs.

At last … if you do not want to live in a world of trust, we can easily add a function to sandbox imported modules, and there would be no need for adding PoLP in NixOS, nor making another language from scratch … But then you would be in charge of validating every imported options, and risk having inconsistent configurations.
An alternative to having PoLP is to use features provided by the Linux kernel to sandbox applications, such as using hardening flags offered by systemd … if only we have a single flag to toggle that on all systemd services …
My values are towards reducing the cognitive load of NixOS users! Trust has to be part of the equation as I am personally unable to validate the hundred millions lines of code involved in building a running system.

Breaking PoLP gives us the ability to add abstractions and integrations. And today, these 2 are lacking:
* Integration is completely left behind in NixOS. One will provide a module, but not plug it into other components, the minimal set you will find is that it will register a systemd service. Great⸮ This is what every over distribution provide out there! While for example, each module could provide an nginx/apache backend for all services which are requiring a website to work. No …, let's document that on a website a redo the same mistakes which motivated me to create the module system in the first place … what a shame!

* Abstraction is also left out of NixOS, the only level of abstractions we have today is to provide installation devices. Great⸮ These already existed a decade ago! In the mean time the community is creating additional forks of NixOS, to provide curated version of it such as [Musnix](https://github.com/musnix/musnix), [Simple NixOS Mailserver](https://gitlab.com/simple-nixos-mailserver/nixos-mailserver) or [SelfPrivacy](https://git.selfprivacy.org/SelfPrivacy/selfprivacy-nixos-config).

Thanks for writing that @nbp! Especially +1 to the above quoted points. :heart:

Before the Principle of least Privilege, there must be a layer left for creativity. Currently NixOS lacks this layer. The writer is working on a NixOS-bootstrapped solution-fork, but this kind of PR would greatly ease his work, so he is giving it his approval.
PS. Earth is a level and motionless plane surrounded by an electromagnetic torus-field. Look it up, but not on google/meta, since those platforms are tools of the globe-perpetrating lie which began a few centuries ago. The ancients knew Earth's true shape, and they built the Great Pyramid, which nobody could dream of building today. The evidence is overwhelming, once the mind has opened up to it. The gag reflex is the main mechanism which was developped to keep the globe-lie in motion. Don't breathe too much vacuum, or get spun off! ;)

@LiGoldragon Thank you for your kind message. You are free to have your own beliefs, but from one person to another, I highly recommend not to use discussions like this one to advertise such topics, because it brings into question why you are participating in the Nix community. Beliefs, whatever they are, however truthful they are, are better conveyed between friends, or in person, or in a context where such a discussion is expected. Technical discussions like those on GitHub or most NixOS matrix channels are not that. Please be considerate of others, respect their beliefs and respect their time. I wish you all the best.

@roberth Sorry, I don't know what beliefs you are referring to. Nor do I understand any of what you wrote. Best of luck. Drink your urine, and maybe you'll make more sense after a few days of urine fast (no food, only water and auto-urine). Power awaits those who are ready to step forward. The whole world is my friend.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/is-it-possible-to-define-systemd-services-in-a-submodule/39538/5


feat: list.toggleKeyboardNavigation => list.find

closes #995

I was also looking for some bindings to select the file after opening. I have found two options:
            {
                "key": "enter",
                "command": "list.select",
                "when": "listFocus && inputFocus && listSupportsFind && treeFindOpen"
            },

This directly opens the file, but leaves the find widget open.
            {
                "key": "enter",
                "command": "list.closeFind",
                "when": "listFocus && inputFocus && listSupportsFind && treeFindOpen"
            },

This closes the find widget, but requires two presses of enter to open the file.
I vote for the second option.

Ref: https://github.com/microsoft/vscode/pull/152481

I've updated the PR, but for the enter, I think this feature is lacking from upstream.

@justinmk From my side, no.
@theol0403 Is there anything missing?

Why can't you add one of the enter bindings? Don't they fix the problem? Just because you made an issue on VSCode does not mean it will be addressed, in fact, I can't think of a single feature request made by this project that wasn't ignored by vscode devs.

@theol0403 Added.


CSS

CSS
User Story
As a bootcamp student
I want the prework notes to be structured on a webpage
So that I can easily find and read the information
Acceptance Criteria
GIVEN  Prework Study Guide website
WHEN I view the study guide
THEN I see a dark blue header and footer, and four boxes with a shadow

Finished adding CSS notes


footer corretion

Removal and correction of white space that existed in the footer.
Closing open tags...

Hi @jirrezdex , thanks for the commit.
We close those tags here, because we temporary closed them here.
Or am I missing something?

Hello, I really liked the template and, in working with it, I noticed the tags without being closed, the action of closing them causes the footer to have the same dimensions on all pages. Already the not quite leaves the footer with different dimes on each page.

Yes, I can see why this would be confusing, so I updated the theme with a different approach plus some other features such as SEO, sitemap etc.
Thanks for the heads up!


Broken Discord Links

Hi, in the repo there are broken links directing to https://pse.dev/discord
The page could be fixed or I can also open a PR and update those links to https://discord.com/invite/sF5CT5rzrR

Hey @omahs! Thank you for pointing it out. Yes, feel free to open a PR. Should I assign you this issue?

@vplasencia Sure! Just opened the PR!


Add method to check component name duplication

If components (e.g., variables, constraints, costs) contain components with the same name, it seems that an unexpected problem will arise (is this understanding correct?)
This Pull Request adds a method to check the name duplication to the Problem class. Doing the same thing in user code is difficult because variables_ is private and there is no access method.
(This is a utility that I use locally. It may be worth sharing in the origin repository, so I'll make a pull request.)

@awinkler What do you think about updating the Composite to store a std::set<std::string> of the names and when adding a component it inserts the name into the set and check the results to see if it was successfully inserted and if not throw an exception?


How can i use codes to implement the 64*64 diffusion model in DDPM-segmentation

In my experiments, i want to use 64*64 image in DDPM-segmentation bue to limited allocation, but code shows "RuntimeError: Error(s) in loading state_dict for UNetModel "

when i use "dim":[256,256,8448] , it occurs [enforce fail at CPUAllocator.cpp:68] . DefaultCPUAllocator: can't allocate memory: you tried to allocate<PHONE_NUMBER>0 bytes. Error code 12 (Cannot allocate memory)

This code doesn't support segmentation for 64x64 diffusion models out of the box but it should be easy to adapt it. Just make sure that you create a correct model for 64x64 images and change the configs accordingly. Regarding "dim":[256,256,8448], replace it with [64,64,].

thanks for your reply, i also want to know how could i use my datasets to train a brand new model from the beginning, and my dataset has a bigger size. if i put all my dataset into featrue_extracter it cost immensely cpu memory. So I'm very eager to know the solution.

preprocess it by batches and store on the disk. Then, one can create a dataset class that access data from the disk without loading all of it at once.

This code doesn't support segmentation for 64x64 diffusion models out of the box but it should be easy to adapt it. Just make sure that you create a correct model for 64x64 images and change the configs accordingly. Regarding "dim":[256,256,8448], replace it with [64,64, (your number of features)].

Hi! May I ask how to set the number of features? Or how is your 8448 calculated?

 In the paper it says all the features from the middle blocks are upsampled to the image size and then concate. In this way, why is the feature number here not 256*256? And I assumed that steps only indicate the denoise progress, what is its relationship with features? Thanks in advance for your answer.

We consider pixel-wise features. The activations after upsampling have 256x256xC dims. Thus, the feature vector for each pixel has a size C.


Add method write_output

Motivation:
Add the method write_output, needed to print the program's output when the output builtin is used in a cairo program
Changes:
Add method write_output
Add call to write_output in cairo_run
Add tests
Checklist

[ ] Linked to Github Issue
[x] Unit tests added
[x] Integration tests added.
[ ] This change requires new documentation.

[ ] Documentation has been added/updated.

Codecov Report

Merging #106 (091f6e0) into main (7a20cbe) will decrease coverage by 0.04%.
The diff coverage is 94.33%.

@@            Coverage Diff             @@
##             main     #106      +/-   ##
==========================================
- Coverage   97.17%   97.13%   -0.05%
==========================================
  Files          20       20
  Lines        3327     3380      +53
==========================================
+ Hits         3233     3283      +50
- Misses         94       97       +3

Impacted Files
Coverage Δ

src/vm/runners/cairo_runner.rs
99.66% <94.23%> (-0.25%)
:arrow_down:

src/cairo_run.rs
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7a20cbe...091f6e0. Read the comment docs.


With PDF output, extra page is added on end

When I convert an AsciiDoc to PDF, it adds an extra blank page on the end. This does not happen at all with HTML output, the HTML output ends right after the end of my last paragraph.

Hi @kholdstayr, AsciidocFX uses Docbook for PDF generation. You should customize Docbook XSLTs for plus needs.


Add test discovery and NoBuild option to test requests

More info about this in the related VSCode extension PR:
https://github.com/OmniSharp/omnisharp-vscode/pull/3601
This adds

A DiscoverTests request, which lets you get info for all tests in an assembly
A NoBuild option for test requests, which lets you run/debug tests without always building first

There is a separate PR with just the runsettings additions.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: Samirat sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@Samirat Thanks!


Sample test classes write files into the subproject's root directory so that the file trees get dirty

The following grpah shows the tree of the project root and -L 2
:~/github/selenium-webdriver-java (kazurayam8)
$ tree -L 2 .
.

├── selenium-ide
│   └── hello-selenium-ide.side
├── selenium-webdriver-junit4
│   ├── build
│   ├── build.gradle
│   ├── gradle
│   ├── gradlew
│   ├── gradlew.bat
│   ├── pom.xml
│   ├── src
│   └── target
├── selenium-webdriver-junit5
│   ├── build.gradle
│   ├── gradle
│   ├── gradlew
│   ├── gradlew.bat
│   ├── pom.xml
│   └── src
├── selenium-webdriver-junit5-seljup
│   ├── build.gradle
│   ├── gradle
│   ├── gradlew
│   ├── gradlew.bat
│   ├── pom.xml
│   └── src
├── selenium-webdriver-testng
│   ├── build.gradle
│   ├── gradle
│   ├── gradlew
│   ├── gradlew.bat
│   ├── pom.xml
│   └── src
├── settings.gradle

Them executed all the test cases of subprojects.
$ ./gradlew clean test

This task ran lon minutes. After all the test cases finished, I retrieved the directory tree under the selenium-webdriver-junit4
.
├── 2023.10.24_<IP_ADDRESS>2-7440524241d0dbd63ca5eec377b6455c.png    --- x
├── 2023.10.24_<IP_ADDRESS>3-7440524241d0dbd63ca5eec377b6455c.png    --- x
├── build
│   ├── allure-results
│   ├── classes
│   ├── downloads
│   ├── generated
│   ├── reports
│   ├── resources
│   ├── test-results
│   └── tmp
├── build.gradle
├── extentReport.html    --- x
├── fullpage-screenshot-chrome.png    --- x
├── gradle
│   └── wrapper
├── gradlew
├── gradlew.bat
├── login.har    --- x
├── my-pdf.pdf    --- x
├── pom.xml
├── screenshot.png    --- x
├── src
│   ├── main
│   └── test
├── target
│   ├── classes
│   ├── generated-sources
│   ├── generated-test-sources
│   ├── maven-status
│   └── test-classes
├── testAccessibility.json    --- x
├── webdrivermanager.pdf    --- x
├── webdrivermanager.png    --- x
└── webelement-screenshot.png    --- x

20 directories, 15 files

After the test executed, there are many new files under the subproject's root directory. I executed git status command to find the files newly created by the tests
$ git status
On branch kazurayam8
Untracked files:
  (use "git add <file>..." to include in what will be committed)
        2023.10.24_<IP_ADDRESS>2-7440524241d0dbd63ca5eec377b6455c.png
        2023.10.24_<IP_ADDRESS>3-7440524241d0dbd63ca5eec377b6455c.png
        extentReport.html
        fullpage-screenshot-chrome.png
        login.har
        my-pdf.pdf
        screenshot.png
        testAccessibility.json
        webdrivermanager.pdf
        webdrivermanager.png
        webelement-screenshot.png

These new files are not gitignored. If I carelessly do the following operation:
$ git add .
$ git commit -m "save changes"

then the newly created temporary files will be commited into the project tree. A disastrous plution of the project tree!
The sample tests in the book should not write files into the project's root directory.
The tests should write files into a particular directory in the project's root directory. You want to list the directroy in the .gitignore file so that Git ignores all the test outputs silently.

At first, I need to identify which test scripts write the dirty temporary files. I have to read the source codes.

   2023.10.24_<IP_ADDRESS>2-7440524241d0dbd63ca5eec377b6455c.png <=

   2023.10.24_<IP_ADDRESS>3-7440524241d0dbd63ca5eec377b6455c.png <=

   extentReport.html <=

   fullpage-screenshot-chrome.png <=

   login.har <=

   my-pdf.pdf <=

   screenshot.png <=

   testAccessibility.json <=

   webdrivermanager.pdf <=

   webdrivermanager.png <=

   webelement-screenshot.png <=

In which directory under the project root should I house the output files from tests?
As for Maven, https://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html writes

The target directory is used to house all output of the build.

As for Gradle, https://docs.gradle.org/current/userguide/directory_layout.html#dir:project_root writes

The build directory of this project into which Gradle generates all build artifacts.

I looked at https://github.com/kazurayam/selenium-webdriver-java/blob/master/.gitignore
It contains a line
test-output

I believe that @bonigarcia had an intention to create the test-output directory.
However, the current test classes are written so that they write the files into the project's root directory, rather than the test-output directory.
Then, I would modify the test class so that they write files into the test-output directory.

How should I change the existing sources so that they write files into the new test-output directory?
At first I should study the existing code. For example, https://github.com/kazurayam/selenium-webdriver-java/blob/master/selenium-webdriver-junit4/src/test/java/io/github/bonigarcia/webdriver/junit4/ch04/event_listeners/MyEventListener.java has the following fragment, which decides the path to write the file into
        SimpleDateFormat dateFormat = new SimpleDateFormat(
                "yyyy.MM.dd_HH.mm.ss.SSS");
        String screenshotFileName = String.format("%s-%s.png",
                dateFormat.format(today), sessionId.toString());
        Path destination = Paths.get(screenshotFileName);

In the selenium-webdriver-junit5 subproject, I executed
$ mvn test

that is, I executed full set of tests by Maven build. The result was as follows:

Just similar set of output files were created in the project root dir as the selenium-webdriver-junit4 subproject.

I need to be careful about the "Current Working Directory". Now I have a statement in my test class source:
    Path static testOutputDir = Paths.get("test-output");

This assumes that the "Current Working Directory" is given with a value equal to the directory selenium-webdriver-java/selenium-webdriver-junit4.
However, this assumption is too fragile according to the runtime environment. In https://discuss.gradle.org/t/how-do-i-set-the-working-directory-for-testng-in-a-multi-project-gradle-build/7379/7,

luke_daley
Gradle Employee
Nov '13
Loading from the filesystem using relative paths during unit tests is problematic because different environments will set a different working directory for the test process. For example, Gradle uses the projects directory while IntelliJ uses the directory of the root project.
The only really safe way to solve this problem is to load via the classpath.

I have developed a new class test/java/com.kazurayam.webdriver.TestHelper with a method Path resolveOutput(String fileName).
I will commit this in the kazurayam8a branch.

As you see in

https://github.com/kazurayam/selenium-webdriver-java/commit/d347900a30d515beb459184a0083f6ba5368a9ac

kazurayam developed a package of classes: com.kazurayam.unittest.TestHelper, ProjectDirectoryResolver and StringSequence. This package resolves this #8 issue.
However this is an issue. This package is contained in the selenium-webdriver-java/selenium-webdriver-junit4 project. On the other hand, the other projects *-junit5, *-junit5seljup and *-testng will also require this package.
I do not like to duplicate the source code of the com.kazurayam.unittest.TestHleper package. Therefore I will move this package into a new independent project. The project name will be kazurayam-unittest-helpe-x.x.x.jar.

I have created a project "com.kazrurayam:unittest-helper:0.1.0"
https://github.com/kazurayam/unittest-helper
I have published the jar to the Maven Central repository, which will become available for download in a few days.

I will modify the branches kazurayam8, kazurayam8a, kazurayam8b, kazurayam8c, kazurayam8d so that the use the "com.kazrurayam:unittest-helper:0.1.0" as one of their dependencies downloaded from the maven repositories.

https://github.com/kazurayam/selenium-webdriver-java/issues/21
I found that the sample codes of the Chapter 9 "File Download" section are sensitive. There are 3 tests which write into output files.
One writes a file into the "Downloads" dir under the System.getProperty("user.home") = OS User Directory.
One writes a file into the dir as new File(".")
One writes a file into the file as new File("", fileName)
It seems that the book author, @bonigarcia, had a specific intention to show the ways how to locate the output files, but I am not sure.

We would need a sample code that writes a file into the "Downloads" directory of a user.
Should I modify the samples that uses new File(".") ?      ------- I wondered a while ----- Yes, I would propose to do so.

I have got an additional requirement. I want to create subdirectory under the output dir. The subdirectory could be any string, and for example a Fully Qualified Class Name of the test case class.
Once implemented then the output would look like this:

selenium-webdriver-java/selenium-webdriver-junit4/test-output/io.github.bonigarcia.webdriver.junit4.event_listener.MyEventListener/2023.10.24_<IP_ADDRESS>2-7440524241d0dbd63ca5eec377b6455c.png

selenium-webdriver-java/selenium-webdriver-junit4/test-output/io.github.bonigarcia.webdriver.junit4.event_listener.MyEventListener/2023.10.24_<IP_ADDRESS>3-7440524241d0dbd63ca5eec377b6455c.png

I have managed to make the following test-output. Please note that

all output files created by test class are written into  a single dedicated directory test-output. No file is written in the project's root directory

Under the test-output directory, we have a layer of directories which represents the Fully Qualified Class Name of the test classes.

I think that this file organization is nice.

The unittest-helper project published a new version v0.3.0
https://github.com/kazurayam/unittest-helper/releases/tag/0.3.0
The API has been changed. The selenium-webdriver-java project should use the unittest-helper v0.3.0.
I will modify the kazurayam8consolidated branch.


[material-ui][docs] Update "Style library interoperability" doc for Tailwind CSS + Next.js App Router

Related page
https://mui.com/material-ui/guides/interoperability/#tailwind-css
Kind of issue
Unclear explanations
Issue description
I was looking for how to integrate material-ui together with nextjs 14 app router and tailwind and it appears that following the guide does not results in a working configuration.
Step three tells you to add the important option using the id of your app wrapper:

Add the important option, using the id of your app wrapper. For example, #__next for Next.js and "#root" for CRA:

The #__next id is only valid for the page router of Nextjs. It is not present in the app router. There is no element at all with an id, so people should insert this them self in their root layout
Context 🔦
No response
Search keywords: interoperability nextjs tailwind app router

Hello,
I found this issue because I developed my entire project using Tailwind and MaterialUI, and it was at build time that I realised that there were differences between the build version and the development version.
I finally realised that this was due to the combination of MaterialUI and Tailwind.
I'm using NextJS 14 with an app router, and given the changes to be made in the documentation, I don't know what to do to solve my problem.
I tried to do this:

Apply the steps described here: https://mui.com/material-ui/integrations/nextjs/, the problems are now the same in production and development.
Then apply the steps described here: https://mui.com/material-ui/integrations/interoperability/#tailwind-css , now depending on the case, everything is displayed without any CSS or everything is strange (depending on the order of my tailwind config, I don't understand).
As explained in this issue, I tried to apply "important: '#root' " in my tailwind config by adding an id to my root div myself. But no results.

Have I misunderstood, and can I only use the preflight of one or the other in the build version? In which case I have to change the style of my entire project again? Or have I misunderstood something else?
Perhaps you have an example project?
Thank you very much for your help ! :)

Does this example help short out your problem?

Does this example help short out your problem?
Yes, thank you, I've been able to solve the problem.
I've also better understood the value of preflights and made the necessary configurations to transpose my project more easily. Thanks a lot!

There's another point that could be made clearer in the docs regarding integration with Next.js. If one uses the enableCssLayer: true option explained in MUI integration with Next.js do we still need to use the <StyledEngineProvider injectFirst>... mentioned in step 4 in MUI integration with TailwindCSS ?


add encryption utility

This probably doesn't quite fit with conventions but hopefully is sufficient to start.

I wonder if the builder API for producing either signed or encrypted or signed and then encrypted JWTs is really the way to go, see #109. The implementations will have a choice which JOSE library to use, etc


docs: clean up terminology authorization token->access token

This addresses a comment received via e-mail about mixed terminology referencing the token used in requests to resource servers.

LGTM

In nmos_sec_3.png should 2- Bearer Token (Contains auth.token) be 2- Bearer Token (Contains Access Token)?

Well spotted. I'll adjust.

Final review on call


Add "Community" tab to site navbar

With the addition of our Discussions tab, I think it may prove beneficial to add a Community tab to the site navbar. This way, users will:

Have easy access to Discussions
Visit our repository and see the organization with its code
Help us build our own community and find an outlet to ask questions, post ideas, see the current activity, and talk about their own modules

Closing this, as it is unnecessary & will crowd the navbar at this point in time.


Style Utilities : Display not valid

In examples
every example name is in the wrong place.
Need a more detailed description.
Need reference document for testing bootstrap doc is not sufficient for this particular section.

The Names are shifted to bottom, and Box1, Box2 label added.
The description is self- explanatory.
please refer to display properties from bootstrap for better understanding.


🍻 Delete lots of tests to check if coverage badge points to broken PRs

🍻 Delete lots of tests to check if coverage badge points to broken PRs

It seems badges still point to master with 100% coverage and not to the latest PR, so we are good! Closing this now.


Add ListObjectsV2 API

But I guess we want to keep the API simple

Let's mention why its not implemented.


Product // Collaborations // Cambrian Protocol

TD to contact Nic regarding collaboration with Cambrian Protocol:
https://cambrianprotocol.com/
Comms through Discord.

30/08/2022
@beam-mw - Nic is coming back to me to arrange a time


TypeError: MyPuppet is not a constructor

I'm getting the TypeError: MyPuppet is not a constructor error when running wechaty after upgrading it:
cd wechaty-getting-started/examples

$ git pull
Already up to date.
Current branch master is up to date.

$ WECHATY_PUPPET=puppeteer node starter-bot.js
00:56:00 INFO PuppetManager install(puppeteer@latest) done
TypeError: MyPuppet is not a constructor
    at Function.<anonymous> (/path/to/wechaty-getting-started/examples/node_modules/wechaty/dist/src/puppet-manager.js:58:34)
    at Generator.next (<anonymous>)
    at fulfilled (/path/to/wechaty-getting-started/examples/node_modules/wechaty/dist/src/puppet-manager.js:4:58)
00:56:00 ERR Wechaty start() exception: MyPuppet is not a constructor
TypeError: MyPuppet is not a constructor
    at Function.<anonymous> (/path/to/wechaty-getting-started/examples/node_modules/wechaty/dist/src/puppet-manager.js:58:34)
    at Generator.next (<anonymous>)
    at fulfilled (/path/to/wechaty-getting-started/examples/node_modules/wechaty/dist/src/puppet-manager.js:4:58)

Here is more info:
$ node -v
v11.5.0

$ npm -v
6.5.0

$ npm -g outdated | wc
      0       0       0

`$ npm outdated | wc
      0       0       0

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.1 LTS
Release:	18.04
Codename:	bionic

Full Output Logs

Show Logs

All modules were freshly installed after node/npm update, which begins with
rm package*
$ WECHATY_LOG=silly WECHATY_PUPPET=puppeteer node starter-bot.js
01:03:52 SILL Config WECHATY_LOG set level to silly
01:03:52 INFO Config registering process.on("unhandledRejection") for development/debug
01:03:52 VERB Config constructor()
01:03:53 SILL Accessory #0<Wechaty> constructor()
01:03:53 VERB Wechaty contructor()
01:03:53 VERB StateSwitch constructor(name=Wechaty)
01:03:53 VERB StateSwitch constructor(name=WechatyReady)
01:03:53 VERB Wechaty on(scan, function) registered
01:03:53 VERB Wechaty onFunction(scan)
01:03:53 VERB Wechaty on(login, function) registered
01:03:53 VERB Wechaty onFunction(login)
01:03:53 VERB Wechaty on(logout, function) registered
01:03:53 VERB Wechaty onFunction(logout)
01:03:53 VERB Wechaty on(message, function) registered
01:03:53 VERB Wechaty onFunction(message)
01:03:53 SILL Wechaty version() form development environment is not availble: ENOENT: no such file or directory, stat '/path/to/wechaty-getting-started/examples/node_modules/wechaty/dist/.git'
01:03:53 INFO Wechaty <puppeteer> start() v0.22.6 is starting...
01:03:53 VERB Wechaty puppet: undefined
01:03:53 VERB Wechaty profile: undefined
01:03:53 VERB Wechaty id: cjpz261cu0000xtfu5a8r7fzu
01:03:53 SILL StateSwitch <Wechaty> on() is false
01:03:53 SILL StateSwitch <WechatyReady> off() is true
01:03:53 VERB StateSwitch <WechatyReady> off(true) <- (true)
01:03:53 SILL StateSwitch <Wechaty> on() is false
01:03:53 VERB StateSwitch <Wechaty> on(pending) <- (false)
01:03:53 VERB MemoryCard constructor(undefined)
01:03:53 VERB MemoryCard getStorage() for storage type: N/A
01:03:53 VERB MemoryCard load() from storage: N/A
01:03:53 VERB MemoryCard load() no storage
01:03:53 VERB Wechaty initPuppet()
01:03:53 VERB MemoryCard multiplex(puppet)
01:03:53 VERB MemoryCard static multiplex(MemoryCard<>, puppet)
01:03:53 VERB MemoryCard constructor({"multiplex":{"name":"puppet","parent":{"payload":{},"multiplexNameList":[]}}})
01:03:53 VERB PuppetManager resolve({wechaty: Wechaty#cjpz261cu0000xtfu5a8r7fzu<>(), puppet: puppeteer, puppetOptions: undefined})
01:03:53 VERB PuppetManager resolveName(puppeteer)
01:03:53 VERB PuppetManager checkModule(puppeteer)
01:03:53 SILL PuppetManager checkModule() puppeteer installed version 1.11.0 NOT satisfied range undefined
01:03:53 INFO PuppetManager install(puppeteer@latest) please wait ...
npm:
><EMAIL_ADDRESS>install /path/to/wechaty-getting-started/examples/node_modules/wechaty/node_modules/puppeteer
> node install.js

npm:

npm: Chromium downloaded to /path/to/wechaty-getting-started/examples/node_modules/wechaty/node_modules/puppeteer/.local-chromium/linux-609904

npm: +<EMAIL_ADDRESS>updated 1 package and audited 64 packages in 23.95s

npm: found 0 vulnerabilities

01:04:18 INFO PuppetManager install(puppeteer@latest) done
TypeError: MyPuppet is not a constructor
    at Function.<anonymous> (/path/to/wechaty-getting-started/examples/node_modules/wechaty/dist/src/puppet-manager.js:58:34)
    at Generator.next (<anonymous>)
    at fulfilled (/path/to/wechaty-getting-started/examples/node_modules/wechaty/dist/src/puppet-manager.js:4:58)
01:04:18 ERR Wechaty start() exception: MyPuppet is not a constructor
TypeError: MyPuppet is not a constructor
    at Function.<anonymous> (/path/to/wechaty-getting-started/examples/node_modules/wechaty/dist/src/puppet-manager.js:58:34)
    at Generator.next (<anonymous>)
    at fulfilled (/path/to/wechaty-getting-started/examples/node_modules/wechaty/dist/src/puppet-manager.js:4:58)

$ npm list --depth=1
/path/to/wechaty-getting-started/examples
├──<EMAIL_ADDRESS>├─┬<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>extraneous
│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ └──<EMAIL_ADDRESS>├─┬<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>deduped
│ ├──<EMAIL_ADDRESS>deduped
│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>deduped
│ ├──<EMAIL_ADDRESS>deduped
│ └──<EMAIL_ADDRESS>deduped
└─┬<EMAIL_ADDRESS>  ├──<EMAIL_ADDRESS>  ├──<EMAIL_ADDRESS>deduped
  ├──<EMAIL_ADDRESS>deduped
  ├──<EMAIL_ADDRESS>deduped
  ├──<EMAIL_ADDRESS>  ├──<EMAIL_ADDRESS>  ├──<EMAIL_ADDRESS>deduped
  ├──<EMAIL_ADDRESS>  ├──<EMAIL_ADDRESS>deduped
  ├──<EMAIL_ADDRESS>deduped
  ├──<EMAIL_ADDRESS>deduped
  ├──<EMAIL_ADDRESS>  ├──<EMAIL_ADDRESS>  ├──<EMAIL_ADDRESS>deduped
  ├──<EMAIL_ADDRESS>deduped
  └──<EMAIL_ADDRESS>
Try again with a cleaner installation:
rm -fr node_modules
rm -f package*

$ npm install wechaty
+<EMAIL_ADDRESS>added 8 packages from 10 contributors, removed 580 packages, updated 38 packages, moved 1 package and audited 216 packages in 32.849s
found 0 vulnerabilities

$ npm install wechaty-puppet-puppeteer

><EMAIL_ADDRESS>install /path/to/wechaty-getting-started/node_modules/puppeteer
> node install.js

Downloading Chromium r609904 - 106.4 Mb [====================] 100% 0.0s
Chromium downloaded to /path/to/wechaty-getting-started/node_modules/puppeteer/.local-chromium/linux-609904
+<EMAIL_ADDRESS>added 87 packages from 76 contributors and audited 431 packages in 31.81s
found 0 vulnerabilities

$ WECHATY_PUPPET=puppeteer node starter-bot.js
09:24:13 INFO Wechaty <puppeteer> start() v0.22.6 is starting...
09:24:13 INFO PuppetManager install(puppeteer@latest) please wait ...
npm:
><EMAIL_ADDRESS>install /path/to/wechaty-getting-started/node_modules/wechaty/node_modules/puppeteer
> node install.js

npm:

npm: Chromium downloaded to /path/to/wechaty-getting-started/node_modules/wechaty/node_modules/puppeteer/.local-chromium/linux-609904

npm: npm
npm:
npm: notice created a lockfile as package-lock.json. You should commit this file.

npm: +<EMAIL_ADDRESS>added 43 packages from 22 contributors and audited 64 packages in 28.327s

npm: found 0 vulnerabilities

09:24:43 INFO PuppetManager install(puppeteer@latest) done
TypeError: MyPuppet is not a constructor
    at Function.<anonymous> (/path/to/wechaty-getting-started/node_modules/wechaty/dist/src/puppet-manager.js:58:34)
    at Generator.next (<anonymous>)
    at fulfilled (/path/to/wechaty-getting-started/node_modules/wechaty/dist/src/puppet-manager.js:4:58)
09:24:43 ERR Wechaty start() exception: MyPuppet is not a constructor
TypeError: MyPuppet is not a constructor
    at Function.<anonymous> (/path/to/wechaty-getting-started/node_modules/wechaty/dist/src/puppet-manager.js:58:34)
    at Generator.next (<anonymous>)
    at fulfilled (/path/to/wechaty-getting-started/node_modules/wechaty/dist/src/puppet-manager.js:4:58)

Yes, you should use WECHATY_PUPPET=wechaty-puppet-puppeteer


 I was planning on looking at the AuthZ middleware at some point and it would be nice to know this makes a reasonable difference before I dive in

For my own benefit, how much of an impact does this have on throughput?

Depends how many; ResourceInvoker was quite good (+3%) https://github.com/aspnet/AspNetCore/pull/9311 but that removed 6; so maybe +0.5% per item on a dotnet new mvc + the PlainTextActionResult
Would be less if the controller was doing more; so it had less impact on Json for example


spatial/r3: add Jacobian layout description to doc comment

Please take a look.

Codecov Report

Merging #1829 (813c6ef) into master (0bc95eb) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1829      +/-   ##
==========================================
+ Coverage   73.47%   73.50%   +0.02%
==========================================
  Files         524      526       +2
  Lines       61939    62025      +86
==========================================
+ Hits        45512    45593      +81
- Misses      13866    13868       +2
- Partials     2561     2564       +3

Impacted Files
Coverage Δ

spatial/r3/mat.go
80.40% <ø> (ø)

graph/coloring/coloring.go
89.72% <0.00%> (-0.61%)
:arrow_down:

graph/iterator/nodes_map_safe.go
91.30% <0.00%> (ø)

graph/iterator/lines_map_safe.go
100.00% <0.00%> (ø)

graph/formats/rdf/graph.go
71.68% <0.00%> (+0.45%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Exit code zero when wranger fails due to permissions error.

🐛 Bug report
Describe the bug
When wrangler is run and exits due to a permission error, the exit code is zero.
Reproduce the bug
A list of steps to reproduce the bug.

Checkout any wrangler project with NPM dependencies (but do not install NPM packages)
CD into the project
Change to a user that does not have permission to write to the project
Run wrangler publish
Wrangler will fail with a permissions error when it tries to call npm install

Expected behavior
I should get a non-zero exit code.
Environment and versions

operating system: Docker, Alpine 3.11.12
output of wrangler -V: 1.19.3
output of node -v: 14.18.0
content of wrangler.toml: REDACTED

Implications
This causes automated deployments to consider the command a success when it has actually failed.

I believe we've fixed this wrangler v2, so I'm going to close this issue.


Fix Typo

Just found this while looking through the docs because of the latest release.

Pull Request Test Coverage Report for Build 389

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 388:
0.0%

Covered Lines:
1619

Relevant Lines:
1619

💛  - Coveralls


replaces 'bundle --path' with 'bundle config set --local path'

The --path flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use bundle config set --local path 'vendor/bundle', and stop using this flag

@tkembo : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@tkembo Thank you for submitting. This specific section is to tell you how the Oryx build automation works. To recommend a change, you should submit an issue directly at https://github.com/microsoft/Oryx. The current documentation still stands. #please-close


Add make in the base images

why?

So the users can still run their existing Makefile scripts without having to copy/paste the commands. If they are using the make things somewhere else, it makes sense to not ask them to duplicate the commands.

Make will be needed for Symfony 4: https://medium.com/@fabpot/symfony-4-best-practices-b4bbd6a9c994

^ yup, and I'm hitting the problem now :)

one thing I dislike about it is that it doesn't have a library way of providing make tasks. Therefore all our projects copy and paste in, which is exactly what Hem and the base image plan.sh was there to solve by allowing custom and shared tasks

closed by #254


Fix error behavior when binding session for the first time

~Changes:~

~If the initial bind-session network request fails due to a network error, the request will be automatically retried. (This is how the pre-emptive token refreshing system works already.)~
~Changed some internal APIs to remove unused arguments and make implementing the above easier~

Fix error behavior when binding session for the first time
Changes:

Improved the refresh handle logic to handle both initial binds and refreshing
Removed the code in RLMSyncUser.mm to handle initial binds
This means that initial binds that fail because of network conditions will use the same retry logic as refresh attempts

I need to think about this a bit more.

Okay, I think this can be reviewed (after #4570 is approved). It's 26 more lines of code, but the code is easier to understand now and logic isn't duplicated anymore. Let me know if there are any questions.

@tgoyne is right: the state stuff is unnecessary. I've changed the code a bit to reflect that.


dev-cmd/typecheck: Stop dry running Spoom sigil bumps

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.
[ ] Have you successfully run brew style with your changes locally?
[ ] Have you successfully run brew typecheck with your changes locally?
[ ] Have you successfully run brew tests with your changes locally?

Sixteen months ago in #14921, I made it so that CI ran brew typecheck --update --suggest-typed and could commit the changes.
Except it never actually ever made any changes because of the --dry option in the Spoom CLI args. Whoops!

Oh, heh, even when it does change files it still exits 1 so Actions thinks the step has failed.


This should bring in the current status of the pipeline - A brave new world!

PR checklist

[x] This comment contains a description of changes (with reason).
[x] If you've fixed a bug or added code that should be tested, add tests!

[x] If you've added a new tool - add to the software_versions process and a regex to scrape_software_versions.py
[x] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
[x] If necessary, also make a PR on the nf-core/pangenome branch on the nf-core/test-datasets repository.

[x] Make sure your code lints (nf-core lint .).
[x] Ensure the test suite passes (nextflow run . -profile test,docker).
[ ] Usage Documentation in docs/usage.md is updated.
[ ] Output Documentation in docs/output.md is updated.
[ ] CHANGELOG.md is updated.
[x] README.md is updated (including new tool citations and authors/contributors).

Hi @FriederikeHanssen,
as discussed today the PR from my DSL2 branch into dev. You already reviewed this in https://github.com/nf-core/pangenome/pull/115 so it should be ready to go. Documentation will fly in in the future.

It seems one status is missing, although all tests ran through successfully. Any ideas?


CHEBI:88763, to delete or to merge with CHEBI:196788

Hello ChEBI Team,
CHEBI:196788 cholest-5-en-3beta-yl eicosanoate is identical to SLM:000500285
https://www.swisslipids.org/#/entity/SLM:000500285/?classification
and its definition is the same as the one for CHEBI:88763 CE(20:0), which has erroneous synonym(s), such as 'Eicosanoate'
Could you consider merging (or deleting ?) CHEBI:88763?
I hope, I do not miss something here because in SwissLipids I see that SLM:000500285 eicosanoyl-cholesterol has 2 parents ?
Thanks,
Cheers,
Anne

Now fixed. Thanks


[SPARK-23555][PYTHON] Add BinaryType support for Arrow in Python

What changes were proposed in this pull request?
Adding BinaryType support for Arrow in pyspark.
How was this patch tested?
Additional unit tests in pyspark for code paths that use Arrow

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1248/
Test PASSed.

Test build #87919 has started for PR 20725 at commit afcb0d5.

This is a WIP as some issues need to be worked out on the Arrow side and need tests for pandas_udfs.  Currently get the following error when converting from pandas:
File "/home/bryan/git/spark/python/pyspark/serializers.py", line 237, in create_array
    return pa.Array.from_pandas(s, mask=mask, type=t)
  File "array.pxi", line 335, in pyarrow.lib.Array.from_pandas
  File "array.pxi", line 170, in pyarrow.lib.array
  File "array.pxi", line 70, in pyarrow.lib._ndarray_to_array
  File "error.pxi", line 85, in pyarrow.lib.check_status
ArrowNotImplementedError: No cast implemented from binary to binary

The corresponding JIRA is https://issues.apache.org/jira/browse/ARROW-2141

Test build #87919 has finished for PR 20725 at commit afcb0d5.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87919/
Test FAILed.

Merged build finished. Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1540/
Test PASSed.

Test build #93854 has started for PR 20725 at commit 45eea0c.

Test build #93854 has finished for PR 20725 at commit 45eea0c.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93854/
Test FAILed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2236/
Test PASSed.

Merged build finished. Test PASSed.

Test build #94838 has started for PR 20725 at commit 461c326.

Test build #94838 has finished for PR 20725 at commit 461c326.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94838/
Test PASSed.

Merged build finished. Test PASSed.

https://rise.cs.berkeley.edu/jenkins/job/ubuntuSparkPRB/79/

https://rise.cs.berkeley.edu/jenkins/job/ubuntuSparkPRB/80/

i've been having trouble getting this build to pass, and i'm testing a few other modules on top of yours.  if i have problems, i'll revert the extraneous changes and build yours.

still having problems getting this to pass:
[error] (sql-kafka-0-10/test:test) sbt.TestsFailedException: Tests unsuccessful
[error] (core/test:test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 14786 s, completed Aug 16, 2018 4:05:09 PM

i rebased upstream changes from the main spark repo on my fork and launched another build.  in ~5 hours we'll know how it went.  :\
if this fails tonite, i'll figure out a hacky way to test this tomorrow morning.

Awesome, thanks for running that @shaneknapp !!  I'll merge this later today if no further comments.


"Requests is the only Non-GMO HTTP library for Python, safe for human consumption."

GMO are safe

This is absolutely not the venue for this discussion.

@Lukasa too serious

@mouuff it's a joke :)

https://github.com/drtshock/Potato

@mouuff haha, behold—art!

What is GMO?
I'm wondering because it's in the first sentence of the repo's README and I've never heard of it and have no clue what it's supposed to mean in a programming context. I googled it and only came up with genetically modified organism. Is this some sort of in-joke I don't get?

Yes :)

I don't know the meaning either, I'm not a native English. @kerstin

@Celthi it's a joke :)

Oh, I see, :) :) :).

I think @kerstin's and @Celthi's reactions sums this up: this has no place in a README for such a widely depended on project.
Funny and confusing? Yes.
Professional? No.

Software is perfectly able to have a sense of humor.

I hadn't thought about it in terms of professionalism – though maybe more because I feel like I can't prescribe what is and what isn't in contexts like these –, but yeah, the problem I see with it is more that inside jokes like this can be pretty confusing for people who are not native speakers of English and/or programming novices, and thus feel exclusionary. ("Is this something I should know about? If I don't, is it a problem?")

I'm sorry to see that this issue has been closed out-of-hand. I absolutely agree that this is not the venue for a discussion of the merits of GMOs, but by including this sentence - whether it be a joke or not - you are implicitly aligning the requests project with one side of that debate.
If you still think this is just a joke rather than a political issue, consider how you would feel if it instead said "The only GMO-enhanced HTTP library for Python: nutritious and healthy". If your reaction to that sentence is also "haha, good joke" then well done: you really are impartial. But don't expect all your users to laugh it off in the same way.
Essentially, I consider your joke to be in poor taste. Perhaps we should also have a joke involving some racial stereotypes for completeness?
As others have already said: a software library should not be taking political sides. It's possible to have a sense of humour without doing so.

I don't think I agree with the notion that software/libraries shouldn't take political sides ever. There's certainly code out there which is written specifically to help activists or spread awareness (about politicial issues, political figures and their actions etc.) or to make people stop and think (a good bunch of art-related code, I'd guess). In my opinion all of that is valid and even important as long as the motivations behind it aren't unsavory.
(Following that, I don't get why one would equate a critical stance on a controversial topic with dystopian qualities – which is how I read it – with racial jokes.)
It's fine if we disagree on this, I just don't want my previous comments to be misinterpreted. As stated above, I believe the issue is that the joke is inaccessible to a good portion of people wanting to use/using the library due of the use of a potentially unfamiliar abbreviation and its general off-topic-ness, not because it's "taking a side".

Life's too short to get caught up by stuff like this. Just ignore the joke if you cannot handle it.

@laneschmidt your subtle pejorative is not missed. One could equally insult you with, "don't enter a conversation if you aren't mature enough to address people like adults" or even "delete your github account if you can't meaningfully add to a discussion".

Echoing what @cornfeedhobo said, I was wondering why you, @laneschmidt, would trigger email notifications for everyone subscribed to this thread so far for what's basically a non-comment, thereby wasting all of our time, when you feel life's too short for stuff like this...

Whether GMO is safe is a serious scientific discussion because of all the miss-information out there. Implying that this statement is a joke and blame people for not getting it shows poor taste.

To ensure that non native english speakers also understand this joke, no abbreviation should be used. A smiley at the end could also contribute to understanding. :-)

This joke is perfectly normal and should not be treated such seriously as "not professional" and confusing. This is just a tagline, nothing else.
@keikoro

This joke is perfectly normal and should not be treated such seriously as "not professional" and confusing. This is just a tagline, nothing else.
@keikoro

@episodeyang is right. There is a lot of miss-information out there and joke-or-not it contributes to it.
It's ignoring such little stuff is what got us into this pseudoscientific mess of a world we have right now.
On the other hand I do appreciate a joke when I see one, especially in professional surroundings.
I'd just replace it with something like "Not tested on animals... they were too scared by the snake." or alike.

Seems  to be finally resolved in the commit 7ca0e86a6752a5a763e5e82d94f34a522a976708.


[go_router] Deep link on desktop

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Use case
I thought deep-linking was supported on all platforms because GoRouter claims to support all platforms on pub.dev. I'm looking for a solution to handle deep-linking on all platforms, and the only option I found is to use GoRouter along with app_links. It's a bit inconvenient to use two libraries for the same purpose, especially when their approaches differ significantly.
Proposal
It would be nice to cover all deep-linking-related tasks in one library.
I understand that it might be required to do some things manually here and there. At the very least deep-linking handling behavior would be predictable and consistent.
If that's not possible it would be nice to have a heads up in the documentation.

Hi @Kiura. Thanks for proposing this. Looking at the documentation on deep linking, I can't see it supported on all platforms as you stated, see https://pub.dev/documentation/go_router/latest/topics/Deep linking-topic.html. Can you point me to more information on this?
Also, using app_links should suit this particular use case well.

Hi @dam-ease. Thx for the quick reply. I agree, the documentation for deep linking does not state support for desktop platforms. I was referring to the support of all platforms by the package, not just on deep linking.

So I am guessing, there are no plans to support Desktop platforms by the library, right?
app_links does solve this particular issue, although it seems like the library has limited support for Linux.

Thanks for your response. Since app_links solves this, I'd suggest using this but I'd label for more insights from the team. Thank you!

The desktop embedding need to be able to send deeplink request to the framework side, after that it should just work.

For Windows, deep links is a bit funky:

The app needs to add a registry key for the app's custom scheme (like myapp://)
When the user opens the deep link, Windows launches a new process for the app...

If this is the app's first process, the newly launched process should navigate to the deep link
If there are already other running processes for the app...

The newly launched process should pick an existing process and notify it of the deep link
The newly launched process should exit

It looks like for app_links package there is some manual set up in windows, is this something we should add to windows embedding?

It looks like for app_links package there is some manual set up in windows, is this something we should add to windows embedding?

Yup, we'll likely want something like that so that the process launched by the deep link can notify the app's existing process of the deep link.

@loic-sharma, @chunhtai
Having to do some manual setup is fine, I had to do this for app_links anyways + extra code in the initialization file.
So, it would be nice to use go_router for all platforms instead of handling special cases separately. Plus, there would be no need for an extra library.