Add a kuberentes secret for Vault token in vault helm installation.

Signed-off-by: Revital Sur<EMAIL_ADDRESS>Co-authored-by: Roee Shlomo<EMAIL_ADDRESS>

Thanks. Please open an issue to improve this as nothing should login as root.

@roee88 I opened an issue. Maybe we should consider installing Vault in non dev mode? it seems that we assume we need the token in k8s secret which missed the point of using dev mode where the token is known...

I don't mind that the value is root and is known when you install it in dev mode or otherwise, it's not our problem and that's not what I meant.
What I meant is that this secret you added here is actually not for vault but rather for the manager to login to vault. A proper method to allow the manager to login to vault should be added instead, not relying on such a secret that always has root in it. Be it a secret dedicated to the manager that the user configures (as part of m4d helm install) or using Kubernetes login.

@roee88 OK thanks. I'll change the issue to reflect that.


Document supported hardware architectures

Hi!
Could you please put the information about supported architectures to the documentation please? E.e. about supported architectures for different operating systems, some specific requirements to the supported instructions, if you have any (e.g. maybe AVX is required - I do not know).
This kind of information is important for the end-users.
Thanks in advance!

@zamazan4ik I assume you reviewed https://zulip.readthedocs.io/en/latest/production/requirements.html and did not find the information you were looking for on that page?

@alya Yes, I have read this page. For me is unclear, which hardware architecture are supported/tested on the supported operating systems? I did not find any mention of x86-64, aarch64 and so on.

I started a discussion thread in the Zulip development community.

#22356 addresses the supported arches.  We're not aware of any instruction set being required -- if you find otherwise, please let us know!


logging callback

Hello. Is it possible to hook log messages to my own callback?

I found it in samples/dispatcher:
telegramError {
    println(error.getErrorMessage())
}


Update broken links on the readme

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Unknown confirmations height

Hi, I am developing app which is using Siad node. Everything was working as expected for about month, but today the application started failing when calling to /wallet/transactions with error error when calling /wallet/transactions: requesting transactions at unknown confirmation heights.
The call being sent to node is: /wallet/transactions?startheight=123783&endheight=123791
Wallet is unlocked
Consesus shows me that node current height is 123791.
I tried restarting the siad node, but it does not help.
What is the reason of this and how can it be fixed?

@khelle you might try asking for help on the Discord channel https://discord.gg/sia.  The community tends to be much more active there than on GitHub issues.  Please comment back on this issue if you find a solution to your problem.
recommendation to devs: tag as 'bug'

I'm going to go ahead and close this.  It hasn't had any action in 6 months and I believe the issue can be resolved by deleting the transactionpool.db file.  Steps to do that can be found here: https://gist.github.com/tbenz9/8066d74853ab28c23bd7bc1d3c7cdeda


fix(curriculum): Fixed typography in "Use body-parser to Parse POST Requests"

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] All the files I changed are in the same world language, for example: only English changes, or only Chinese changes, etc.

Closes #39269

The changes have been made in the challenge description of "Use body-parser to Parse POST Requests" to make it more informative.

Congrats @nakul251197 on your first pull request (PR)! 🎉
Thank you for your contribution to the page! 👍
We're happy to accept these changes, and look forward to future contributions. 📝

@nakul251197 Note that you should not be creating PRs from your master branch.  Instead, you should create a separate branch off of master, then make your changes.  This keeps your master always in sync with Free Code Camp's master.

@nakul251197 Note that you should not be creating PRs from your master branch. Instead, you should create a separate branch off of master, then make your changes. This keeps your master always in sync with Free Code Camp's master.

Thanks a lot!
I'll keep this in mind from the next time.


Created decorators for aliasd and deprecated functions and parameters

The decorator has two effects:

A note will be added to the docstring of the function below the title line to indicate the deprecated status.
When the function is called at the first time, a deprecation warning will be displayed.

The following functions or parameters are aliased or deprecated:
@deprecated:
partial_beta_diversity
local_pairwise_align_ssw
make_identity_substitution_matrix
TreeNode.deepcopy
TreeNode.unrooted_deepcopy
TreeNode.create_caches

@aliased
sobs <- observed_otus
multi_replace <- multiplicative_replacement
TreeNode.lca <- lowest_common_ancestor
remove_by_func <- remove_deleted
depth <- accumulate_to_ancestor
total_length <- descending_branch_length
maxdist <- get_max_distance
cophenet <- tip_tip_distances
compare_cophenet <- compare_tip_distances
clear_caches <- invalidate_caches

@params_aliased
ancom : p_adjust <- multiple_comparisons_correction
nj : clip_to_zero <- disallow_negative_branch_length
TreeNode.lca : nodes <- tipnames
TreeNode.shuffle : shuffler <- shuffle_f
TreeNode.total_length : nodes <- tip_subset
TreeNode.compare_subsets : shared_only <- exclude_absent_taxa
TreeNode.compare_cophenet : shuffler <- shuffle_f, metric <- dist_f

Please complete the following checklist:

[x] I have read the contribution guidelines.

[ ] I have documented all public-facing changes in the changelog.

[ ] This pull request includes code, documentation, or other content derived from external source(s). If this is the case, ensure the external source's license is compatible with scikit-bio's license. Include the license in the licenses directory and add a comment in the code giving proper attribution. Ensure any other requirements set forth by the license and/or author are satisfied.

It is your responsibility to disclose code, documentation, or other content derived from external source(s). If you have questions about whether something can be included in the project or how to give proper attribution, include those questions in your pull request and a reviewer will assist you.

[x] This pull request does not include code, documentation, or other content derived from external source(s).

Note: This document may also be helpful to see some of the things code reviewers will be verifying when reviewing your pull request.

@qiyunzhu Results from my functional tests are as follows:
All deprecation warnings for the @deprecated decorator work as intended.
All aliased functions decorated with @aliased work correctly. However, the following issues were found.

TreeNode.lowest_common_ancestor and TreeNode.compare_tip_distances both raise a deprecation warning for the create_caches function, not for themselves.
The following aliased TreeNode functions do not raise any warning at all, when they should raise deprecation warnings for themselves:  accumulate_to_ancestor, descending_branch_length, get_max_distance, and tip_tip_distances.

All aliased parameters decorated with @params_aliased work as intended.

@mataton Thank you for testing the code! I think that the methods you mentioned that didn't raise a warning by themselves, were because they are not deprecated, therefore no warning is needed. Instead, they have aliases that will live with them for a while. In the latest commit, I undeprecated create_caches, such that the first issue you described (I agree that this is confusing) won't happen.


Update aws-core, http-client-spi, ... to 2.17.211

Updates

software.amazon.awssdk:aws-core
software.amazon.awssdk:http-client-spi
software.amazon.awssdk:netty-nio-client

from 2.17.104 to 2.17.211.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "software.amazon.awssdk" } ]

labels: library-update, early-semver-patch, semver-spec-patch, commit-count:1

Superseded by #183.


Prevent the automated workflow from running in fork

Issue #, if available:
No issue available, but when forking this workflow we end up with a scheduled workflow that runs but w/o the token.
Description of changes:
Make the job conditional on whether or not it is executing from this repository.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Action run failed here, pointing at the string contents. It looks like the GH docs use single quotes (') instead of doubles ("),


update ffi >=2.0.0 <3.0.0

update ffi >=2.0.0 <3.0.0

Hmmm: is there a reason for this? This package is compatible with both v1.x and v2.x.

Closing for now. Feel free to file an issue if there's something that is not working that makes you think this is necessary.


missing file osmdroid-android-6.1.14.aar

version 0.86
need manual put file into path ft8CN\app\src\libs\osmdroid-android-6.1.14.aar

Thanks for this report. But we have confirmed no need to manually add this file.


1708 - Fixed branch tagging

:broken_heart: Build Failed

Buildkite Build
Commit: 6b03f42b5a11970b0c40bd12afce3a3afcdcb7ed

Failed CI Steps

Publish PR Docker Image

:green_heart: Build Succeeded

Buildkite Build
Commit: aae9887d3391f5c006a19efb8acf50e12baeccad

History

:broken_heart: Build #26 failed 6b03f42b5a11970b0c40bd12afce3a3afcdcb7ed


Replace double quotes with single quotes in Gruntfile.js

The .jshintrc has the following attribute set:"quotmark": "single"
Causing some warnings upon running grunt commands. There might be a reason for using some double qoutes that I'm not understanding, but if not this should be an easy fix by either removing the double quotes or altering the .jshintrc.
System (Ubuntu 14.04): nvm 0.22.1, npm 2.1.18, ruby 2.2.0, bundler 1.7.12
Example log:
Running "jshint:all" (jshint) task

Gruntfile.js
  line 54  col 24  Strings must use singlequote.
  line 55  col 21  Strings must use singlequote.
  line 56  col 34  Strings must use singlequote.
  line 65  col 42  Strings must use singlequote.
  line 82  col 24  Strings must use singlequote.
  line 83  col 21  Strings must use singlequote.
  line 84  col 34  Strings must use singlequote.

✖ 7 problems

Warning: Task "jshint:all" failed. Use --force to continue.

Aborted due to warnings.

thanks


update magic number constants for N3

Updating magic number constants for N3 to be aligned with https://github.com/neo-project/neo-node/releases/tag/v3.0.2

oh I totally forgot we had this (and just closed #774). Thanks!


Add Support for Merge Queues

We are happily using Merge-Gatekeeper in our github projects.
We are also looking into Github Merge Queues, which are GA since July.
I played a little bit and triggered merge-gatekeeper for github.event_group and handing down the github ref. It kinda looked promising but It had some side affects. It stopped working for PRs. ;-).
But since you do not officially support merge queues I will not report a bug. Instead I would like to ask for that support? Or guidence how to configure merge-gatekeeper for merge queues if it support it by default?

Hi @Fitzbert-Fitz, thanks for raising this! ☺
I was aware of the merge queue support, but didn't think it would have any impact to Merge Gatekeeper 😅 We haven't made use of it at UPSIDER, and while we may not have an immediate need for it, we would certainly want to support it if feasible. Let us have a closer look from our side to see what would be needed to support it -- if you have any ideas or suggestions, we would love to hear as well 🥰


Enhance the Style of the Badge Explorer Page #50

Fixes #50

@Ali-Sdg90 Can I see the resulting UI somewhere?

Sure, In the video you can see the resulting UI in action:
https://youtu.be/ZMwNCdbo2BM


Case auto-completion

It would be great to insert the branches of a case statement while typing. For example
Example.show(obj: JSON): String
  case obj {

should auto-fill the following:
Example.show(obj: JSON): String
  case obj {
    null:
    bool:
    number:
    string:
    array:
    object:
  }

registerCompletionItemProvider allows for completing these types of things.


redis: refactor

use fetchFromGitHub instead of downloading from redis.io
use fetchpatch2 instead of fetchurl and fetchpatch
use optional instead of optionals in several places
remove with lib;
enable tlsSupport by default
format with nixfmt-rfc-style
remove preBuild which does not have any motivation behind
add which and python3 to nativeBuildInputs

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Due to the number of rebuilds changed the target branch to staging, there seems to be a conflict

Due to the number of rebuilds changed the target branch to staging, there seems to be a conflict

fixed


kie-issues#965: Make sure kie-tools is using the latest timestamped SNAPSHOT from Kogito

Closes https://github.com/apache/incubator-kie-issues/issues/965
Still need to test it - Please don't merge.

Can you share the repository you're using as a base to apply the accelerator?

@thiagoelg https://github.com/yesamer/incubator-kie-sandbox-quarkus-accelerator/

@tiagobento Ok I tested it and it works.
I previously had that error because I didn't change the gitRepositoryGitRef to point to my branch kie-issues#965.
I only changed the gitRepositoryUrl.
With this setup, it works:


Some tests fail

OS: Guix
SBCL: 2.3.2
On commit 73822d9f480cbad00971b45ee80117297a67fb53:
;; Failures:
   3/  12 tests failed in ORG.SHIRAKUMO.DEPOT.TEST::DEPOT-PATHNAME
   1/   6 tests failed in ORG.SHIRAKUMO.DEPOT.TEST::WRITE-CONFLICT
The test form   (capture-error (with-open (transaction entry :output 'character) (finish (write-to transaction "testing")) (is string= "pre-test" (read-from entry 'character)) (finish (write-to entry "post-test"))))
evaluated to    ()
when            write-conflict
was expected to be equal under TYPEP.

   1/   6 tests failed in ORG.SHIRAKUMO.DEPOT.TEST::READ-INVALIDATED
The test form   (capture-error (with-open (transaction entry :input 'character) (finish (write-to entry "testing")) (is string= "testing" (read-from entry 'character)) (is string= "pre-test" (read-from transaction 'character))))
evaluated to    ()
when            read-invalidated
was expected to be equal under TYPEP.

   1/   5 tests failed in ORG.SHIRAKUMO.DEPOT.TEST::ATTRIBUTES
The test form   (finish (setf (attributes entry) ()))
exited by       [unsupported-operation] The operation
  (setf org.shirakumo.depot:attributes)
is not supported by the object
  #<file /home/ambrevar/common-lisp/depot/test/attributes>

Yeah, the conflict tests fail because it uses modification timestamps to check for them, which don't work if the test completes under a second. (See pathname.lisp:238)


create a self service appointment via the api

Is there a way to create an appointment for a client via the api, but still allow the user to manage it, in an ideal world the system would send the email as 'normal' but it would still be ok if we had to send the email - The question is how to get the manage link

I suggest using https://developer.microsoft.com/en-us/graph/support to submit questions and feature requests. Thanks.


Can you make a already compiled version of the engine?

What is your question?
I want a compiled version so I can just make a mod folder mod using ice engine, can you do that?

I did it but i think it's better compile the source code by yourself. IceEngine.zip

Check Latest Release.

seriously. just check the actions tab, go to the latest finished run, and scroll to the artifacts at the bottom of the page without your mouse being inside of the box in the middle of the screen. It doesnt scroll when positioned inside of it for some reason. Then click the artifact name of the os you're using, and tada, the absolute latest build.

^


Amend boot-setup docs to help with missing target

A fresh clone of om-cookbook repo is missing the target directory om-cookbook/recipes/boot-setup/target which makes boot dev fail. Instruct the user to take an extra preparatory step that establishes the directory in case it is missing.

Thx!


BREAKING CHANGE: fullscreen gallery thumbnail pagination

References DM-785
Motivation and context
So that the user can see a preview of what other pictures of the car/bike are available.
Before
no pagination thumbnails on the fullscreen gallery
After
pagination on the fullscreen gallery (except landscape mobile)
How to test
https://talamcol-pagination-carousel.components-pkg.branch.autoscout24.ch/?path=/story/components-data-display-carousel--full-screen

Just one think, I don't see the previous arrow.

thanks 🙏 that is intentional. If you are at the beginning, you don't see the previous arrow, if you are at the end you don't see the next arrow. https://autoricardo.atlassian.net/browse/DM-785?focusedCommentId=25210

Even on the third one?

Just one think, I don't see the previous arrow.

thanks 🙏 that is intentional. If you are at the beginning, you don't see the previous arrow, if you are at the end you don't see the next arrow. https://autoricardo.atlassian.net/browse/DM-785?focusedCommentId=25210

Even on the third one?

I see now what you mean 🤦 my bad. I fixed it. It toggles the visibility always after exactly one slide now


[SSoC 2.0] Generate Leap years in a Range of Years

Describe the project
A clear and concise description of the project you are going to add to this repository.

Front end project
Created using HTML, CSS and JS
This project has a attractive and responsive user interface
Generate number of leap years in between two entered years

Issue Type:

[x] New Project

Additional context
Add any other context or screenshots about the project here.

This project will tell the number of leap years and enlist those years
This project can also check the entered year is a leap year or not

Screenshots

If applicable, add screenshots to help explain your problem.

@TusharKesarwani @zugzwang03 Sirs, please assign this project to me to work on it and give a valuable contribution in this repository under SSoC 2.0 program

is this solved.?


haraka_require

In your code here, you should be able to use haraka_require instead. Example:
    haraka_require('smtp_client');

Hi @msimerson, I've replaced the include in latest commit https://github.com/sriccio/haraka-plugin-recipient-routes-probe/commit/9c6e2ba533eab130d885df2898741bf43ed8bbed
However for the tests (they are kinda useless yet, but I might add some relevant tests later) it obviously looks like I need to import at least haraka core for them to be usable.
Any idea how I could do this the best way ?
haraka-plugin-recipient-routes-probe> npx mocha

  recipient-routes-probe
    1) "before each" hook for "loads"

  0 passing (8ms)
  1 failing

  1) "before each" hook for "loads":
     recipient-routes-probe:3
const smtp_client_module = haraka_require("smtp_client.js");


[KNOWN ISSUES v0.1]

Here is a current list of known issues, and workarounds that I am aware of:
EDIT
New Step Button has been added. If you have issues with the GUI refreshing, click the refresh button and change the step you are working on.
WORST CASE SCENARIO: PLEASE CLOSE THE GUI, DELETE gui_settings.json, AND REOPEN THE GUI
Clip Guidance, Symmetry, cut_overview, etc all need to be in a schedule format [Ex]:
[3500]*250+[10000]*750
If you want to just use one value for the whole duration of rendering you can do any value * 1000 like this:
[5000]*1000
I'm sure there is plenty more. I hope you guys bear with me. Feel free to comment here or make a new issue if you feel it is deserving. I most likely won't be able to dedicate any serious time in the next few days but if an issue pops up hopefully I'll be able to squash it fairly quickly.

All known issues are fixed. CGS, Symmetry, etc can be input in integer or string format, new users shouldn't need or be able to screw it up.


Introduce Jarcho to rust-highfive as a new Clippy reviewer :tada:

Hey, @Jarcho, welcome to the reviewing rotation of Clippy. And thank you very much for the reviews you've already helped us with. :tada:
r? @flip1995 or @Manishearth
cc: https://github.com/rust-lang/team/pull/780

r? @rylev
welcome, Jarcho!

r? @Mark-Simulacrum


Button component :focus state

Bug Report

What is the issue?
The button component does not display the 4px border on :focus.

Done and merged: https://github.com/nhsuk/nhsuk-frontend/pull/407
Closing.


Gather recording boolean in a data structure

Problem
The transaction processing functions utilize three booleans to enable recordings that are passed down as arguments to many functions in chain. This clutters the code and hinders the easy usage of the SVM.
Summary of Changes
I crated a data structure to contain the three booleans.

Codecov Report
Attention: Patch coverage is 94.73684% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 81.8%. Comparing base (b6f6fdb) to head (ab94be9).
Report is 14 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #134   +/-   ##
=======================================
  Coverage    81.8%    81.8%
=======================================
  Files         838      838
  Lines      225947   225955    +8
=======================================
+ Hits       184923   184933   +10
+ Misses      41024    41022    -2


Add missing install_requires

Just a question: How does this help? Won't Django be a dependency of all projects anyway? Why does it have to be specified explicitly?

This package imports from Django, therefore Django is a requirement. It helps because then tools like pip can ensure you have a consistent environment.

That's right, but I'd assume that Django is installed anyway.
I think that upgrading just one package (e.g. pip install -U django-js-asset) also upgrades all its dependencies. Upgrading Django itself in this case isn't very helpful, especially if you're not using the newest Django release (major or minor, not patch release) everywhere. There are ways around that and you should be using wheels and rebuilding virtualenvs anyway, but still.
So I still don't see the benefit of adding Django to install_requires (except for purity) and I see some minor downsides. Which makes me want to reject this contribution even though it's a contribution from a first-time contributor to this repository.

By default pip doesn't upgrade dependencies. The option is --upgrade-strategy and the default is only-if-needed (ie. it only updates the dependencies if they are required by the upgraded package).

Alright, maybe this has changed in the meantime. I'm convinced. Thanks all!


Error installing on RPi

bytes/string conversion issue. seams to be Py3k compatibility issue.
Downloading/unpacking dump==0.0.3 (from -r /var/www/haproxy-wi/requirements.txt (line 7))
  Downloading dump-0.0.3.tar.gz
  Running setup.py (path:/tmp/pip-build-uasclc1x/dump/setup.py) egg_info for package dump
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip-build-uasclc1x/dump/setup.py", line 13, in <module>
        version = re.search("^__version__\s*=\s*[\'\"]([^\'\"]+)", f.read(), flags=re.I | re.M).group(1)
      File "/usr/lib/python3.4/re.py", line 166, in search
        return _compile(pattern, flags).search(string)
    TypeError: can't use a string pattern on a bytes-like object
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip-build-uasclc1x/dump/setup.py", line 13, in <module>

    version = re.search("^__version__\s*=\s*[\'\"]([^\'\"]+)", f.read(), flags=re.I | re.M).group(1)

  File "/usr/lib/python3.4/re.py", line 166, in search

    return _compile(pattern, flags).search(string)

TypeError: can't use a string pattern on a bytes-like object

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip-build-uasclc1x/dump
Storing debug log for failure in /root/.pip/pip.log
Unable to install Required Packages, Please check Pip error log and Fix the errors and Rerun the script

oops wrong, project, meant to post to lib project

OK )

unless of course you have a working workaround?

Yes, you first had this problem :(

I will test fully when I get home from work. it installed without via ssh though.

will test later when I am on the same network again.

Just installed it on RPi3b (dietpi up-to-date) and I did not encountered this issue.


Add login with github button

Adds authorization with Github on behalf of a user.

Login (first time)

When the user clicks login, the frontend redirects the user to install the github app
When they finish authorizing, github redirects back to the website, with a code as a query parameter
The frontend stores this code temporarily in localstorage, so that it's available to the original window
The frontend calls the server's login API with this code, and removes it from localstorage
The server calls github's access_token API to exchange the code for a user access token
The server then uses the token to request the user's information, stores the token in a session store, and responds to the client. Currently the session store is just using the server state. In a future PR, this will be moved to a session database table.
The client stores the session ID in localstorage.

Login (returning)

When the user clicks login, the frontend redirects the user to install the github app
The same redirection flow occurs, and the immediately closes without user input.
The same login flow occurs, with a new session ID.

Already logged in

The frontend checks if there is a session ID in localstorage. If so, it calls the server's session API
The server looks up the session and returns the User, if one is found.

Logout

The frontend clears the session ID from localstorage and sets the githubUser react state to null

This is still early in development. Future PRs will include:

Storing sessions in a SQL database table
Expiring sessions appropriately
Returning more information about the user
Potential improvements on the frontend to call the session API less often.
Tests! I will spend time writing tests once the implementation is more fleshed out.

I also started building out the search bar and the react routing for different pages.

Nice work, great to see this coming along.
I just pulled down and tried to run locally. The first time I clicked login with github, the pop-up window opened and I clicked Authorize. The pop-up window went away but did not seem to log my in. Each subsequent time I click on the window it does what is shown in the video below. Just wondering if I'm doing something wrong?
https://github.com/FuelLabs/forc.pub/assets/1289413/c53952cc-71de-468c-b03e-28c326c9c979

Just tested again with 3 browsers
Brave: doens't work -> same as video above
Chrome: works
Safari: works

Just tested again with 3 browsers
Brave: doens't work -> same as video above Chrome: works Safari: works

Good catch! The problem has to do with the login  request - the server never receives it.
If I change <IP_ADDRESS>  to localhost, it works in Brave. Maybe it’s some extra security setting in brave. I think this would only be an issue testing locally.


Set rparticipant variable before emit event

This PR sets the rparticipant variable to None in the data_received event. This addresses the error Local variable referenced before assignment. This may have negative effects in users callback functions if there isn't a None type check before referencing participant.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


No specific subroutine for the generic ‘nuopc_updatetimestamp’

On branch feature/schism and ubuntu/gcc/openmpi I get:
mpif90 -g -ffree-line-length-none -fno-range-check -fbacktrace  -I/home/lemmen/opt/mod/modg/Linux.gfortran.64.openmpi.ESMF_8_2_0_beta_snapshot_00 -I/home/lemmen/opt/include -I/usr/include -I/usr/include  -I/usr/include -IENS_Cpl -I.  -I/home/lemmen/devel/noaa/CoastalApp/NEMS/src/schism -I/home/lemmen/devel/noaa/CoastalApp/SCHISM/build/include -c module_MEDIATOR.tmp.f90
module_MEDIATOR.tmp.f90:3892:59:

 3892 |     call NUOPC_UpdateTimestamp(NState_AtmExp, clock, rc=rc)
      |                                                           1
Error: There is no specific subroutine for the generic ‘nuopc_updatetimestamp’ at (1)

@pvelissariou1

This is still a problem, occurring now on my ubuntu ESMF/8.2.0 system @pvelissariou1

OK, so the problem is that ESMF_VERSION_MAJOR is not known?  At least, it is not defined on the command line:
cpp -P -traditional -DENABLE_SMP -DCHNK_RRTM=8   -DFRONT_ATMESH=ATMESH     -DFRONT_WW3DATA=wav   -DFRONT_SCHISM=schism_nuopc_cap -D'SVN_INFO="(lemmen) Fr 10. Jun 11:16:03 CEST 2022 r581d8bea1ab6 https://github.com/noaa-ocs-modeling/NEMS.git"' -D'CMP_YEAR=2022' -D'CMP_JD=161' module_MEDIATOR.F90 > module_MEDIATOR.tmp.f90
mpif90 -g -ffree-line-length-none -fno-range-check -fbacktrace  -I/home/lemmen/opt/mod/modg/Linux.gfortran.64.openmpi.ESMF_8_2_0_beta_snapshot_00 -I/home/lemmen/opt/include -I/usr/include -I/usr/include  -I/usr/include -IENS_Cpl -I.  -I/home/lemmen/devel/noaa/CoastalApp/ATMESH -I/home/lemmen/devel/noaa/CoastalApp/ATMESH_INSTALL -I/home/lemmen/devel/noaa/CoastalApp/WW3DATA -I/home/lemmen/devel/noaa/CoastalApp/WW3DATA_INSTALL -I/home/lemmen/devel/noaa/CoastalApp/NEMS/src/schism -I/home/lemmen/devel/noaa/CoastalApp/SCHISM/build/include -c module_MEDIATOR.tmp.f90

It seems that NEMS' build system does not include the ESMF_CPPFLAGS, possibly not even about ESMF itself?

In GNUMakefile, the transfer of the version information to CPPFLAGS should be here:
# Loop over all defined variables, calling add_to_cppflags (see above)
# on each.  The $(and ...,) ensures the result is the empty string.
$(and $(foreach var,$(.VARIABLES),$(if $(call startswith,$(var),ESMF_VERSION),$(call add_to_cppflags,$(var)))),)

This can be worked around  by exporting ESMF_MAJOR_VERSION=8, but that is not a sustainable solution.  Somewhere along the way, the ESMF information is not transferred through the build system.

@pvelissariou1 , indeed the problem might be in the CoastalPaa rather than in NEMS. On our HPC gnu system levante and on my Desktop gnu system these errors appear.  Something related to the shell environment?  Here's the module files for my gnu system:
myDIRS="$(cd "$(dirname "$(realpath -s "${BASH_SOURCE[0]}" )" )" && pwd -P)"
myDIRS="${myDIRS} . ${APPMODS_DIR}"
####################

# Install via apt instsall libparmetis-dev
export METIS_DIR=/usr

####################
### (2) Set some environments varaiables related to the loaded
###     modules and required to compile the NEMS application properly.
funcs="$( find ${myDIRS} -type f -iname "PlatformFuncs" | head -n 1 )"
if [ -f "${funcs}" ]; then
  source "${funcs}"

  get_env_hdf5
  get_env_netcdf
fi
unset funcs myDIRS

# apt install libnetcdff-dev libnetcdf-dev libhdf5-openmpi-dev
export NETCDFHOME=/usr
export HDF5HOME=/usr
export METISHOME=/usr
export PARMETISHOME=/usr

export NETCDF_CONFIG=${NETCDFHOME:+${NETCDFHOME}/bin/nc-config}
export NETCDF_INCDIR=${NETCDFHOME:+${NETCDFHOME}/include}
export NETCDF_LIBDIR=${NETCDFHOME:+${NETCDFHOME}/lib}

export ESMFMKFILE=${ESMFMKFILE}

So ESMFMKFILE is defined but somehow gets lost on the way from CoastlApp to NEMS...

NEMS loads ESMF internally as long as the ESMFMKFILE is set by the OS.
Are all modules loading properly?
Panagiotis Velissariou, Ph.D., P.E.
UCAR Scientist
National Ocean and Atmospheric Administration
National Ocean Service
Office of Coast Survey CSDL/CMMB
Project Lead - Coastal Coupling
cell:<PHONE_NUMBER>
email: @.***
On Fri, Jun 10, 2022 at 9:53 AM Carsten Lemmen @.***>
wrote:

@pvelissariou1 https://github.com/pvelissariou1 , indeed the problem
might be in the CoastalPaa rather than in NEMS. On our HPC gnu system
levante and on my Desktop gnu system these errors appear. Something related
to the shell environment? Here's the module files for my gnu system:
myDIRS="$(cd "$(dirname "$(realpath -s "${BASH_SOURCE[0]}" )" )" && pwd -P)"
myDIRS="${myDIRS} . ${APPMODS_DIR}"
####################
Install via apt instsall libparmetis-dev
export METIS_DIR=/usr
####################
(2) Set some environments varaiables related to the loaded
modules and required to compile the NEMS application properly.
funcs="$( find ${myDIRS} -type f -iname "PlatformFuncs" | head -n 1 )"
if [ -f "${funcs}" ]; then
source "${funcs}"
get_env_hdf5
get_env_netcdf
fi
unset funcs myDIRS
apt install libnetcdff-dev libnetcdf-dev libhdf5-openmpi-dev
export NETCDFHOME=/usr
export HDF5HOME=/usr
export METISHOME=/usr
export PARMETISHOME=/usr
export NETCDF_CONFIG=${NETCDFHOME:+${NETCDFHOME}/bin/nc-config}
export NETCDF_INCDIR=${NETCDFHOME:+${NETCDFHOME}/include}
export NETCDF_LIBDIR=${NETCDFHOME:+${NETCDFHOME}/lib}
export ESMFMKFILE=${ESMFMKFILE}
So ESMFMKFILE is defined but somehow gets lost on the way from CoastlApp
to NEMS...
—
Reply to this email directly, view it on GitHub
https://github.com/noaa-ocs-modeling/CoastalApp/issues/89#issuecomment-1152448257,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APC7TP5VSOVQXK6NR7APO73VONJHHANCNFSM5JPHH2IA
.
You are receiving this because you were mentioned.Message ID:
@.***>

Modules are loaded and ESMFMKFILE is correctly defined on the shell.  So, NEMS does not load it despite it being defined.

Can we set a meeting to see how this can be fixed? Also the #102 issue.
Maybe today at 1:15 pm CT?
Panagiotis Velissariou, Ph.D., P.E.
UCAR Scientist
National Ocean and Atmospheric Administration
National Ocean Service
Office of Coast Survey CSDL/CMMB
Project Lead - Coastal Coupling
cell:<PHONE_NUMBER>
email: @.***
On Fri, Jun 10, 2022 at 10:03 AM Carsten Lemmen @.***>
wrote:

Modules are loaded and ESMFMKFILE is correctly defined on the shell. So,
NEMS does not load it despite it being defined.
—
Reply to this email directly, view it on GitHub
https://github.com/noaa-ocs-modeling/CoastalApp/issues/89#issuecomment-1152457116,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APC7TPYGZPPA4DKV5MD6INLVONKJZANCNFSM5JPHH2IA
.
You are receiving this because you were mentioned.Message ID:
@.***>

Sure, but we have the VIMS meeting at 1 pm if I am not mistaken.  I'd be available 12-1pm CT or after the VIMS meeting (from 2pm CT)

This problem occurs also on my Mac laptop with ESMF 8.4 installed:
module_MEDIATOR.tmp.f90:3881:59:

 3881 |     call NUOPC_UpdateTimestamp(NState_AtmExp, clock, rc=rc)
      |                                                           1
Error: There is no specific subroutine for the generic ‘nuopc_updatetimestamp’ at (1)
module_MEDIATOR.tmp.f90:3893:80:

 3893 |         call NUOPC_UpdateTimestamp(NState_AtmExp, clock_invalidTimeStamp, rc=rc)
      |                                                                                1
Error: There is no specific subroutine for the generic ‘nuopc_updatetimestamp’ at (1)
make[1]: *** [GNUmakefile:239: module_MEDIATOR.o] Fehler 1

Closing the issue. If @platipodium still reports problems we may reopen it.


capturing custom attributes added to code blocks?

Apologies if this was asked before, but could not find a solution by googling or searching the issues.
mdbook adds custom attributes to code blocks after lang specifiers e.g.,
```rust, ignore
fn() {}

When parsing such a file, pulldown returns the `,ignore' fragment as part of the code block content.
, ignore
fn() {}

Is there a way to obtain the attribute string (after ,) separately from the code block content?
Thanks,

Unfortunately, that extension is not part of CommonMark but exclusive of mdBook, so it is out of the scope of this project.
Such extensions can easily be thrown away using a post-processor, as mdBook does: (https://notriddle.github.io/pulldown-cmark/):
fn clean_codeblock_headers(event: Event<'_>) -> Event<'_> {
    match event {
        Event::Start(Tag::CodeBlock(CodeBlockKind::Fenced(ref info))) => {
            let info: String = info
                .chars()
                .map(|x| match x {
                    ' ' | '\t' => ',',
                    _ => x,
                })
                .filter(|ch| !ch.is_whitespace())
                .collect();

            Event::Start(Tag::CodeBlock(CodeBlockKind::Fenced(CowStr::from(info))))
        }
        _ => event,
    }
}

Right, the whole info line should be included.

When parsing such a file, pulldown returns the `,ignore' fragment as part of the code block content.

I'm unable to reproduce this. Could you provide a full example? Which version?


Flatgeobuf doc

For https://github.com/developmentseed/cloud-optimized-geospatial-formats-guide/issues/13

I think this should be good to merge now

I see this is merged 🎉  I made a few minor comments but none of them are significant enough to reopen this PR.

I made https://github.com/developmentseed/cloud-optimized-geospatial-formats-guide/pull/22 as a follow up for your review 🙂


Copy paste the content from MS Word,  extra spacing between words

Jodit Version:  3.4.xxxxx
Browser:  Chrome
OS:  Windows
Is React App:  True
Copy paste the content from MS Word to Jodit Editor showing extra spacing between  words
Expected behavior:

Actual behavior:

Can you attach a document here?


Modification for darkmode

landing page:

main page:

@huchenme Hi, so I had tried to write a version for dark mode, I will try to implement the dark/light mode toggle button in future, because I am still new to reactjs (like, real new, just started to learn a few days ago :C ) but I am very interested to contribute to this extension C:

Thanks, I already have a plan to add dark mode, stay tuned
On Mon, 14 Oct 2019 at 12:59 AM, pupubird<EMAIL_ADDRESS>wrote:

Closed #29 https://github.com/huchenme/hacker-tab-extension/pull/29.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/huchenme/hacker-tab-extension/pull/29?email_source=notifications&email_token=AAP3NNNRXW3VUVTTW2WRSCTQONHXRA5CNFSM4JADQ6C2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOUF2IXWA#event-2708769752,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAP3NNNJ44XCAN4VNW2KHFDQONHXRANCNFSM4JADQ6CQ
.


Integração com API reset password e correção de bugs #29 #59 #61

Issue
This pull request refers to Issue #59 , #29 , #61
Description
Ajuste da integração resetar senha, correções de bugs e adição de toast de confirmação nova senha.
Types of changes

Bug fix #28

What changed:

[ ] UI
[ ] Business rule
[ ] Documentation
[ ] Gradle or Build
[x] Navigation
[ ] Tests
[x] Resource files
[ ] Project architecture
[ ] Code style

Checklist

[x] Have tested the changes.
[x] Met all the acceptance requirements of this task.
[x] Verified if branch is up-to-date with develop (if not - rebase it or merge it).
[x] Resolve git conflicts

If any of the acceptance criteria are not met, please explain why below
What is the criteria? Why it is different?

Comentarios removidos


[Bug Report] subtitleViewConfiguration does not work in fullscreen

[windows platform]
subtitleViewConfiguration  does not work in full-screen mode.
...
          child: Video(
            controller: _controller,
            subtitleViewConfiguration: const SubtitleViewConfiguration(style: TextStyle(fontSize: 10)),
...

subtitle configuration changes in normal mode but does not change in fullscreen mode. even i can not hide that with  visible: false

It's actually a bug. This feature was added in later versions & I forgot to pass it here (probably few other too):
https://github.com/media-kit/media-kit/blob/12600d4da12f67615b412ff89766ad0b21d4d34b/media_kit_video/lib/media_kit_video_controls/src/controls/methods/fullscreen.dart#L41-L53
You can send us a PR if you wish to.

@mhkarami
A new update has been released. Please refer to the "Installation" section of the README. It should work correctly now.
Thanks!

The update has been really delayed. Apologies.


selling this code

Hello I found an YouTube video where he is selling this code for money
https://www.youtube.com/watch?v=C_px3Js_Ucw&lc=UgwnrOYReDeaiYGJ8L14AaABAg.9w_QYYCiByE9w_bJfZeMl1

Hello I found an YouTube video where he is selling this code for money
https://www.youtube.com/watch?v=C_px3Js_Ucw&lc=UgwnrOYReDeaiYGJ8L14AaABAg.9w_QYYCiByE9w_bJfZeMl1

谢谢你的提醒，我的账号在youtube无法评论了，期望有善良的开发者能够在评论中留下开源项目地址，并告诉大家这个项目的使用是免费开源的。
Thank you for reminding me that my account cannot be commented on youtube. I hope some kind developers can leave the open source project address in the comments and tell everyone that the use of this project is free and open source.

Hi I have leave a comment with the link to the repository and the demo site.

Hi I have leave a comment with the link to the repository and the demo site.

Seems like He hide my comment. lol

Hi I have leave a comment with the link to the repository and the demo site.

Seems like He hide my comment. lol

嗨，谢谢你的举措，我也不确定有什么办法能够让他不隐藏，信息不对称让他产生了利润，不过侧面这个项目还是有一点点价值的 哈哈。
Hi, thank you for your move, I'm not sure there is any way to make him not hide, information asymmetry makes him generate profits, but the side of this project is still a little bit of value haha.


missing three files with "setup" command

Describe the bug
Go to the doc repo top-level directory and run:
$ documentable setup

After the command finishes, the following files are NOT found:
Makefile
app-start
app.pl

To Reproduce
Clone and install the HEAD of the master branch of module Documentable.
Run the command as shown in the problem step.
Expected behavior
To find the reported missing files in the doc repo at the top-level.
Desktop (please complete the following information):

OS: Debian 10 (Buster)
Documentable version: 1.0.9
Perl 6 version: This is Rakudo Star version 2019.03.1 built on MoarVM version 2019.03
implementing Perl 6.d

Additional context
My clones of repos Documentable and doc are in separate directories.

I can't see how this would happen. That command downloads a tar file and unzips it. Here's the content

Maybe we should check that it's been uncompressed correctly. Also I don't know why assets changes name instead of being used directly...
I'll try and add a test for this anyway.

I see an "assets.db" file but it's an ASCII file with contents:
(will update shortly)

By the way I tried:
$ documentable clean

and got:
Cleaning up the directory...
rm: cannot remove 'app.pl': No such file or directory
The spawned command '    rm -rf html && rm -rf assets && rm -rf highlights \
    && rm app.pl && rm app-start && rm Makefile \
    && rm -rf template
' exited unsuccessfully (exit code: 1)
  in sub RUN-MAIN at /usr/local/rakudo.d/share/perl6/site/sources/8C56C68E804C20D14911B45EA26CF2450E29CFC8 (Documentable::CLI) line 28
  in block <unit> at /usr/local/rakudo.d/share/perl6/site/resources/1A2F2F2753AC23B9ED83E45593B48A7F719F159B line 3
  in sub MAIN at /usr/local/rakudo.d/share/perl6/site/bin/documentable line 3
  in block <unit> at /usr/local/rakudo.d/share/perl6/site/bin/documentable line 1

What's the name of the downloaded archive file?

Now it works!  But the file "assets.db" doesn't get removed with a "documentable clean."

Trying to see the docs served:
$ make run
Starting local server…
./app-start
Server available at http://<IP_ADDRESS>:3000
[2019-09-02 13:34:39.80146] [10680] [info] Copying assets...
[2019-09-02 13:34:39.80704] [10680] [info] ...done
[2019-09-02 13:34:56.42005] [10680] [debug] GET "/" (612fddc8)
[2019-09-02 13:34:56.42091] [10680] [debug] Routing to a callback
[2019-09-02 13:34:56.42119] [10680] [debug] Static file "/index.html" not found
[2019-09-02 13:34:56.42225] [10680] [debug] Template "not_found.development.html.ep" not found
[2019-09-02 13:34:56.42287] [10680] [debug] Template "not_found.html.ep" not found
[2019-09-02 13:34:56.42307] [10680] [debug] Rendering template "mojo/debug.html.ep"
[2019-09-02 13:34:56.43784] [10680] [debug] Your secret passphrase needs to be changed
[2019-09-02 13:34:56.43920] [10680] [debug] 404 Not Found (0.019168s, 52.170/s)

I will take a look at:

[ ] Mojolicious passphare, ... => Though I have to say those are not important warnings, we are using the development server after all. Right now html files are served using nginx so it's not a problem.
[ ] Fails in 'documentable clean' and assets.db not getting removed. (assets.db is an intermediate file used by sass command I think, I am not sure)

Mojolicious is part of perl6/doc, not this repo. And it will eventually be
replaced.

These documentation creation problems are why I have been re-writing the system for producing documentation. It is called Collection-raku-documentation.
It should work out of the box by calling RakuDoc in an empty  directory.  There is a need to create a clone of this repository (but you have already done that) and to point the config file there.
I wrote the Renderer (raku-pod-render) based on the BigFile code. However BigFile died from bit rot some time ago, unless it has been revived.
Since all of the rendering of the podfiles are in templates, it should be relatively straightforward (though tedious) to create the Mode (see collection) for a single documentation file and to generate a pdf or epub file directly. I have simply not had the time yet to do this.


Fix broken e2e link spec

Got broken by #2058 which changed the error output format

@SpComb PTAL, helps me captaining as I'm running e2e spec for almost all PRs I'm merging and now this gives a false negative :)


⚠ Bofrost.be has degraded performance

In 72499fd, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1497 ms

Resolved: Bofrost.be performance has improved in 54c8a97 after 6 minutes.


Terraclimate recipe

TerraClimate recipe and yaml metadata file.
Could use some feedback on the metadata! I filled unknown fields with "?" in meta.yml

Welcome, @norlandrhagen! On first glance your recipe.py looks great. If any snafu's arise in the execution phase, I'll be happy to advise.
Your uncertainty on the bakery section of meta.yaml is understandable, that aspect of Pangeo Forge is the newest and there are fewer examples to work from. I will defer to @sharkinsspatial for his recommendations there, as I am also not 100% clear on current best practice.

@cisaacstern - what's the current best practice for testing recipe execution?

We currently do not have any true (i.e. pytest, etc) test suite for recipes. (Though that may be worth developing down the line.) For now, these four simple steps can be executed sequentially in a notebook:
https://pangeo-forge.readthedocs.io/en/latest/execution.html#manual-execution
For testing purposes, it probably makes sense to run these steps on a "pruned" version of the recipe:
!pip install git+https://github.com/pangeo-forge/pangeo-forge-recipes.git

from recipe import recipe
recipe = recipe.copy_pruned()  # preserves first two input files only, by default

Then before manual execution, set the recipe's input_cache, metadata_cache, and target attributes:
from fsspec.implementations.local import LocalFileSystem
from pangeo_forge_recipes.storage import MetadataTarget, CacheFSSpecTarget

fs_local = LocalFileSystem()

recipe.input_cache = CacheFSSpecTarget(fs_local, "<filepath_for_input_cache>")
recipe.metadata_cache = MetadataTarget(fs_local, "<filepath_for_metadata>")
recipe.target = MetadataTarget(fs_local, "<filepath_for_zarr_store>")

Note that the filesystems for any or all of these attributes don't have to be LocalFileSystem instances. They can be any fsspec-compatible filesystem to which you have write access, such as s3fs, gcsfs, etc.

Finally, for some visibility into what's happening, it's worth setting up logging:
def setup_logging():
    import logging
    import sys
    formatter = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
    logger = logging.getLogger("pangeo_forge_recipes")
    logger.setLevel(logging.DEBUG)
    sh = logging.StreamHandler(stream=sys.stdout)
    sh.setFormatter(formatter)
    logger.addHandler(sh)
setup_logging()

Let me know if any further clarification would be useful, or if you run into any issues. We can always troubleshoot here or jump on whereby.com/pangeo.

@norlandrhagen Thank you for the PR 🎊  and raising the issue of lacking guidance for recipe configuration.  We are working to improve the project's overall documentation so it is great to identify areas where the docs need clarification. You can find more details about meta.yaml configuration in this ADR doc.  At the moment, the only publicly configured bakery is our development and testing cluster which you can use for initial recipe testing and validation (hopefully we will have a permanent public bakery available for you to reference soon to run and store the entire recipe output).  As a reference you can use meta.yaml settings in this PR as a reference.
Note that the worker nodes in this bakery currently only support pangeo-notebook release 2021.06.05 so you'll need to update your meta.yaml accordingly or the CI process will report a version mismatch exception (we are looking for feedback on the best way to communicate these compatible versions to recipe developers since bakery clusters may not always be operating the most recent version).
@jhamman On the testing front, once a PR has been reviewed an administrator from staged-recipes can issue a slash-command comment in the PR which triggers the run-recipe-test workflow.  This workflow will

Register a pruned version of the recipe flow with the target bakery.
Submit a flow run of the pruned recipe to run on the bakery as a full integration test.
Update the comment with flow's run status after it succeeds or fails.

I'm unsure what the final guidance around testing should be but I think we should consider a 2 stage process where developers use the approach @cisaacstern described for "local" testing and then repo administrators later run these integration tests to identify any potential issues with how the recipe interacts with its target cloud infrastructure.

Thanks for the feedback @cisaacstern and @sharkinsspatial.
-I updated the pango-notebook version to 2021.06.05 in the metadata yml file.
-I ran the recipe locally successfully. I did have two questions:

The recipe.copy_prunded() selected the first two time dimensions, but did not trim the # of variables. Just wanted to check that this is the expected behavior.
XarrayZarrRecipe(file_pattern=<FilePattern {'time': 2, 'variable': 14}>, inputs_per_chunk=1, target_chunks={'lat': 1024, 'lon': 1024, 'time': 12}....)

Just wanted to check in about this warning and see if there are any subsetting inputs I should modify.  pangeo_forge_recipes.recipes.xarray_zarr - WARNING - Variable swe of<PHONE_NUMBER> bytes is 7.17 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with subset_inputs = {"time": 8}. If len(ds["time"])< 8, substitute "time" for any name in ds["swe"].dims with length >= 8 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
Thanks!

recipe.copy_pruned() selected the first two time dimensions, but did not trim the # of variables. Just wanted to check that this is the expected behavior.

This is expected; copy_pruned() only subsets along the ConcatDim.

Just wanted to check in about this warning and see if there are any subsetting inputs I should modify.

Great to see that warning doing it's job! (cc @rabernat 😄 ) ... yes, you should follow the warning's recommendation and add subset_inputs={"time": 8} to your XarrayZarrRecipe instantiation here. This will allow your ~3.5 GB variable array to be loaded into memory in ~500 MB subsets, so that the recipe doesn't exceed the bakery workers' memory limits.

@norlandrhagen If you can update your meta.yaml to use the same cluster settings as here I can trigger a CI test of your recipe.

Sounds great, just updated the meta.yaml

/run-recipe-test

@cisaacstern We are temporarily blocked here as our CI actions are running with pangeo-forge-recipes v0.4.0 and subset_inputs is not yet available in a tagged release.  See the error reported here.  We'll need to

Cut a new release of pangeo-forge-recipes.
Publish a new tag in pangeo-forge-bakery-images.
Update the CI action to use this updated image.
@norlandrhagen Will need to update the versions in the meta.yaml accordingly.

I'll create another issue for the release in pangeo-forge-recipes referencing this comment.

@norlandrhagen If you can also update your pangeo_notebook_version to 2021.07.17 we should then have the subset_inputs parameter available and I can re-run the CI tests 👍

/run-recipe-test

@cisaacstern Even with the current usage of subset_inputs we are seeing workers being killed for exceeding container memory limits when combining chunks.
[2021-08-12 14:41:56+0000] INFO - pangeo_forge_recipes.recipes.xarray_zarr \| Subsetting input according to time-7
--
INFO:pangeo_forge_recipes.recipes.xarray_zarr:Subsetting input according to time-7
DEBUG:pangeo_forge_recipes.recipes.xarray_zarr:Subsetting dataset with indexer {'time': slice(7, 12, None)}
[2021-08-12 14:41:56+0000] DEBUG - pangeo_forge_recipes.recipes.xarray_zarr \| Subsetting dataset with indexer {'time': slice(7, 12, None)}
[2021-08-12 14:41:56+0000] INFO - pangeo_forge_recipes.recipes.xarray_zarr \| Combining inputs for chunk 'time-1,variable-4,time-7'
INFO:pangeo_forge_recipes.recipes.xarray_zarr:Combining inputs for chunk 'time-1,variable-4,time-7'
DEBUG:pangeo_forge_recipes.recipes.xarray_zarr:<xarray.Dataset>
Dimensions:  (lat: 4320, lon: 8640, time: 5, crs: 1)
Coordinates:
* lat      (lat) float64 89.98 89.94 89.9 89.85 ... -89.85 -89.9 -89.94 -89.98
* lon      (lon) float64 -180.0 -179.9 -179.9 -179.9 ... 179.9 179.9 180.0
* time     (time) datetime64[ns] 1959-08-01 1959-09-01 ... 1959-12-01
* crs      (crs) int16 3
Data variables:
q        (time, lat, lon) float64 dask.array<chunksize=(5, 4320, 8640), meta=np.ndarray>
Attributes: (12/49)
method:                          These layers from TerraClimate were deri...
title:                           TerraClimate: monthly climate and climat...
summary:                         This archive contains a dataset of high-...
keywords:                        WORLDCLIM,global,monthly, temperature,pr...
id:                              Blank
naming_authority:                edu.uidaho.nkn
...                              ...
geospatial_vertical_resolution:  0.0
geospatial_vertical_positive:    Up
references:                      Abatzoglou, J.T., S.Z. Dobrowski, S.A. P...
source:                          WorldClim v2.0 (2.5m), CRU Ts4.0, JRA-55
version:                         v1.0
Conventions:                     CF-1.6
[2021-08-12 14:41:56+0000] DEBUG - pangeo_forge_recipes.recipes.xarray_zarr \| <xarray.Dataset>
Dimensions:  (lat: 4320, lon: 8640, time: 5, crs: 1)
Coordinates:
* lat      (lat) float64 89.98 89.94 89.9 89.85 ... -89.85 -89.9 -89.94 -89.98
* lon      (lon) float64 -180.0 -179.9 -179.9 -179.9 ... 179.9 179.9 180.0
* time     (time) datetime64[ns] 1959-08-01 1959-09-01 ... 1959-12-01
* crs      (crs) int16 3
Data variables:
q        (time, lat, lon) float64 dask.array<chunksize=(5, 4320, 8640), meta=np.ndarray>
Attributes: (12/49)
method:                          These layers from TerraClimate were deri...
title:                           TerraClimate: monthly climate and climat...
summary:                         This archive contains a dataset of high-...
keywords:                        WORLDCLIM,global,monthly, temperature,pr...
id:                              Blank
naming_authority:                edu.uidaho.nkn
...                              ...
geospatial_vertical_resolution:  0.0
geospatial_vertical_positive:    Up
references:                      Abatzoglou, J.T., S.Z. Dobrowski, S.A. P...
source:                          WorldClim v2.0 (2.5m), CRU Ts4.0, JRA-55
version:                         v1.0
Conventions:                     CF-1.6
DEBUG:pangeo_forge_recipes.recipes.xarray_zarr:Converting variable time of 40 bytes to `numpy.ndarray`
[2021-08-12 14:41:56+0000] DEBUG - pangeo_forge_recipes.recipes.xarray_zarr \| Converting variable time of 40 bytes to `numpy.ndarray`
DEBUG:pangeo_forge_recipes.recipes.xarray_zarr:Acquiring locks ['time-1']
[2021-08-12 14:41:56+0000] DEBUG - pangeo_forge_recipes.recipes.xarray_zarr \| Acquiring locks ['time-1']
[2021-08-12 14:41:56+0000] DEBUG - pangeo_forge_recipes.utils \| Acquiring lock pangeo-forge-time-1...
DEBUG:pangeo_forge_recipes.utils:Acquiring lock pangeo-forge-time-1...
[2021-08-12 14:41:56+0000] DEBUG - pangeo_forge_recipes.utils \| Acquired lock pangeo-forge-time-1
DEBUG:pangeo_forge_recipes.utils:Acquired lock pangeo-forge-time-1
[2021-08-12 14:41:56+0000] INFO - pangeo_forge_recipes.recipes.xarray_zarr \| Storing variable time chunk time-1,variable-4,time-7 to Zarr region (slice(19, 24, None),)
INFO:pangeo_forge_recipes.recipes.xarray_zarr:Storing variable time chunk time-1,variable-4,time-7 to Zarr region (slice(19, 24, None),)
[2021-08-12 14:41:56+0000] DEBUG - pangeo_forge_recipes.utils \| Released lock pangeo-forge-time-1
DEBUG:pangeo_forge_recipes.utils:Released lock pangeo-forge-time-1
[2021-08-12 14:41:56+0000] DEBUG - pangeo_forge_recipes.recipes.xarray_zarr \| Converting variable q of<PHONE_NUMBER> bytes to `numpy.ndarray`
DEBUG:pangeo_forge_recipes.recipes.xarray_zarr:Converting variable q of<PHONE_NUMBER> bytes to `numpy.ndarray`
[2021-08-12 14:41:56+0000] WARNING - pangeo_forge_recipes.recipes.xarray_zarr \| Variable q of<PHONE_NUMBER> bytes is 2.99 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {     "time": 3 }`. If `len(ds["time"])` < 3, substitute "time" for any name in ds["q"].dims with length >= 3 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
WARNING:pangeo_forge_recipes.recipes.xarray_zarr:Variable q of<PHONE_NUMBER> bytes is 2.99 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {     "time": 3 }`. If `len(ds["time"])` < 3, substitute "time" for any name in ds["q"].dims with length >= 3 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.CopyWARNING:pangeo_forge_recipes.recipes.xarray_zarr:Variable q of<PHONE_NUMBER> bytes is 2.99 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {     "time": 3 }`. If `len(ds["time"])` < 3, substitute "time" for any name in ds["q"].dims with length >= 3 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
distributed.worker - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker.html#memtrim for more information. -- Unmanaged memory: 2.71 GiB -- Worker memory limit: 3.73 GiB
distributed.worker - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker.html#memtrim for more information. -- Unmanaged memory: 2.61 GiB -- Worker memory limit: 3.73 GiB
distributed.worker - WARNING - Worker is at 82% memory usage. Pausing worker.  Process memory: 3.07 GiB -- Worker memory limit: 3.73 GiB
distributed.worker - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker.html#memtrim for more information. -- Unmanaged memory: 3.07 GiB -- Worker memory limit: 3.73 GiB
distributed.worker - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker.html#memtrim for more information. -- Unmanaged memory: 3.19 GiB -- Worker memory limit: 3.73 GiB
distributed.worker - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker.html#memtrim for more information. -- Unmanaged memory: 3.34 GiB -- Worker memory limit: 3.73 GiB
distributed.worker - WARNING - Unmanaged memory use is high. This may indicate a memory leak or the memory may not be released to the OS; see https://distributed.dask.org/en/latest/worker.html#memtrim for more information. -- Unmanaged memory: 3.52 GiB -- Worker memory limit: 3.73 GiB
distributed.nanny - WARNING - Worker exceeded 95% memory budget. Restarting
distributed.nanny - INFO - Worker process 312 was killed by signal 15

A few questions here.

Currently our worker containers do not have any custom environment settings applied so MAX_MEMORY is using the default 500000000 bytes.  Should I be setting PANGEO_FORGE_MAX_MEMORY to the worker container's memory limit at registration time?  This would affect the recommended subset_inputs but given that we are using the default our 3.73 GB container limit should be sufficient with the {"time": 8} setting, correct?
Should we be bumping the worker memory limits up in the meta.yaml?
As illustrated in the logging, this memory growth is reported by Dask as Unmanaged.  Previously we were automatically trimming allocation with MALLOC_TRIM_THRESHOLD_  but I had disabled this on the containers in order to see how recent Dask updates might improve our previous allocation issues.  Should we re-enable automated trimming on containers?
cc @rabernat @TomAugspurger

@sharkinsspatial, a limitation of our MAX_MEMORY warning is that it makes a subsetting recommendation based on the first oversize variable array it encounters. So based on the warning reported in https://github.com/pangeo-forge/staged-recipes/pull/68#issuecomment-896164710, we specified subsetting based on the size of variable swe:
Variable swe of<PHONE_NUMBER> bytes ... Consider re-instantiating recipe with subset_inputs = {"time": 8}

But from the worker logs above, it appears that even with this subsetting, variable q subsets are still > 500 MB:
Variable q of<PHONE_NUMBER> bytes is 2.99 times larger ...

If q is still ~3 times larger than 500 MB with subset_chunks = {"time": 8}, then I'd recommend we try re-assigning subset_chunks = {"time": 24} to bring each subset of q under the MAX_MEMORY threshold. This will make subsets of swe down to approx. 160 MB per subset, which doesn't seem problematically small.

/run-recipe-test

@cisaacstern We also seem to be experiencing intermittent fsspec hanging issues when opening from cache
[2021-08-12 17:50:17+0000] INFO - pangeo_forge_recipes.recipes.xarray_zarr \| Caching input 'time-0,variable-13'
--
INFO:pangeo_forge_recipes.recipes.xarray_zarr:Caching input 'time-0,variable-13'
[2021-08-12 17:50:17+0000] INFO - pangeo_forge_recipes.storage \| Caching file 'http://thredds.northwestknowledge.net:8080/thredds/fileServer/TERRACLIMATE_ALL/data/TerraClimate_PDSI_1958.nc'
INFO:pangeo_forge_recipes.storage:Caching file 'http://thredds.northwestknowledge.net:8080/thredds/fileServer/TERRACLIMATE_ALL/data/TerraClimate_PDSI_1958.nc'
[2021-08-12 17:50:17+0000] INFO - pangeo_forge_recipes.storage \| Coping remote file 'http://thredds.northwestknowledge.net:8080/thredds/fileServer/TERRACLIMATE_ALL/data/TerraClimate_PDSI_1958.nc' to cache
INFO:pangeo_forge_recipes.storage:Coping remote file 'http://thredds.northwestknowledge.net:8080/thredds/fileServer/TERRACLIMATE_ALL/data/TerraClimate_PDSI_1958.nc' to cache
[2021-08-12 17:50:17+0000] DEBUG - pangeo_forge_recipes.storage \| entering fs.open context manager for s3://pangeo-forge-aws-bakery-flowcachebucketdasktest4-10neo67y7a924/pangeo-forge/staged-recipes/terraclimate/cache/abf9d7327b04f292bf67da36762dd6fa-http_thredds.northwestknowledge.net_8080_thredds_fileserver_terraclimate_all_data_terraclimate_pdsi_1958.nc
DEBUG:pangeo_forge_recipes.storage:entering fs.open context manager for s3://pangeo-forge-aws-bakery-flowcachebucketdasktest4-10neo67y7a924/pangeo-forge/staged-recipes/terraclimate/cache/abf9d7327b04f292bf67da36762dd6fa-http_thredds.northwestknowledge.net_8080_thredds_fileserver_terraclimate_all_data_terraclimate_pdsi_1958.nc
[2021-08-12 17:50:17+0000] DEBUG - pangeo_forge_recipes.storage \| FSSpecTarget.open yielding <File-like object S3FileSystem, pangeo-forge-aws-bakery-flowcachebucketdasktest4-10neo67y7a924/pangeo-forge/staged-recipes/terraclimate/cache/abf9d7327b04f292bf67da36762dd6fa-http_thredds.northwestknowledge.net_8080_thredds_fileserver_terraclimate_all_data_terraclimate_pdsi_1958.nc>
DEBUG:pangeo_forge_recipes.storage:FSSpecTarget.open yielding <File-like object S3FileSystem, pangeo-forge-aws-bakery-flowcachebucketdasktest4-10neo67y7a924/pangeo-forge/staged-recipes/terraclimate/cache/abf9d7327b04f292bf67da36762dd6fa-http_thredds.northwestknowledge.net_8080_thredds_fileserver_terraclimate_all_data_terraclimate_pdsi_1958.nc>
[2021-08-12 17:50:18+0000] DEBUG - pangeo_forge_recipes.storage \| _copy_btw_filesystems total bytes copied: 10000000
DEBUG:pangeo_forge_recipes.storage:_copy_btw_filesystems total bytes copied: 10000000
[2021-08-12 17:50:18+0000] DEBUG - pangeo_forge_recipes.storage \| avg throughput over 0.02 min: 9.75 MB/sec
DEBUG:pangeo_forge_recipes.storage:avg throughput over 0.02 min: 9.75 MB/sec
[2021-08-12 17:50:22+0000] DEBUG - pangeo_forge_recipes.storage \| _copy_btw_filesystems total bytes copied: 70000000
[2021-08-12 17:50:22+0000] DEBUG - pangeo_forge_recipes.storage \| avg throughput over 0.10 min: 12.17 MB/sec
DEBUG:pangeo_forge_recipes.storage:_copy_btw_filesystems total bytes copied: 70000000
DEBUG:pangeo_forge_recipes.storage:avg throughput over 0.10 min: 12.17 MB/sec
[2021-08-12 17:50:27+0000] DEBUG - pangeo_forge_recipes.storage \| _copy_btw_filesystems total bytes copied: 130000000
DEBUG:pangeo_forge_recipes.storage:_copy_btw_filesystems total bytes copied: 130000000
[2021-08-12 17:50:27+0000] DEBUG - pangeo_forge_recipes.storage \| avg throughput over 0.17 min: 12.85 MB/sec
DEBUG:pangeo_forge_recipes.storage:avg throughput over 0.17 min: 12.85 MB/sec
[2021-08-12 17:50:27+0000] DEBUG - pangeo_forge_recipes.storage \| FSSpecTarget.open yielded
DEBUG:pangeo_forge_recipes.storage:FSSpecTarget.open yielded
[2021-08-12 17:50:27+0000] DEBUG - pangeo_forge_recipes.storage \| _copy_btw_filesystems done
DEBUG:pangeo_forge_recipes.storage:_copy_btw_filesystems done
[2021-08-12 17:50:27+0000] INFO - pangeo_forge_recipes.recipes.xarray_zarr \| Caching metadata for input 'time-0,variable-13'
INFO:pangeo_forge_recipes.recipes.xarray_zarr:Caching metadata for input 'time-0,variable-13'
[2021-08-12 17:50:27+0000] INFO - pangeo_forge_recipes.recipes.xarray_zarr \| Opening input with Xarray time-0,variable-13: 'http://thredds.northwestknowledge.net:8080/thredds/fileServer/TERRACLIMATE_ALL/data/TerraClimate_PDSI_1958.nc'
INFO:pangeo_forge_recipes.recipes.xarray_zarr:Opening input with Xarray time-0,variable-13: 'http://thredds.northwestknowledge.net:8080/thredds/fileServer/TERRACLIMATE_ALL/data/TerraClimate_PDSI_1958.nc'
[2021-08-12 17:50:27+0000] INFO - pangeo_forge_recipes.storage \| Opening 'http://thredds.northwestknowledge.net:8080/thredds/fileServer/TERRACLIMATE_ALL/data/TerraClimate_PDSI_1958.nc' from cache
INFO:pangeo_forge_recipes.storage:Opening 'http://thredds.northwestknowledge.net:8080/thredds/fileServer/TERRACLIMATE_ALL/data/TerraClimate_PDSI_1958.nc' from cache
[2021-08-12 17:50:27+0000] DEBUG - pangeo_forge_recipes.storage \| file_opener entering first context for <contextlib._GeneratorContextManager object at 0x7fe8540c0dc0>
DEBUG:pangeo_forge_recipes.storage:file_opener entering first context for <contextlib._GeneratorContextManager object at 0x7fe8540c0dc0>
[2021-08-12 17:50:27+0000] DEBUG - pangeo_forge_recipes.storage \| entering fs.open context manager for s3://pangeo-forge-aws-bakery-flowcachebucketdasktest4-10neo67y7a924/pangeo-forge/staged-recipes/terraclimate/cache/abf9d7327b04f292bf67da36762dd6fa-http_thredds.northwestknowledge.net_8080_thredds_fileserver_terraclimate_all_data_terraclimate_pdsi_1958.nc
DEBUG:pangeo_forge_recipes.storage:entering fs.open context manager for s3://pangeo-forge-aws-bakery-flowcachebucketdasktest4-10neo67y7a924/pangeo-forge/staged-recipes/terraclimate/cache/abf9d7327b04f292bf67da36762dd6fa-http_thredds.northwestknowledge.net_8080_thredds_fileserver_terraclimate_all_data_terraclimate_pdsi_1958.nc
[2021-08-12 17:50:27+0000] DEBUG - pangeo_forge_recipes.storage \| FSSpecTarget.open yielding <File-like object S3FileSystem, pangeo-forge-aws-bakery-flowcachebucketdasktest4-10neo67y7a924/pangeo-forge/staged-recipes/terraclimate/cache/abf9d7327b04f292bf67da36762dd6fa-http_thredds.northwestknowledge.net_8080_thredds_fileserver_terraclimate_all_data_terraclimate_pdsi_1958.nc>
DEBUG:pangeo_forge_recipes.storage:FSSpecTarget.open yielding <File-like object S3FileSystem, pangeo-forge-aws-bakery-flowcachebucketdasktest4-10neo67y7a924/pangeo-forge/staged-recipes/terraclimate/cache/abf9d7327b04f292bf67da36762dd6fa-http_thredds.northwestknowledge.net_8080_thredds_fileserver_terraclimate_all_data_terraclimate_pdsi_1958.nc>
[2021-08-12 17:50:27+0000] DEBUG - pangeo_forge_recipes.storage \| file_opener entering second context for <File-like object S3FileSystem, pangeo-forge-aws-bakery-flowcachebucketdasktest4-10neo67y7a924/pangeo-forge/staged-recipes/terraclimate/cache/abf9d7327b04f292bf67da36762dd6fa-http_thredds.northwestknowledge.net_8080_thredds_fileserver_terraclimate_all_data_terraclimate_pdsi_1958.nc>
DEBUG:pangeo_forge_recipes.storage:file_opener entering second context for <File-like object S3FileSystem, pangeo-forge-aws-bakery-flowcachebucketdasktest4-10neo67y7a924/pangeo-forge/staged-recipes/terraclimate/cache/abf9d7327b04f292bf67da36762dd6fa-http_thredds.northwestknowledge.net_8080_thredds_fileserver_terraclimate_all_data_terraclimate_pdsi_1958.nc>
[2021-08-12 17:50:27+0000] DEBUG - pangeo_forge_recipes.recipes.xarray_zarr \| about to enter xr.open_dataset context on <File-like object S3FileSystem, pangeo-forge-aws-bakery-flowcachebucketdasktest4-10neo67y7a924/pangeo-forge/staged-recipes/terraclimate/cache/abf9d7327b04f292bf67da36762dd6fa-http_thredds.northwestknowledge.net_8080_thredds_fileserver_terraclimate_all_data_terraclimate_pdsi_1958.nc>
DEBUG:pangeo_forge_recipes.recipes.xarray_zarr:about to enter xr.open_dataset context on <File-like object S3FileSystem, pangeo-forge-aws-bakery-flowcachebucketdasktest4-10neo67y7a924/pangeo-forge/staged-recipes/terraclimate/cache/abf9d7327b04f292bf67da36762dd6fa-http_thredds.northwestknowledge.net_8080_thredds_fileserver_terraclimate_all_data_terraclimate_pdsi_1958.nc>

Which results in a hung worker.  This has occurred in 3 of my 5 test runs for this data (each time with the cache buckets emptied prior).  refs https://github.com/dask/dask/issues/7547
cc @martindurant.
With
dask-2021.7.0
distributed-2021.7.0
fsspec-2021.7.0
s3fs-2021.7.0
h5py-3.3.0

Again, I invite you to try with https://github.com/intake/filesystem_spec/pull/694 . Other reports say that it is garbage collection of the h5py object while in the wrong thread which is causing this kind of problem.

@sharkinsspatial, if intake/filesystem_spec#694 doesn't resolve this, we can try setting copy_input_to_local_file=True, as suggested by @rabernat in https://github.com/pangeo-forge/pangeo-forge-recipes/issues/177#issuecomment-892425152 (for what is presumably a related issue).

copy_input_to_local_file=True is going to be pretty inefficient for large values of subset_inputs. The reason is that isn't possible to do a partial copy of a netcdf file. If we are subsetting a 10 GB file in 20 500 MB chunks, the full 10 GB file will still be copied locally for each chunk with copy_input_to_local_file=True.
Why are q and swe such different sizes? Do they have different dimensionality?

Another possibility would be to try to do this recipe over OPeNDAP, which would bypass the caching step completely.

If we are subsetting a 10 GB file in 20 500 MB chunks, the full 10 GB file will still be copied locally for each chunk

I can think of a way to access only chunks from within a netCDF dataset without copying the whole thing! Also would avoid opening with h5py, which is where all the problems are coming from. Ready to have reference-maker as the precursor to such recipes? :)

Martin, I agree that being able to bypass h5py and read the netcdf file from remote storage in a different way would be a useful feature. Let's discuss that idea in a new issue.

done

I just want to note two very positive observations that come to mind when reviewing this thread.

Our logging has become very useful, helping us pinpoint the source of problems quickly! 🏆
Our distributed team is communicating well and able to iterate quickly! 🏆

So while the technical issues can be frustrating at times, let's feel good about the process! 😁

Another possibility would be to try to do this recipe over OPeNDAP, which would bypass the caching step completely.

When originally developing this recipe, I found that it was very easy to tip the OPeNDAP endpoint over with the sort of requests we were hitting it with. Scaling back made the recipe very slow to execute.

@martindurant Running workers with with intake/filesystem_spec#694 exhibits the same hanging behavior.

I would also try
--- a/fsspec/asyn.py
+++ b/fsspec/asyn.py
@@ -1,6 +1,7 @@
 import asyncio
 import asyncio.events
 import functools
+import gc
 import inspect
 import os
 import re
@@ -53,6 +54,7 @@ def sync(loop, func, *args, timeout=None, **kwargs):
     coro = func(*args, **kwargs)
     result = [None]
     event = threading.Event()
+    gc.collect()
     asyncio.run_coroutine_threadsafe(_runner(event, coro, result, timeout), loop)
     while True:

Following @martindurant 's suggestion from the coordination meeting this week I re-tested the pruned version of this recipe with workers using the explicit garbage collection plugin demonstrated here https://github.com/dask/dask/issues/7547#issuecomment-906939846.  My 4 test runs all resulted in hung opening so this does not appear to rectify our ongoing issue ref https://github.com/pangeo-forge/pangeo-forge-recipes/issues/177#issuecomment-892425152.

The bug that won't die. Then, indeed, making references might be better for all things everywhere. The difference is, that it deals with h6py files one-by-one, with no xarray involvement and no circular (object) references.

Just wanted to follow-up on this recipe. Is there anything I can help out debugging wise?

Thanks for checking in @norlandrhagen.
The blocking issue with h5py has been a pernicious one, and the solution likely extends beyond the specifics of this recipe.
Last I heard, @rabernat had some ideas of how to solve it, but I'm not sure if he's gotten a chance to work on them yet.

The brute force solution is to use copy_input_to_local_file=True. Have we tried that yet.
One longer-term workaround is to use fsspec reference maker to index the input files and open them as virtual Zarrs. If I'm the one to code that, it could be many weeks before I have time.
The ideal solution is of course to fix this "upstream", wherever that may be.

@norlandrhagen As @rabernat suggested if you include the copy_input_to_local_file=True setting this may alleviate the blocking fsspec issues.   If you add this, please ping me and I can re-run the tests.

Thanks all! @sharkinsspatial I just added the copy_input_to_local_file=True flag, so if you get a chance can you re-run the tests?

/run-recipe-test

Now the recipe test has failed with this error
{'PREFECT_PROJECT_NAME': '***', 'PREFECT__CLOUD__AUTH_TOKEN': '***', 'GH_API_TOKEN': '***', 'PYPI_TOKEN': '***', 'DOCKER_USER': '***', 'DEVSEED_BAKERY_DEVELOPMENT_AWS_US_WEST_2_SECRET': '***', 'ACTIONS_BOT_TOKEN': '***', 'DEVSEED_BAKERY_DEVELOPMENT_AWS_US_WEST_2_KEY': '***', 'github_token': '***', 'PREFECT_ACCESS_TOKEN': '***', 'DOCKER_ACCESS_TOKEN': '***', 'DEVSEED_BAKERY_DEVELOPMENT_AZURE_WESTEUROPE_CONNECTION_STRING': '***'}
  File "/process_recipe.py", line 35, in main
    register_flow(meta_yaml, bakeries_yaml, secrets, versions, prune=prune)
  File "/srv/conda/envs/notebook/lib/python3.8/site-packages/pangeo_forge_prefect/flow_manager.py", line 367, in register_flow
    targets = configure_targets(bakery, meta.bakery, recipe_meta.id, secrets, extension)
  File "/srv/conda/envs/notebook/lib/python3.8/site-packages/pangeo_forge_prefect/flow_manager.py", line 105, in configure_targets
    secret = secrets[target.private.storage_options.secret]
KeyError: 'DEVSEED_BAKERY_DEVELOPMENT_AZURE_UKWEST_CONNECTION_STRING'
ERROR conda.cli.main_run:execute(33): Subprocess for 'conda run ['python3', '/process_recipe.py']' command failed.  (See above for error)

https://github.com/pangeo-forge/staged-recipes/runs/3681121935?check_suite_focus=true

@rabernat  So this is an interesting failure that my limited git knowledge doesn't fully understand.  There is an unrelated error but I'll explain that in a moment.  If you examine the output carefully, this action is actually first running the workflow for the gpm-imerg recipe.  The workflow first needs to determine if the PR contains a meta.yaml file and what its location is in order to attempt to register the recipe as a flow.  To accomplish this, we first determine the files changed in the PR commits and then extract the first one that is a meta.yaml https://github.com/pangeo-forge/staged-recipes/blob/master/recipes/gpm-imerg/meta.yaml.  In this case the command https://github.com/pangeo-forge/staged-recipes/blob/master/.github/workflows/run-recipe-test.yaml#L26 is actually producing the full diff between master and the merge branch, the first meta.yaml in that diff list is the for the gpm-imerg recipe which has already been merged into master.
I know we had to switch from using https://github.com/marketplace/actions/get-all-changed-files to obtain this diff since this action did not support workflow-dispatch.  If someone with superior Git-fu than me can suggest the proper way to obtain only the changes from the PR branch in the diff I would love suggestions 🙏 . I'm also not thrilled about the lack of tests (or testability) for these workflow cases.  I'd also like some suggestions on how we can improve workflow testability to avoid incorrect behaviors like this one.
On the actual failure side, we updated the Github secrets key name used by the test Azure bakery but I failed to update the targets portion of the yaml. https://github.com/pangeo-forge/bakery-database/blob/main/bakeries.yaml#L41 so purely my error.  I'll rectify this in a moment

No worries Sean! It's a pretty complicated beast we have put together here.
There are quite a few git-fu ninjas associated with Jupyter / 2i2c we can call on for help here. I think the first step would be to try to turn this into a more compact, standalone issue. At that point we can tag a few people and see if they have ideas.
I agree it would be good to have more testability. One way to accomplish that might be to more and more of the heavy lifting of the actions into pangeo-forge-prefect and other CLI utilities.

👍 @rabernat I'll post a new consolidated issue today.

This recipe has been merged from https://github.com/pangeo-forge/staged-recipes/pull/151 to become https://github.com/pangeo-forge/terraclimate-feedstock. The recipe contained therein, like this one, is also directly adapted from the official tutorial example in the docs, so I believe there is no longer a need for this PR. Closing now, but please let me know if there's a need to re-open and we can discuss that.


Roll Flutter Engine from 2db85cbe6a01 to 7f2ced265e49 (4 revisions)

https://github.com/flutter/engine/compare/2db85cbe6a01...7f2ced265e49
2023-04-20<EMAIL_ADDRESS>Roll Fuchsia Linux SDK from Tun7i4VLz6ncx8JJJ... to AoPEjX8Xfq1v0h4kx... (flutter/engine#41357)
2023-04-20<EMAIL_ADDRESS>Roll Dart SDK from 88a3b66b50d6 to 7d165bd0bb5e (1 revision) (flutter/engine#41356)
2023-04-20<EMAIL_ADDRESS>Manual Skia roll from ad90b6bd4760 to c21e7df194c3 (flutter/engine#41341)
2023-04-20<EMAIL_ADDRESS>[impeller] convert src over to src for solid color (flutter/engine#41351)
Also rolling transitive DEPS:
fuchsia/sdk/core/linux-amd64 from Tun7i4VLz6nc to AoPEjX8Xfq1v
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC chinmaygarde@google.com,rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


Fix Markdown Checker GitHub Action

What type of suggestions?
Wrong Information
What page is concerned?
No response
What is the issue?
Fix Markdown Checker GitHub Action as it was created for Markdown files, and now it's not working on MDX. It may be just about adjusting the actions to take those in consideration, and the tool seems to support MDX, just not sure if it will work with our way of using them in the docs, so we may have to change it completely. I need to do some testing when I have a couple of minutes.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Need to fix all the issues before enabling them again.


[BUG] Fix the moksh logo in navbar of the home page

Description
Describe the bug:
The moksh logo in the navbar of home page is broken
Expected behavior:
Moksh logo should appear
Actual behavior:
As shown in the screenshot
Reproduction Steps
Steps to reproduce the behavior:

Go to 'index.html'
See navbar

Screenshots

the image is not broken per-se , the format used is works mostly on the latest versions of most browser. However, chrome supports it, try to open it in latest version of chrome.


Just installed and it doesn't accept any text input

I have the same issue

+1

Same issue. The terminal is totally blank and has no response to the text input.

I've checked the previous answer, which said terminal-plus is no longer maintained and recommended Termination by Fred-Barclay.

I have the same issue.


Unable to override default confluent monitoring interceptor properties

spring:
   cloud:
    stream:
      kafka:
        binder:
          brokers: localhost:9092
          configuration:
            confluent:
              monitoring:
                interceptor:
                  topic: my_intercept_topic
          producerProperties:
              interceptor:
                classes: io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor
          consumerProperties:
              interceptor:
                classes: io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor

I want to override default intercept confluent monitoring topic from _confluent-monitoring to my_intercept_topic. But unable to do so when using spring-cloud-stream-binder-kafka

You cannot set arbitrary properties via ...binder.configuration, only those properties defined by ProducerConfig and ConsumerConfig.
You can, however, set arbitrary propertied via the ...binder.producerProperties (and consumerProperties).
So
spring:
   cloud:
    stream:
      kafka:
        binder:
          brokers: localhost:9092
          producerProperties:
            interceptor:
              classes: io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor
            confluent:
              monitoring:
                interceptor:
                  topic: my_intercept_topic
          consumerProperties:
            interceptor:
              classes: io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor
            confluent:
              monitoring:
                interceptor:
                  topic: my_intercept_topic

should work ok.

Seeing this ProducerConfig log on the console:
[kafka-producer-network-thread|producer-3]o.a.k.c.p.ProducerConfig: The configuration 'confluent.monitoring.interceptor.topic' was supplied but isn't a known config which was misleading to be seen as misconfiguration earlier.
However able to see the following log as well:
confluent.monitoring.interceptor.publishMs=15000
confluent.monitoring.interceptor.topic=my_intercept_topic
Also able to produce monitoring messages to the topic which confirms the configurations is working.
Thanks!!

was supplied but isn't a known config

That's an unfortunate (bogus) message emitted by the KafkaProducer and is outside of our control.

Thanks for confirmation Gary


ExcelImporter: Import fields for Reference objects

add capability to use this kind of mapping
e.g.
Client
FirstName
LastName
Organization.Name  <- DefaultProperty
Organization.Address
This would require a dropdown ExcelColumnMap.PropertyName
that allows selecting properties of reference objects.
Makes sense?

I am sorry I do not make sense, please elab about the Organization properties

Have to think a bit more on this.
Can go to backlog

Closing issue for age. Feel free to reopen it at any time.
.Thank you for your contribution.


[Protected Request]  report 5754 "Create Pick" PickWhseWkshLine

Hello,
Is it possible to get the following variable as protected so we can use it in an extension.
PickWhseWkshLine

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


added my repo file

🚀 Congratulations for your submission, your points will be reflected in the leaderboard till night.


SmallWeather Wind Speed and Wind Direction

Wind speeds should use the descriptor category to set cloud speed, example values below. Values above 3 may be too fast and too compute intensive for WebGL so these are just scalability to test, may be worth to slowly scale and just cap at 3.

| Calm | 0.05
Light Air 0.1
Light Breeze 0.2
Gentle Breeze 0.35
Moderate Breeze 0.5
Fresh Breeze 0.7
Strong Breeze 1
Near Gale 1.5
Gale 2
Strong Gale 3
Whole Gale 4
Storm Force 5
Hurricane Force 6

Wind Direction based on the degree passed by weather API. This requires that every canvas has a new setting where DM sets the offset of North that the map uses as North (i.e. North is West, the offset is 270 or -90). The setting may use just an integer field, or a dropdown with "North is..." followed by North/East/South/West/Custom and an integer field to enter custom value. But generally DM should be used to degrees by integer, so enough to just support that.

Implemented in the new release :)


A new child of metalloendopeptidase activity

Hi,
May I request a new child term of metalloendopeptidase
activity (GO:0004222).
Suggestion:
metalloendopeptidase activity; GO:0004222
- matrix metalloproteinase 23 activity; GO:new
Synonym:
MMP-23.
Explanation:
matrix metalloproteinase 23 (MMP-23) is a new
metalloendopeptidase containing the distinct domain
structures, substrate specificity and cellular localisation.
It does not belong to any of the subfamilies, such as
collagenases, gelatinase, stromelysins or membrane-type
MMPs.
PMID: 9988691
Protein ID: Q9UBR9
Cheers,
Vivian
Reported by: vlee
Original Ticket: "geneontology/ontology-requests/1669":https://sourceforge.net/p/geneontology/ontology-requests/1669

thanks www.movied.org


Update web-api-help-pages-using-swagger.md

Data annotation attributes are used is on model's properties not the controller.

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

@pouladpld Congratulations on your 1st docs contribution, and thank you for fixing this! Your change will go live within the next 24 hours.


Bug Fixes

"~define neck" fails.
posting a file without any message may result in erroneous scoring on the leaderboard.
activity tracker should include message edits.
clean activity tracker tables in MariaDB.
clean up print statements OR create logging system to make it easier to detect potential issues on crash

Posting a file without any message results in zero points being awarded. I think this is fine, for now.

Currently, message edits do not reliably populate the author field. This is causes unnecessary inconsistencies.


SP age calculator update to start page

Added extra information about what SP age is and how it's calculated based on Feedex comments from DWP.
Trello: https://trello.com/c/MZA1qk94/471-13-oct-check-state-pension-age-more-information-content-change-request

Superseded by https://github.com/alphagov/smart-answers/pull/2781
Thank you @gabrielleacosta for your help.


Netlify addapter https imports in api too

Describe the problem
So using the Netlify adapter with edge enabled I saw it's possible to import external libs like in +page.server.ts like:
import { createClient } from "https://esm.sh/@supabase/supabase-js";
But it is not possible to do the same in APIs file like routes/api/test/+server.ts because you'll get at build: Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data are supported by the default ESM loader. Received protocol 'https:' would be nice if it was possible to run API on deno too.
I see that network imports are coming in node too, I don't know if that later will fix the issue.
Describe the proposed solution
Would be nice if this feature could be implemented so you can run API under deno too, not sure if is possible by changing the adapter only.
Also maybe a potential workaround is just to use directly Netlify edge functions I didn't try that to see how it goes.
Alternatives considered
No response
Importance
nice to have
Additional Information
No response

LE for other users:
The workaround works:
example netlify.toml:
[build]
  command = "npm run build"
  publish = "build"
  edge_functions = "edge-functions"

[[edge_functions]]
  path = "/edge-api/email"
  function = "email"

and for example in email.ts you can do HTTPS imports from denoland or esm.sh or other compatible sources and will work provided that the lib works with the Netlify Deno limitations, will not break the render function deployed by svelteKit and will be deployed as edge functions alongside the render one from svelteKit.
Probably the adapter could deploy functions from routes/api but since you can do it manually probably is not a very big priority.

I don't understand — you're saying an import from https://esm.sh/@supabase/supabase-js works in a +page.server.ts but not a +server.ts? There's no reason it should work in one but not the other, so this issue would benefit from a reproduction if you're able to create one

@Rich-Harris
So on further inspection, it just so happens that the supabase local node lib works in edge deno too.
That means that in reality, https module imports(network imports) work neither in +page.server.ts nor +server.ts as it will break the build - ERR_UNSUPPORTED_ESM_URL_SCHEME ( though as an edge function deployment should work in theory).
So yeah the only way to use external libs in svelteKit on edge is to use local node libs and hope it works on Deno too.
And for a json REST API you can deploy additional edge functions ( besides the render deployed by svelteKit adapter for SSR) where external https module imports will work.
I made a quick repo here: https://github.com/andrei0x309/sk-deno-api-route
As a matter of fact, I don't know if this can be overcome somehow, in order to not break the build, might be very complicated, and node already offers experimental network imports in v19+, so with time this will be resolved by itself might take 2 years until the features will reach LTS status though 😢
So I guess this can be closed.

This feels like the sort of thing that could probably be fixed with a Vite plugin. I'm not sure if an appropriate one already exists or if it would need to be created

In case someone discovers this issue.
This will also work if you don't want to deploy standalone edge functions, will be much better than trying to use Vite to transform imports:
	"scripts": {
		"build": "node --experimental-network-imports ./node_modules/vite/bin/vite build",
	},


Analyze different code highlighters to pick one for universal use

@rbbeeston Should this be closed?

@julioest has it been implemented everyplace it needs to, including the rouge highlighting?

completed


Update query editor to use new form components

This PR removes the legacy form components in favor of the inline field components. For more information: https://developers.grafana.com/ui/latest/index.html?path=/story/forms-inlinefieldrow--single
It also changes from multiple Select component to a single MultiSelect component.

Feel free to pick any of the changes (if any). If you'd like to keep the previous one-Select-per-row, I recommend that you use non-primary colors. Here's an example for inspiration: https://github.com/marcusolsson/grafana-csv-datasource/blob/6acc1429a79cd681259cf56f1bc64d7f76a5bfc0/src/SchemaEditor.tsx#L64-L69

Thanks for opening this pull request! Unfortunately, I can't get it running. I always receive the following error message:
https://reactjs.org/docs/error-decoder.html/?invariant=130&args[]=undefined&args[]=
I tried a lot of things to get around this error, but can't even find the origin.
Do you know where this error comes from? Did you perhaps forgot to commit another file?

Sorry, my bad. The new form components require >=7.3.0. I'll let you decide whether to use the new form components, or to stay compatible with versions <7.3.0

It is working now. Thanks for contributing and removing the legacy ui components

It is working now. Thanks for contributing and removing the legacy ui components


Cursor decorator performance

Discussed in https://github.com/BitPhinix/slate-yjs/discussions/293

Originally posted by bjarkih August 18, 2021
Hi all, and thanks for great work on Slate Yjs!
I have a question around the performance of the cursor decorator provided by the sample useCursors implementation. When editing a document that another user is editing at the same time, we're updating the decorate function as the remote cursor positions change, which leads to frequent re-renderings of the Slate editor. For me, this leads to a visibly sluggish editor experience as soon as multiple users are editing the same part of the document tree.
Has anyone else encountered the same problem and, hopefully, found some solutions or workarounds? Any input appreciated!
Bjarki

Left a comment in the discussion with our approach: https://github.com/BitPhinix/slate-yjs/discussions/293#discussioncomment-1285673.

Next has a new approach for a cursor plugin solving this issue


adjust simdjson header path

simdjson does this weird thing in cmake where it copies a header file into the root directory.  This prevents us from using this file in other build systems like buck which does not copy the header file.  I modified the cmake file so that it keeps the header file in the standard location, so that files that import it can be built with cmake and other build systems.

CC: @wanweiqiangintel

I need to update the docker container used in linux tests

@kevinwilfong ^^

This was resolved in other PRs


Fiori App Using launchpad -- Need to refresh browser to click on "Go to Site"

Hello,
Tutorial Link: https://developers.sap.com/tutorials/cp-portal-cloud-foundry-new-sapui5.html#4278ff9d-57b8-4e33-a2c8-1a5a5030076b
Step 6: Review the site
Issue:
After creating the app, group and assigning the app to the group, on navigating back to the "Site Directory", We are unable to click on "Go to Site", without refreshing the browser.
Please refer the image:

Labels to be added:
Sprint Label: SCPTest-2106A
Landscape Labels: trial1, trial2, trial3
prio: High
Scenario Label: SCPTest-psite
Kind regards,
Shubham Agarwal

Hi
I'm sorry for the late reply but I must have missed the notification of this issue.  I've never come across this issue before so I'm not able to help.  To have your issue inspected, please open a BCP for the EP-CPP-CF-DEV component.
Thanks
Lindsay

Thank you for your feedback. The issue seems to be resolved, so I am closing the issue.
If you still have questions, feel free to reopen the issue.