Add ARP app

We need an ARP app on both the clean and dirty side of SnabbDDoS so that we can respond to incoming ARP requests (i.e. router wants to resolve our MAC), typically for the dirty side, as well as resolving the MAC address of the router on the egress/clean side.
There is currently no ARP app for Snabb so it has to be written. @kbara has contemplated writing one for  the lwAFTR and we should incorporate that :)

It's my main priority for the week, since I decided to do it rather than take the week off. :-)

@kbara can't help but feel bad that you are deciding not to go for vacation and code instead. Nevertheless we are grateful :)

Eh, I had a 4-day weekend and am having a 3-day workweek, no need to feel bad. :-)

@kbara have you many any progress on the ARP app or been busy with other things? :)

Last week got eaten by other things, but I've made a bunch of progress on it this morning (all of the scaffolding/integration-y bits), and between that and the old draft I'm optimistic I'll have a working version up tonight. :)

@kbara so how do you intend to implement this? Will the ARP app only respond / query for ARP? What app is responsible for actually writing dst mac on egress frames? Responding to incoming ARP requests seems like no-brainer, we just start the app with our IP and our MAC. But resolving ARP entry for remote IP is trickier, I guess it needs to be a function that is called from another app, right?
code could be something like:
default_route = "<IP_ADDRESS>"

-- somewhere at the end of packet processing logic
earp = engine.app_table.egress_arp
egress_mac = earp.arp_table[default_route]
if egress_mac ~= nil then
    -- no ARP entry yet, send ARP query, throw away this packet and hope we have an ARP entry for the next packet
    earp.arp_query(default_route)
    packet.free(p)
    return
end
rewrite_dst_mac(p, egress_mac)
link.transmit(output, p)

I guess something needs to rate-limit this thing as well so we don't spew out X Mpps of ARP queries.

At the moment, it's entirely analogous to the way we're doing NDP for the lwaftr: there's a file that can make requests/replies (ndp.lua and arp.lua), and some glue code in (ipv4_apps, ipv6_apps) that calls those functions and acts as a Snabb app; that's where the egress_mac ~= nil style logic goes. Our setup.lua file shows how the apps go together.
I'm just writing some end-to-end tests at the moment, but the code is 'done', to the level the NDP code is. Diego's written some timer-retry code for requests for NDP today; I'll do something similar for ARP tomorrow (tonight's version will try once.)

Ah of course. I didn't think of the nd_lite app. Naturally we should follow the way it's designed! I'll have a look at that.
I'm very glad to hear about the progress you've made :)

We're not using nd_lite, for what it's worth - there was just enough design mismatch that I ended up rolling my own. The ndp.lua and ipv6_apps.lua files in src/apps/lwaftr/ are the best place to look for overall design until I PR this in an hour or two - the overall structure in the arp stuff I've just written is essentially identical. :-)

https://github.com/Igalia/snabbswitch/pull/295 :-)


Can't build docker image for TF2 and GPU/CPU

This is just a demo for the https://github.com/Xilinx/Vitis-AI/issues/1403 issue. I want to show in the .github/workflows/ci.yml that using the latest version of the repository the list of the following commands  fails:
cd ./docker/
./docker_build.sh -t cpu -f tf2

I attach a picture of the failure:

I close the pull request as it was just a demo for issue #1403 and it has been solved.


Update for TCA template to version 1.0?

Hi @crelies,
First off, thank you so much for your work! I've been using it and find it very valuable.
I wanted to inquire if you have plans to update the TCA template to version 1.0? I'm looking to utilize it in my upcoming project, and having an updated template would be fantastic.
Thank you in advance for your response and for all the hard work you've put into this.
Best regards!

Hey @ferencIOS ,
I was busy in another world 😆
Now I'm back and invested some time in improving and updating the template by taking the latest development into account.
Checkout the branch feature/1.7.x for a sneak preview.
I will create a new release tag soon.


[WIP] Create scripts to bundle dependencies

Problem
Bundled dependencies are not included in monorepo. Also their dependencies do not get installed
Solution
Create scripts to bundle them and install their dependencies

License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Should be noted that bundling script at prepack script proposed here is needed, as npm has bugs when packing bundleDependencies https://github.com/npm/cli/issues/7137


Create an SAPUI5 App from the Command Line

Tutorial URL: https://developers.sap.com/tutorials/cp-cf-sapui5-local.html
As a windows user, step 4 was giving me a hard time, as
"cp cf_deployment_resources/package.json  dist/package.json" will not work.
I added a cp.js file to root containing the line

require("fs").copyFile(process.argv[2], process.argv[3], err => {if (err) throw err});

replace "cp" in the script with "node cp.js" and that part is good to go.
please also hint us winnies to use "choco install make" (i found chocolateley easiest to install)
to "make" the last part of the script work.
edit: you might find this entirely normal, but build will also fail when you don't update your .npmrc file (see: https://blogs.sap.com/2017/05/16/sap-npm-registry-launched-making-the-lives-of-node.js-developers-easier/ )
no, i don't think devs usually do this a lot. i pearsonally would wish my live was a littlee harder and i did i had not to find out.

Hi @ND3V,
thank you for your very good feedback! I change the script to avoid the platform specific cp command and added a note to install make.
Can you explain the issue you're facing with the .npmrc file? The file should already come with all information to connect to the npm repository.
Thanks,
Marius

Hi IObert,
thanks for the update. Back then i was experiencing issues with the @sap modules, which went away once i updated my user-specific .npmrc - I was not able to reproduce this, as now the project-specific setting seems to suffice. I suspect a connectivity-issue or somesuch which coincidently resolved as i was creating a user-npmrc.
In my eyes, this can be closed.


.github: update pull-request template

Changes
The main goal of this change is to make sure that, not editing
anything from the template makes the release-note check unhappy
instead of happy, so that it "gives" feedback that something needs to
be updated.
Signed-off-by: Vincent Demeester<EMAIL_ADDRESS>/kind misc
Submitter Checklist
As the author of this PR, please check off the items in this checklist:

[X] Docs included if any changes are user facing
[ ] Tests included if any functionality added or changed
[X] Follows the commit message standard
[X] Meets the Tekton contributor standards (including
functionality, content, code)
[X] Release notes block below has been filled in
(if there are no user facing changes, use release note "NONE")

Release Notes
NONE

/cc @tektoncd/core-maintainers
Not sure if it's the right change but…

/retest

/lgtm


Error if required schematic properties without defaults are not provided

From @dansanderson on June 14, 2017 18:15
Currently if a schematic property is in schema.json's "required" list, nothing different happens if the option is not provided on the command line and doesn't have a "default". It should report a user error in this case.
Copied from original issue: angular/devkit#6

This still appears to reproduce on the latest version.
Initialized a new schematic with:
$ schematics blank --name test-schematic

Then added src/test-schematic/schema.json with:
{
  "properties": {
    "name": {
      "type": "string",
      "minLength": 1
    }
  },
  "required": [ "name" ]
}

This requires a single name argument without a default value. Used this in the schematic:
import { Rule, SchematicContext, Tree } from '@angular-devkit/schematics';

export function testSchematic(_options: any): Rule {
  return (tree: Tree, _context: SchematicContext) => {
    _context.logger.info('The name is', _options.name);
    return tree;
  };
}

And ran with:
$ npm run build && schematics test-schematic:test-schematic
The name is

Definitely should throw an error in such a case.
douglasparker@basilisk (~/Source/fixit-test)
[20-05-26 15:31:02]$ (cd test-schematic/ && ng version)

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/

Angular CLI: 9.1.7
Node: 12.16.1
OS: linux x64

Angular: 9.1.9
... animations, common, compiler, compiler-cli, core, forms
... platform-browser, platform-browser-dynamic, router
Ivy Workspace: Yes

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.901.7
@angular-devkit/build-angular     0.901.7
@angular-devkit/build-optimizer   0.901.7
@angular-devkit/build-webpack     0.901.7
@angular-devkit/core              9.1.7
@angular-devkit/schematics        9.1.7
@angular/cli                      9.1.7
@ngtools/webpack                  9.1.7
@schematics/angular               9.1.7
@schematics/update                0.901.7
rxjs                              6.5.5
typescript                        3.8.3
webpack                           4.42.0

Thanks for reporting this issue. This issue is now obsolete due to changes in the recent releases. Please update to the most recent Angular CLI version.
If the problem persists after upgrading, please open a new issue, provide a simple repository reproducing the problem, and describe the difference between the expected and current behavior.

Thanks for reporting this issue. This issue is now obsolete due to changes in the recent releases. Please update to the most recent Angular CLI version.
If the problem persists after upgrading, please open a new issue, provide a simple repository reproducing the problem, and describe the difference between the expected and current behavior.


Invalid onboarding script - (...) might wait for user input

Steps to reproduce:

Download install.ps1 from this github
Download onboarding package and script from https://security.microsoft.com
Place all three files on the same, dedicated, folder
Run: install.ps1 -OnboardingScript .\WindowsDefenderATPLocalOnboardingScript.cmd

Fails with the following output:
Exit-Install : [:10 23/05/23T08:17:36.923 install.ps1:468] Invalid onboarding script:
.\WindowsDefenderATPLocalOnboardingScript.cmd might wait for user input
At C:\temp\23maio2023\install.ps1:468 char:13

        Exit-Install -Message:"Invalid onboarding script: $Onboar ...

        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Exit-Install

Remove the pause in the Exit function in your onboarding script. It's at the end. Worked on mine.

I've updated the warnings indicating why we have this behavior (i. e. the Install.ps1 script would simply hang when pause is invoked).


 If this looks good, please merge.
bors d+

Please run git rebase -i origin/master and remove the commits that are not yours, then git push origin --force-with-lease.

bors merge


ffi-hunspell on Windows

Hello,
On Windows,
gem install ffi-hunspell    ...         gem install ffi --platform=ruby  ...  gem install ffi-hunspell
Successfully installed ffi-1.13.1-x64-mingw32
Successfully installed ffi-hunspell-0.5.0
require 'ffi/hunspell'
--> C:/Ruby27-x64/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- ffi/hunspell (LoadError)
...
i_lib': Could not open library 'hunspell-1.7': Le module sp�cifi� est introuvable.\r (LoadError)
.
Could not open library 'hunspell-1.7.dll': Le module sp�cifi� est introuvable.\r
.
Could not open library 'libhunspell-1.7.so.0': Le module sp�cifi� est introuvable.\r
.
Could not open library 'libhunspell-1.7.so.0.dll': Le module sp�cifi� est introuvable.\r
.
Could not open library 'hunspell-1.6': Le module sp�cifi� est introuvable.\r
....
Could not open library 'libhunspell-1.2.so.0.dll': Le module sp�cifi� est introuvable.\r
===============
Shall I Give up ?

Did you also install the hunspell DLL?

Thank you very much for your answer.
I tried to add hunspellx64 from
https://www.trichview.com/help-actions/hunspell.htm to
C:\Ruby27-x64\lib\ruby\gems\2.7.0\gems\ffi-hunspell-0.5.0
and C:\Windows\SysWOW64.
With the same error
C:/Ruby27-x64/lib/ruby/gems/2.7.0/gems/ffi-1.13.1-x64-mingw32/lib/ffi/library.rb:145:in
`block in ffi_lib'
I give up.
Perhaps I shall try again, one day, with Python.
Thanks again.
Le sam. 28 nov. 2020 à 08:40, Postmodern<EMAIL_ADDRESS>a
écrit :

Did you also install the hunspell DLL?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/postmodern/ffi-hunspell/issues/28#issuecomment-735051745,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACN2DT6YXG4ZK3R7ECCAVILSSCSQBANCNFSM4UFH5X4A
.

I don't have much experience running Ruby on Windows. You might consider trying Windows Linux SubSystem (WLS) or running in a Linux VM with VMWare Player.

Thank you for your help.
Le sam. 28 nov. 2020 à 17:51, Postmodern<EMAIL_ADDRESS>a
écrit :

I don't have much experience running Ruby on Windows. You might consider
trying Windows Linux SubSystem (WLS) or running in a Linux VM with VMWare
Player.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/postmodern/ffi-hunspell/issues/28#issuecomment-735255187,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACN2DT24U5UYE633TTWLQMLSSETCZANCNFSM4UFH5X4A
.


Could not find any locustfile! Ensure file ends in '.py'

Hello!
I follow the instructions in README.
Created configmaps from my local files (validated that they were created successfully):
kubectl create configmap my-loadtest-locustfile --from-file locustfile.py
kubectl create configmap my-loadtest-lib --from-file lib
This is the content of locustfile.py:
from locust import HttpUser, task, between

class QuickstartUser(HttpUser):
    wait_time = between(1, 2.5)

    @task
    def call_a(self):
        self.client.get("/a")

The contents or lib folder were copied from the example.
Then I install Helm chart with the command from README:
helm install locust deliveryhero/locust \
   --set loadtest.name=my-loadtest \
   --set loadtest.locust_locustfile_configmap=my-loadtest-locustfile \
   --set loadtest.locust_lib_configmap=my-loadtest-lib

After this I see the following error in master and worker pods:
Could not find any locustfile! Ensure file ends in '.py' and see --help for available options.
Please help :-)
I am just the README steps and it is not working...

I had the same problem and found a solution:

Rename your locustfile ("locustfile.py") to "main.py"
Remove configmap my-loadtest-locustfile
Recreate it again with kubectl create configmap my-loadtest-locustfile --from-file locustfile.py
Try installing helm chart again

It worked for me.


docker-compose up: write sysctl key kernel.domainname: open /proc/sys/kernel/domainname: permission denied

Until yesterday everything worked properly, then I updated my letsencrypt certificates on the host machine and now I can't start the container anymore. I don't know if it's related to the certificate renew but it's the only thing I changed.
Expected Behavior
docker-compose up mail should start the container
Actual Behavior
I get the following error message:
Removing mail
Recreating 7332a667ca93_mail ... error

ERROR: for 7332a667ca93_mail  Cannot start service mail: b'OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:430: container init caused \\"write sysctl key kernel.domainname: open /proc/sys/kernel/domainname: permission denied\\"": unknown'

ERROR: for mail  Cannot start service mail: b'OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:430: container init caused \\"write sysctl key kernel.domainname: open /proc/sys/kernel/domainname: permission denied\\"": unknown'
ERROR: Encountered errors while bringing up the project.

Your Environment
I'm running the container on a vServer with Ubuntu 18.04 and Kernel 5.2.4-1.el7.elrepo.x86_64 with the latest mailserver version. 1GB RAM is available.
docker-compose --version: docker-compose version 1.24.0, build 0aa59064
docker --version: Docker version 19.03.1, build 74b1e89
.env snippet for hostname/domain:
HOSTNAME=mail
DOMAINNAME=mydomain.eu
CONTAINER_NAME=mail

# empty => uses the `hostname` command to get the mail server's canonical hostname
# => Specify a fully-qualified domainname to serve mail for.  This is used for many of the config features so if you can't set your hostname (e.g. you're in a container platform that doesn't let you) specify it in this environment variable.
OVERRIDE_HOSTNAME=

It seems like a combination of the latest docker update (maybe related to https://github.com/docker/for-linux/issues/743) and restricted access to kernel parameters on my vServer. I downgraded docker to version 18.09 and it's working again:
sudo apt install docker-ce=5:18.09.8~3-0~ubuntu-bionic


add nginx-wsgi-django-mysql example

Signed-off-by: IML<EMAIL_ADDRESS>

This is an implementation example written in Nginx + Wsgi (gunicorn + Django) + MySQL with docker compose.


[WIP] Test: Split SqlServerDatabaseCreator tests

Verifying on AppVeyor first.
In xunit each class is collection of tests. Collections are run in parallel but tests in particular collection are run sequentially.
On my dev box, SqlServer.FunctionalTests take 160s time on average. If I remove SqlServerDatabaseCreatorTest then it takes 70s. So CreatorTest is bottleneck here. This PR splits them out in different classes so that they can be run in parallel.
After this change, average time for SqlServer.FunctionalTests on my dev box is ~90s.

AppVeyor passed.

You can keep them in one file to preserve the logical grouping.
Maybe they can even be just nested classes, this way they can have better names.

Closing this in favour of #8028


Add pre/post hooks support for compile and clean

Can we publish a release? This is needed to resolve https://github.com/tuncer/re2/issues/13.

Have you tested it? I'll publish to hex if you verified it worked for
re2
--
Tristan Sloughter
<EMAIL_ADDRESS>On Thu, Jul 14, 2016, at 12:28 PM, Tuncer Ayaz wrote:

Can we publish a release? This is needed to resolve tuncer/re2#13[1].
— You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub[2], or mute the
thread[3].

Links:

https://github.com/tuncer/re2/issues/13
https://github.com/blt/port_compiler/pull/24#issuecomment-232767490
https://github.com/notifications/unsubscribe/AACNg42PULLImEUgUL7lbAD1QjqUTQoLks5qVo3wgaJpZM4JMsZN

Verified that compile and clean work in re2.

@tsloughter, ping?

I'll get this published today.

Published 1.3.0 https://hex.pm/packages/pc/1.3.0

Thanks.


GHC 8.2.2

I'm guessing we'll want to move forward getting a GHC 8.2.2 nightly out pretty soon, so it'd be awesome if this could get resolved quickly.
base-<IP_ADDRESS> is out of bounds for:

[ ] crypto-enigma-<IP_ADDRESS> (>=<IP_ADDRESS> && < =<IP_ADDRESS>). Roy Levien<EMAIL_ADDRESS>@orome. @orome. Used by: library, test-suite

ghc-8.2.2 is out of bounds for:

[ ] intero-0.1.23 (>=7.8 && < 8.2.2). Chris Done @chrisdone. @commercialhaskell. Used by: executable

ghc-prim-<IP_ADDRESS> is out of bounds for:

[ ] ghc-compact-<IP_ADDRESS> (==<IP_ADDRESS>). No maintainer.. Used by: library

crypto-enigma-<IP_ADDRESS> should work now.

@chrisdone intero is the only package remaining so I disabled it for now so I can kick of the 8.2.2 build.

No probs. With a GHC change usually the API's quite a bit different, so this might take a while. I'll test it out, though.


SideScroller not working with Package

Package does not seem to work.  Tried every combination.
Glad to see the player up and running again with vectors!  Nice work.
http://scratch.mit.edu/projects/120171080/
http://phosphorus.github.io/#120171080
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36

Embed seems to have similar issues.  Using Chrome win 10.

Package requires the fix that Nathan made to index.html (canvg stuff), be also applied to app.html
Don't know about embed though... maybe similar.

Fixed

Thanks!  You are awesome and appreciated.


URL Cleanup

This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).
HTTP URLs that Could Not Be Fixed
These URLs were unable to be fixed. Please review them to see if they can be manually resolved.

http://irisclasson.com/2018/09/18/asp-net-core-openidconnect-why-is-the-claimsprincipal-name-null/ (200) with 1 occurrences could not be migrated:
(https result SSLHandshakeException).

Fixed URLs
Fixed But Review Recommended
These URLs were fixed, but the https status was not OK. However, the https status was the same as the http request or http redirected to an https URL, so they were migrated. Your review is recommended.

http://network.pivotal.io/open-source (301) with 1 occurrences migrated to:
https://network.pivotal.io/open-source (https result ReadTimeoutException).
http://Default_OAuthServiceUrl/oauth/authorize?response_type=code&client_id=Default_ClientId&redirect_uri=http%3A%2F%2Ffoo.bar%2Fredirect&scope= (UnknownHostException) with 1 occurrences migrated to:
https://Default_OAuthServiceUrl/oauth/authorize?response_type=code&client_id=Default_ClientId&redirect_uri=http%3A%2F%2Ffoo.bar%2Fredirect&scope= (https result UnknownHostException).
http://credhubServer/api/ (UnknownHostException) with 1 occurrences migrated to:
https://credhubServer/api/ (https result UnknownHostException).
http://default_oauthserviceurl/oauth/authorize (UnknownHostException) with 1 occurrences migrated to:
https://default_oauthserviceurl/oauth/authorize (https result UnknownHostException).
http://domain (UnknownHostException) with 11 occurrences migrated to:
https://domain (https result UnknownHostException).
http://foo.bar (UnknownHostException) with 6 occurrences migrated to:
https://foo.bar (https result UnknownHostException).
http://foo.bar.com/keyurl (UnknownHostException) with 1 occurrences migrated to:
https://foo.bar.com/keyurl (https result UnknownHostException).
http://foo.bar/redirect (UnknownHostException) with 1 occurrences migrated to:
https://foo.bar/redirect (https result UnknownHostException).
http://redirectUri (UnknownHostException) with 1 occurrences migrated to:
https://redirectUri (https result UnknownHostException).
http://uaa-server (UnknownHostException) with 1 occurrences migrated to:
https://uaa-server (https result UnknownHostException).
http://uaa-server/oauth/token (UnknownHostException) with 2 occurrences migrated to:
https://uaa-server/oauth/token (https result UnknownHostException).

Fixed Success
These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

http://redis.io/ with 1 occurrences migrated to:
https://redis.io/ (https result 200).
http://redis.io/documentation with 1 occurrences migrated to:
https://redis.io/documentation (https result 200).
http://redis.io/topics/license with 1 occurrences migrated to:
https://redis.io/topics/license (https result 200).
http://aspnetwebstack.codeplex.com/wikipage?title=Contributors with 1 occurrences migrated to:
https://aspnetwebstack.codeplex.com/wikipage?title=Contributors (https result 301).

Ignored
These URLs were intentionally ignored.

http://localhost with 8 occurrences
http://localhost/ with 6 occurrences
http://localhost/signin-oidc with 1 occurrences
http://localhost/tokenUrl with 7 occurrences
http://localhost:81 with 2 occurrences
http://localhost:82 with 2 occurrences

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: spring-operator sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Add transaction styling for aiohttp integration

I doubt using path is generally useful. It can contain a lot of extra data that varies between requests. I would go for using the route pattern instead.

thx, can you fix flake8 errors?
https://travis-ci.com/github/getsentry/sentry-python/jobs/401912287

@untitaker Done

Can you file a PR against https://github.com/getsentry/sentry-docs for documenting this? In the meantime I will merge this, thanks!

https://github.com/getsentry/sentry-docs/pull/2546

This is so useful, thanks!


POC - Server Plugin Init + Fix error adding extension

Add sinple server endpoint in jupysql-plugin
Fix the server enable command issue jupyter server extension enable jupysql-plugin -> jupyter server extension enable jupysql_plugin

@edublancas
I've discovered the server extension example, this is the helloworld endpoint we might use for connecting with our dashboard backend service.

I think this PR is demonstrating we can create the server extension with basic endpoint, also fixing the server enable command issue in the README.md (should be jupyter_plugin)

does this need review? @tonykploomber  looks like this is a proof of concept and the tests are failing

@edublancas
Yes, this is only used for POC and we might apply this in the future.
Close the PR


Feature request: Airflow custom operator for meltano ELT jobs

Having a Airflow operator to run ELT jobs will be a really helpful feature. Especially for someone who already uses airflow for their pipelines on MWAA(AWS).

Thanks for the issue! I think this will make more sense once we have a managed offering.

Still relevant


is there a plan to support HunyuanVideo 13B Model?

Great Work!!!
I believe our community needs an effective, play-and-plugin method to accelerate large video generation models.
It would be wonderful to support HunyuanVideo!

Thank you for your interest in our work. I may try it, but I cannot guarantee. You can also have a try!

Dear author, thank you for your response.
I have carefully reviewed the code and the paper, and I noticed that the ### coefficients for the rescale_func need to be calculated in advance. Could you please provide the code for calculating these coefficients?

You may refer to issue 2. Support for HunyuanVideo will be released in a few days. Please stay tuned.

Hi, @Owen718 . Teacache supports HunyuanVideo now. You can try it!

Thanks !


[WIP] Attempt at making service worker autoUpdate faster

I know this isn't complete, and there should be a new configuration option. After reading over this: https://developers.google.com/web/fundamentals/instant-and-offline/service-worker/lifecycle#waiting
I realized, even though auto update works and downloads a new service worker, it won't be used until all tabs are closed and reopened.
My idea is to add a config option, forceRefreshOnUpdate or something. When a new service worker is downloaded, it will be force updated, and once installed, it will reload the page.
This way when pushing out new app releases, everyone will immediately be on it. Without doing a page refresh, you can run into problems with the old code fetching new files, which is why I want to do a page reload. This should be fast, since it is already cached by the new worker.
It would also be cool to add some sort of hook into this, so that in your app you can display a notice saying the app was just updated.
Thoughts?

I think what you is this:
// in config
new OfflinePlugin({
  ServiceWorker: { events: true },
  AppCache: { events: true },
  autoUpdate: true,
});

// in runtime
const OfflinePluginRuntime = require('offline-plugin/runtime');

OfflinePluginRuntime.install({
  onUpdateReady: function() {
    OfflinePluginRuntime.applyUpdate();
  },
  onUpdated: function() {
    window.location.reload();
  }
})

Also, seems to be supported since chrome 42? https://developer.mozilla.org/en-US/docs/Web/API/ServiceWorkerGlobalScope/skipWaiting

@zackify right, it's supported there in ServiceWorkerGlobalScope, no in main thread as you did.

@NekR ah, thanks. I will try out your code example

@zackify sure. I think that should be it. If it doesn't work (but it supposed to), then we can fix that :-)

This is awesome! https://www.youtube.com/watch?v=58Dpy48-OKI&feature=youtu.be Thanks for the help!

@zackify sure, you are welcome 😉


Logs search and filter functionality #213

What this PR does / why we need it:
This PR adds search and filter functionality to log view.
Which issue(s) this PR fixes

Fixes #213

Special notes for your reviewer:
Note: this is not final UI, still waiting for final design

Starting to look at this now.

Opened #701 which is work that can happen on top of this after it is merged.


[Test Framework] Integrate apps with Galley

The most recent refactoring broke the apps component when Pilot is being used with Galley. The apps register their services with the ServiceManager directly. When Pilot is configured with Galley, however, it doesn't use the ServiceManager, which means that the app services are never properly registered with Pilot.

Changed the Pilot and Apps component to require Galley to be configured, to avoid confusion.

Removed the ServiceManager altogether - Galley is used for service registration.

Fixes #13090

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: nmittler
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: ozevren
If they are not already assigned, you can assign the PR to them by writing /assign @ozevren in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

galley/OWNERS
pkg/OWNERS [nmittler]
tests/integration/OWNERS [nmittler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nmittler: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
0eca595d8a0d7ff8261decebd195cee135d3cebe
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
0eca595d8a0d7ff8261decebd195cee135d3cebe
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
0eca595d8a0d7ff8261decebd195cee135d3cebe
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: PR needs rebase.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: howardjohn, nmittler
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: ozevren
If they are not already assigned, you can assign the PR to them by writing /assign @ozevren in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

galley/OWNERS
pkg/OWNERS [nmittler]
tests/integration/OWNERS [nmittler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: howardjohn, nmittler
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pkg/OWNERS [nmittler]
tests/integration/OWNERS [nmittler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@howardjohn @smawson PTAL
I've added a hold for now. I've just pushed a major change to refactor the echo component so that it can be a complete replacement for the apps component. I've added full capabilities for both native and kube.  I've tested native, but will work tomorrow on swapping over the existing apps tests to use echo instead.

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
0eca595d8a0d7ff8261decebd195cee135d3cebe
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
5dcf36e9feed3679a954a5414a3f559717578c90
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
5dcf36e9feed3679a954a5414a3f559717578c90
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
5dcf36e9feed3679a954a5414a3f559717578c90
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
5dcf36e9feed3679a954a5414a3f559717578c90
link
/test istio-integ-k8s-tests

prow/istio-integ-local-tests.sh
4028704c6c49956f2e78897b520ba25b4e52e4cd
link
/test istio-integ-local-tests

prow/istio-pilot-multicluster-e2e.sh
4028704c6c49956f2e78897b520ba25b4e52e4cd
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-local-tests.sh
4028704c6c49956f2e78897b520ba25b4e52e4cd
link
/test istio-integ-local-tests

prow/istio-pilot-multicluster-e2e.sh
4028704c6c49956f2e78897b520ba25b4e52e4cd
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
4028704c6c49956f2e78897b520ba25b4e52e4cd
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@smawson

This PR is rather big, is it possible to split into more manageable pieces? Can we get here more incrementally?

Yes, sorry ... I've reverted the changes to echo and moved them over to #13175.

It feels like we're doing a lot of hackery around how we create the sidecars, I'd like us to figure out how to clean that up and have a roadmap for making this all much simpler. That may require work on our actual components to make them run the way we want, but we should do that anyway :)
I initially had planned on having a separate "sidecar" or "envoy" component rather than having apps take care of it, and using as much of the real code as possible. We should discuss how we want that to look in the long term at some point (I only have half-baked opinions but something doesn't quite feel right here).

Absolutely agree. This PR is really just taking baby steps to:

Fix a bug in apps that was introduced with the new framework changes
Merge 2 components that are really the same thing (apps and echo).

I've raised #13177 to track progress toward addressing the hackery. This is very important.

/test istio-integ-local-tests

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: howardjohn, nmittler
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pkg/OWNERS [nmittler]
tests/integration/OWNERS [nmittler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
4028704c6c49956f2e78897b520ba25b4e52e4cd
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
56b17de09b780dd388cf141173d56624af7c2609
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
56b17de09b780dd388cf141173d56624af7c2609
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
56b17de09b780dd388cf141173d56624af7c2609
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
05a0b6098d99c8f0101b74a6ce2049ca21647ebb
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn I had to rebase ... PTAL

/lgtm
Remove the hold label if you want this in

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: howardjohn, nmittler
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pkg/OWNERS [nmittler]
tests/integration/OWNERS [nmittler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nmittler: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
05a0b6098d99c8f0101b74a6ce2049ca21647ebb
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


WP CLI: Add resize support

When process.stdout resizes, we can send a resize command through to the CLI runner to ensure the rows/columns are accurate after resize.

Not critical to custom commands; bump to v5.


feat(app): App advanced settings u2e info

Overview
Move USB-to-Ethernet adapter information from Robot settings to AppSettings Advancedsettings.
Display the adapter information such as Description(deviceName), Manufacturer, and Driver version(for Windows).
This fixes #9849
Design
https://www.figma.com/file/bVtCogdwiXj3ndpk4AWXCx/Milestone-1%3A-Read-and-Execute?node-id=1297%3A87906

Changelog

Added a new section to Advanced Settings
Added test cases for U2E adapter information to Addvanced Settings test

Review requests

Risk assessment
low

This addition looks good!
I noticed that the background color of the page is slightly different (grey/white) right around where this setting appears.
This is definitely unrelated to your changes, but it looks like making one of the top level divs in the NextGenApp.tsx file have min-height: 100% instead of height: 100% fixes the issue.
It might be worth making that change in this PR since it is so contained.
      <Box width="100%">
        <Breadcrumbs pathCrumbs={pathCrumbs} />
        <Box
          position={POSITION_RELATIVE}
          width="100%"
          height="100%" // <--- change this to minHeight
          backgroundColor={COLORS.background}
          overflow={OVERFLOW_SCROLL}
        >
          <ModalPortalRoot />

I though minHeight made sense, but actually, it fixed the entire screen and didn't allow me to scroll for items under U2E information. We should keep height="100%", shouldn't we?

@b-cooper I noticed the background thing wouldn't come from NextGenApp.tsx but AppSettings.tsx
The current AppSettings is a little bit different from Figma's design.

@b-cooper I updated a few files to modify AppSettings. (added padding and fixed the issue you mentioned in this PR).
Could you take a look again?
Also, I noticed a couple of typos in labware.


#1 Solving problems with Geolocation

https://github.com/flaviostutz/ruller-sample-feature-flag/issues/1

You sent your personal maxmind key along with the commit. Please remove it :)

On the database copy to /opt, please don't use *, make it explicit about the file(s) your are copying because this folder will be present on the final build and we want to keep the imagem as small as possible.

The modifications are done.
Thank you a lot for your advices! I really appreciated them.


Update ESACCI Landcover CMORizer (python version) and downloader (pft yearly data, v2.0.8)

Description
This PR provides an updated downloading script and a new formatting script for ESACCI LANDCOVER v2.0.8 pft data (yearly) in Python replacing the old NCL version. This python version is supporting yearly data for variables BARE, GRASS-MAN and GRASS-NAT, TREES-BD, TREES-BE, TREES-ND, TREES-NE, SHRUBS-BD, SHRUBS-BE, SHRUBS-ND, SHRUBS-NE and it is producing 5 output files for each year as follow:

treeFrac = TREES-BD + TREES-BE + TREES-ND + TREES-NE
shrubFrac = SHRUBS-BD + SHRUBS-BE + SHRUBS-ND  + SHRUBS-NE
grassFrac = GRASS-NAT
cropFrac = GRASS-MAN
baresoilFrac = BARE

Before you get started

[ ] ☝ Create an issue to discuss what you are going to do

Checklist
It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

[x] 🛠 This pull request has a descriptive title
[x] 🛠 Code is written according to the code quality guidelines
[ ] 🛠 Documentation is available
[ ] 🛠 Tests run successfully
[x] 🛠 The list of authors is up to date
[x] 🛠 Any changed dependencies have been added or removed correctly
[ ] 🛠 All checks below this pull request were successful

New or updated data reformatting script

[ ] 🛠 Documentation is available
[ ] 🛠 The dataset has been added to the CMOR check recipe
[ ] 🛠 The dataset has been added to the shared data pools of DKRZ and Jasmin by the @ESMValGroup/OBS-maintainers team
[ ] 🧪 Numbers and units of the data look physically meaningful

To help with the number of pull requests:

🙏 We kindly ask you to review two other open pull requests in this repository

I just tried this CMORizer. Downloading the input files was successful but running the formatting script results in the following error for variable shrubs-bd:
ValueError: Coordinate 'time' has different points for the LHS cube 'unknown' and RHS cube 'Broadleaved deciduous shrubs'. 2
Variables baresoilFrac, cropFrac and grassFrac could be formatted successfully.

@schlunma Apparently trying to amend the fix_coords function in the utilities in order to reuse the code to fix lat, lon and time for this CMORizer is creating an issue during the commit related to the yearly data. I have therefore created a dedicated fix_coords in the esacci_landcover script and kept the original fix_coords in utilities and this fixed the commit issue.

Thanks for your reviews @schlunma @axel-lauer! I have addressed all your change requests and the cmorizer and the downloader are working as expected and the output files are passing the ref_check_obs check. However  the commit is failing due to some issues in the updated fix_coords function in utilities. Any suggestion on how to fix it as well?

I guess  have identified the issue in this line of code:
                    cube = cube = cube.intersection(longitude=(0.0, 360.0))
that should be:
                    cube = cube.intersection(longitude=(0.0, 360.0))
It seems that all the checks are fine now but it is strange that the output file were created fine also with the previous version.

* The output for variables `shrubFrac` and `treeFrac` contain the variable name `grassFrac` and seemingly also the actual data for `grassFrac` instead of  the correct data. Not sure how this can happen.

See https://github.com/ESMValGroup/ESMValTool/pull/3727#discussion_r1698689237

quick question: why all the changes in all the cmorizers that now redefine the cube or cubesave etc instead of applying the fix_cube method in place, as it was before?

The old code that was supposed to put the cube in [0, 360] longitude didn't do it's job, so it has been replaced with the (easier)
cube = cube.intersection(longitude=(0.0, 360.0))

But since cube.intersection is not in-place, we need to actually return the cube 👍

quick question: why all the changes in all the cmorizers that now redefine the cube or cubesave etc instead of applying the fix_cube method in place, as it was before?

The old code that was supposed to put the cube in [0, 360] longitude didn't do it's job, so it has been replaced with the (easier)
cube = cube.intersection(longitude=(0.0, 360.0))

But since cube.intersection is not in-place, we need to actually return the cube 👍

good call, thanks, Manu! :beer:

@schlunma if you approve the PR then I can have a bird's eye view, and merge - cheers both for putting this together and reviewing :beer: x2

oops cheers to @axel-lauer for testing etc too :beer:

@schlunma if you approve the PR then I can have a bird's eye view, and merge - cheers both for putting this together and reviewing 🍺 x2

Thanks V! Will approve once this is ready 👍 @diegokam is on holidays at the moment, but this is not super urgent.

sounds good, take your times, guys :beer:

I am back to work! :) Thanks for your patience @schlunma @valeriupredoi @axel-lauer and final fix and comments.
I'll try to address them asap!

Hi @schlunma @valeriupredoi @axel-lauer , I do confirm that I have addressed all your latest comments and change requests related to the cmorizer and downloader. Please note that in the cmorizer I had to reduce the block_size to 25 as I was experiencing again memory issues whenever the shrubFrac and treeFrac variables/ files were processed/created.
I am still trying to fix the output filename yyyy01-yyyy12 format as a first working solution I tried locally, by modifying the save_variable funtion in the utilities, it is creating a commit issue whenever I try to push it.
The code is therefore now ready for your final review, thank you!

Great!

Please note that in the cmorizer I had to reduce the block_size to 25 as I was experiencing again memory issues whenever the shrubFrac and treeFrac variables/ files were processed/created.

Where do you run the CMORizer? How much memory do you have? This really shouldn't be a problem. One year of input data is 1.7G, so 30 years should be around 50G. This shouldn't be a problem on Levante.

I am still trying to fix the output filename yyyy01-yyyy12 format as a first working solution I tried locally, by modifying the save_variable funtion in the utilities, it is creating a commit issue whenever I try to push it.

You could add a frequency: yr to the ESACCI-LANDCOVER.yml config file and check for that in the utilities code. This would also make it fully backwards-compatible.

Great!

Please note that in the cmorizer I had to reduce the block_size to 25 as I was experiencing again memory issues whenever the shrubFrac and treeFrac variables/ files were processed/created.

Where do you run the CMORizer? How much memory do you have? This really shouldn't be a problem. One year of input data is 1.7G, so 30 years should be around 50G. This shouldn't be a problem on Levante.

I am still trying to fix the output filename yyyy01-yyyy12 format as a first working solution I tried locally, by modifying the save_variable funtion in the utilities, it is creating a commit issue whenever I try to push it.

You could add a frequency: yr to the ESACCI-LANDCOVER.yml config file and check for that in the utilities code. This would also make it fully backwards-compatible.

Thanks @schlunma ! I do confirm that by following your suggestions I was able to fix the remaining issues with the cmorizer and that now it is giving as output files what we are expecting. I changed back the block_size to 100 and this has advantages both in terms of output file size, that is smaller than thos generated with a block_size=25, and in terms of overall processing time. Looking forward to your final review then, thanks!

very many thanks, chaps! A couple of very minor suggestions from me 🍺

Thank you for your suggestions @valeriupredoi! I have now addressed them as well! :)

@axel-lauer would you be a goodman and have another look and see if your review comments have been addressed, then merge this, please, bud :beer:

@axel-lauer would you be a goodman and have another look and see if your review comments have been addressed, then merge this, please, bud 🍺

frenly ping monsieur @axel-lauer :beer:

Thanks @axel-lauer for testing the downloader and the formatter and for your suggestion regarding the updated information in the input.rst. I do confirm that I have just addressed it and everything look in good shape to be merged then :)


support for ND reshape

Description
Please include a short summary of the change.
this PR adds support for cp.reshape(order='C') on nd expressions
Issue link (if applicable): #2534
Type of change

[x] New feature (backwards compatible)
[ ] New feature (breaking API changes)
[ ] Bug fix
[ ] Other (Documentation, CI, ...)

Contribution checklist

[x] Add our license to new files.
[x] Check that your code adheres to our coding style.
[x] Write unittests.
[x] Run the unittests and check that they’re passing.
[x] Run the benchmarks to make sure your change doesn’t introduce a regression.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.William Zijie Zhang seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

That's amazing it works just with these changes! Does transpose work already?

That's amazing it works just with these changes! Does transpose work already?

yea I think it does! I noticed that numpy would just reverse the shape tuple for transposing ndarrays, so I tried that for reshape in cvxpy and it seemed to work!

I'm going to merge this since the CI failures are unrelated.


Boat going online/offline every hour

Since upgrading to the latest version of the signalk plugin, I am continuously getting "Boat went Online" and "Boat went Offline" email messages.  These messages occur about once an hour, about 5 minutes apart.

I did notice the same issue, will check it out.


adding codecov

Added the XML file

@mpourostad
I think modification toREADME will also be required

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@c731fe7). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #93   +/-   ##
=========================================
  Coverage          ?   97.08%
=========================================
  Files             ?        8
  Lines             ?      480
  Branches          ?        0
=========================================
  Hits              ?      466
  Misses            ?       14
  Partials          ?        0

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


empty PNG asset from contextual menu, ability to change size

added pages for sprite edit panel
-nine patch became a page of sprite edit panel
-added resolution resizing page to sprite edit panel

conflicts @Daeva-Git


Test for group creation, but not user creation

In https://github.com/bgruening/docker-ipython-notebook/commit/ea2a92df12049cf5926d043c875f835103665bed it's mentioned that the host folder GID was in use by the docker image. Would it not be unreasonable to think that the host folder UID might also already exist in the system?

Users are always above 1000 isn't it? In this container we do not have any other user over 1000. At least this was my thinking. 😊

I was thinking of the case where the admin has set the galaxy user on the host as a system user, thus with an ID < 1000.

If, like most of us, galaxy is a regular user, then it would be just fine.

No one has reported issues with this, so closing the old issue.


Feature universe

Added the universe simulator. And switched to Dask.

Codecov Report

Merging #7 into master will decrease coverage by 1.41%.
The diff coverage is 84.01%.

@@            Coverage Diff             @@
##           master       #7      +/-   ##
==========================================
- Coverage   92.43%   91.01%   -1.42%
==========================================
  Files          32       37       +5
  Lines        1137     1425     +288
==========================================
+ Hits         1051     1297     +246
- Misses         86      128      +42

Impacted Files
Coverage Δ

cosmogrb/grb/gbm_grb.py
100.00% <ø> (ø)

cosmogrb/utils/logging.py
100.00% <ø> (ø)

cosmogrb/utils/file_utils.py
47.61% <47.61%> (ø)

cosmogrb/utils/package_utils.py
71.42% <60.00%> (-6.35%)
:arrow_down:

cosmogrb/grb/grb.py
95.23% <81.25%> (-3.52%)
:arrow_down:

cosmogrb/config.py
86.20% <86.20%> (ø)

cosmogrb/universe/universe.py
91.57% <91.57%> (ø)

cosmogrb/__init__.py
100.00% <100.00%> (ø)

cosmogrb/test/test_basic.py
100.00% <100.00%> (ø)

cosmogrb/test/test_universe.py
100.00% <100.00%> (ø)

... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cc997d1...82bd29d. Read the comment docs.


Envoy resets http2 stream after downstream response has started

If you are reporting any crash or any potential security issue, do not
open an issue in this repo. Please report the issue via emailing
<EMAIL_ADDRESS>where the issue will be triaged appropriately.
Title: Upstream stream reset during gRPC forwarding, causing client failures
Description:

Describe the issue.

We're currently in the process of migrating our sidecars from Envoy 1.10 -> 1.17; we're also introducing an Amazon ALB as the sole upstream to our sidecars, which forward to gRPC targets.  This ALB listens on 443, so our sidecars forward gRPC with TLS to this ALB.  The gRPC client is a containerized Node service, uses grpc-js, and has been operational for a few years now.
After migration (so grpc client service -> Envoy sidecar (1.17) -> AWS ALB -> target), we're noticing the following client failure under load, at around a 2% clip (but increases as more request connections are made):
RST_STREAM with code 0

After further investigation, we're seeing a 1:1 correlation between the above failure (timestamp + count) with the following Envoy debug log group:
[2021-04-29 02:37:45.669][43][debug][router] [source/common/router/router.cc:1026] [C44][S4956500439520369696] upstream reset: reset reason: remote reset, transport failure reason:
[2021-04-29 02:37:45.669][43][debug][http] [source/common/http/filter_manager.cc:828] [C44][S4956500439520369696] Resetting stream due to upstream_reset_after_response_started{remote reset}. Prior headers have already been sent
[2021-04-29 02:37:45.669][43][debug][http] [source/common/http/conn_manager_impl.cc:189] [C44][S4956500439520369696] doEndStream() resetting stream
[2021-04-29 02:37:45.669][43][debug][http] [source/common/http/conn_manager_impl.cc:1533] [C44][S4956500439520369696] stream reset
[2021-04-29 02:37:45.669][43][debug][http2] [source/common/http/http2/codec_impl.cc:873] [C44] sent reset code=0
[2021-04-29 02:37:45.669][43][debug][http2] [source/common/http/http2/codec_impl.cc:964] [C44] stream closed: 0

The relevant log item here appears to be this statement:
Resetting stream due to upstream_reset_after_response_started{remote reset}. Prior headers have already been sent

Based on this comment in the Envoy project, I'm interpreting this as the upstream resetting the stream (eg "remote reset") prior to the response fully being transmitted back down.
My questions are:

Is this the correct interpretation of this behavior?  If not, what is going on here?
Are there any specific cluster/route configurations I can apply to mitigate this?  I'm a relative gRPC novice, but I've been toying with some cluster timeouts and max_stream_duration configurations, but with no avail.

[optional Relevant Links:]

Any extra documentation required to understand the issue.

Envoy configuration below, showing the egress listener + upstream cluster only:
# egress listener -> HTTP Connection Manager -> ALB upstream cluster
static_resources:
    listeners:
        - name: egress_http
          traffic_direction: OUTBOUND
          address:
              socket_address:
                  address: <IP_ADDRESS>
                  port_value: 11000
          filter_chains:
              - filters:
                    - name: envoy.http_connection_manager
                      typed_config:
                          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                          codec_type: auto
                          stat_prefix: egress_http
                          normalize_path: true
                          route_config:
                              name: routing_config_to_internal_routing_service
                              virtual_hosts:
                                  - name: internal_routing_service
                                    domains: ["*"]
                                    routes:
                                        - match:
                                              prefix: "/"
                                          route:
                                              cluster: internal-routing-service-secure
                                              timeout: 90s
                                              max_stream_duration:
                                                  max_stream_duration: 0s
                                              retry_policy:
                                                  retry_on: connect-failure,refused-stream,gateway-error,deadline-exceeded,unavailable
                                                  num_retries: 2

    # ALB cluster (called "internal-routing-service-secure")
    clusters:
        - name: internal-routing-service-secure
          connect_timeout: 5s
          upstream_connection_options:
            tcp_keepalive:
                keepalive_time: 100
          circuit_breakers:
            thresholds:
                - max_connections: 10000
                  max_pending_requests: 10000
                  max_requests: 10000
                  max_retries: 5000
          type: strict_dns
          lb_policy: round_robin
          typed_extension_protocol_options:
              envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
                  "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
                  explicit_http_config:
                      http2_protocol_options: {}
          # 'transport_socket' ensures that the request is forwarded as HTTPS
          transport_socket:
              name: envoy.transport_sockets.tap
              typed_config:
                  "@type": type.googleapis.com/envoy.extensions.transport_sockets.tap.v3.Tap
                  common_config:
                    static_config:
                        match_config:
                            any_match: true
                        output_config:
                            sinks:
                                - format: PROTO_BINARY
                                  file_per_tap:
                                    path_prefix: /etc/envoy
                  transport_socket:
                      name: envoy.transport_sockets.tls
                      typed_config:
                          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
          load_assignment:
              cluster_name: internal-routing-service-secure
              endpoints:
                  - lb_endpoints:
                        - endpoint:
                              address:
                                  socket_address:
                                      address: {{ print $internal_routing_host }}
                                      port_value: 443

POSSIBLE related Envoy issue:
#11931
#11369

From the logs it looks like reset was initiated by the ALB. Do you have any diagnostics from the ALB?
Also you may get better luck asking this the envoy-users slack channel.

hey @yanavlasov - I've posted this question in the slack channel as well, thanks for that.

Do you have any diagnostics from the ALB

I passed a packet capture log (PCAP) to Amazon support after reproducing, I'm waiting to hear back from them.  However, I'm wondering if there are any general Envoy-related guidance on managing this type of upstream behavior

Retries

thanks @yanavlasov - based on what I provided in terms of the logs, do you have a reco on which of these error states I should retry on?
https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retry-grpc-on

also, FWIW - i do have this retry_on policy set on the route configuration
                                              retry_policy:
                                                  retry_on: connect-failure,refused-stream,gateway-error,deadline-exceeded,unavailable
                                                  num_retries: 2

....that said, I don't see the x-envoy-retry-on OR x-envoy-retry-grpc-on headers in the debug logs.....I only seem to get something like this:
':status', '200'
'date', 'Wed, 05 May 2021 22:55:06 GMT'
'content-type', 'application/grpc'
'accept-encoding', 'identity,gzip'
'grpc-accept-encoding', 'identity,deflate,gzip'
'server', 'envoy'
'x-envoy-upstream-service-time', '208'

[2021-05-05 22:55:06.182][41][debug][http] [source/common/http/conn_manager_impl.cc:1500] [C1][S15404213778297715938] encoding trailers via codec:
'grpc-message', 'OK'
'grpc-status', '0'

I think you should add reset in your retry policy, since this is the error you are getting.

thanks @yanavlasov - i'm adding both internal,reset to the policy to account for this, and will report back
additionally, i saw the host_selection_retry_max_attempts in the docs; this is supposed to limit retrying on the same host if it is bad.  but in the case of my example (single upstream is an AWS ALB), does this do what I think it does? or is it a noop
retry_on: connect-failure,refused-stream,gateway-error,deadline-exceeded,unavailable,internal,reset
    num_retries: 3
    host_selection_retry_max_attempts: 3
    retry_host_predicate:
        - name: envoy.retry_host_predicates.previous_hosts

@yanavlasov update:
enabling http2 debug logs + reproducing the issue in a hosted instance is showing a 1:1 relationship between the error and these logs:
[2021-05-12 20:09:22.406][42][debug][http2] [source/common/http/http2/codec_impl.cc:964] [C1113] stream closed: 2
[2021-05-12 20:09:22.406][42][debug][http2] [source/common/http/http2/codec_impl.cc:873] [C2762] sent reset code=0

any clue as to what this is telling us?

@yanavlasov is there a better place I can post this to get active discussion on the above?

Sorry for not responding earlier, I was busy with the 1.18.3 security release.
I've looked through the data, thanks for collecting it. Envoy does not retry because reset occurs after it already sent some response data to the client, so it is dead end. It seems that some interaction between ALB and Envoy causes this.
The log that you have pasted indicated that Envoy closed a stream (not clear if this is client or ALB stream) with the internal error which is not super helpful. It seems to only occur when Envoy is having problems sending frames.
Would you be able to collect TCP trace between Envoy and ALB, please? I think this is where the problem might be.
Also @antoniovicente I think was looking at this problem. Adding him as well.

no worries @yanavlasov , I really appreciate you helping me look into / understand the situation.

Would you be able to collect TCP trace between Envoy and ALB, please? I think this is where the problem might be.

I actually performed a packet capture (with tcpdump again) between the Envoy container + the upstream ALB.  The bad news is that AWS ALBs require listening on HTTPS to forward to an http2 target, meaning the packet data (outside of TCP syn/acks) are TLS encrypted [1]:
[1]

I can try to work on getting these decoded somehow, but in the meantime did you mean something else when you say "TCP trace"?

@yanavlasov update:
enabling http2 debug logs + reproducing the issue in a hosted instance is showing a 1:1 relationship between the error and these logs:
[2021-05-12 20:09:22.406][42][debug][http2] [source/common/http/http2/codec_impl.cc:964] [C1113] stream closed: 2
[2021-05-12 20:09:22.406][42][debug][http2] [source/common/http/http2/codec_impl.cc:873] [C2762] sent reset code=0

any clue as to what this is telling us?

"stream closed: 2" is interesting.  The 2 in a RST_STREAM frame refers to an internal error.  Do you happen to know which of the two connections above is the downstream and the downstream connection?  I'm guessing that the upstream connection to ALB is getting back an internal error and that is resulting in the proxy closing the downstream stream.

hey @antoniovicente - the request pattern is:
# every "node" (aside from the ALB) is a docker container

node grpc client =>
Envoy (sidecar A) =>

ALB =>

Envoy (sidecar B) =>
node grpc server

so, these logs/metrics were collected from sidecar A, which is the sidecar for the grpc client (i.e. application that MAKES the request).
In that case, my best understanding is:

upstream = ALB and beyond
downstream = node grpc client

Finally, the very odd thing is - when we print the exception on the client side, this is the full RST_STREAM frame + error flag
RST_STREAM with code 0

which, based on both the http2 spec + the nghttp2 codec docs, is a NO_ERROR

Ok, so in the log above it seems that C2762 is the downstream connection and C1113 is the upstream connection.  ALB seems to be sending a RST_STREAM with error code of 2 (NGHTTP2_INTERNAL_ERROR), which results in envoy sending a NO_ERROR RST_STREAM to the downstream client.

Yes, encrypted TCP dump is not very helpful, I was looking for decrypted one, so we can see HTTP protocol, between Envoy and ALB.
One other possibility is to collect logs at the trace level, which may be impractical if the issue is hard to reproduce. This might give us a better idea what is going between ALB and Envoy.

@parkedwards
Have you solved that eventually?
I'm getting same error when I try to implement my own rate-limiting service on top of gRPC. The call goes like this:
client curl http -> envoy (istio sidecar proxy) -> grcp service
and I get 500 most of the times - like 1 of 10 requests is successful, the rest gets a http 500 and http stream closed: 0 error.

Is there any progress here, I met the same error too

Have the same issue.

I have the same issue as well

Is this related to https://github.com/envoyproxy/envoy/issues/30149 ?

你好，我是刘海伟，QQ 806864128，1310627032，微博 http://weibo.com/1688577641是华南理工09网络工程的学生，你的来信我已收到！请放心！！！


Pin kubernetes-mixin v0.1.0

We should use the released version of kubernetes-mixin. We can always release another kubernetes-mixin v0.1.x if we need anything updated.

This will be even more important once we start merging Kubernetes 1.14 patches to the kubernetes-mixin master branch and thus would break people by updating the project's dependency here.

We lock the exact version in the lock file already, here I would just “pin” against release-0.1, that’ll make updating a bit easier, especially for things that use the mixin as a transitive dependency.

Updated the PR to pin release-0.1. Hope the CI works, as I haven't generate again.

lgtm :+1:


Tile design fixes

Address styling issues with Mosaic Tile component(s)

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 63.261%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1306

Relevant Lines:
1802

💛  - Coveralls


Update how ACCESS and WIDA data is shown in profile page

Part of https://github.com/studentinsights/studentinsights/issues/1990.
Who is this PR for?
educators, particularly HS NEST teachers and HS NGE/10GE teachers
What problem does this PR fix?
The data on ACCESS, WIDA Model and WIDA Screener are merged together, and so subtests from different dates are shown as if they were the same test time.  Also, the numbers don't mean anything to classroom teachers if they aren't familiar with WIDA scoring rubrics.
What does this PR do?
Organizes the scores by composite and subtest, shows the scores on a visual scale, adds in data about when the subtest score is from.  Also links to WIDA interpretation guides.
This PR does not update the import-side code to preserve data about the particular test that was taken, but that would be a good future improvement.
This also aims to not break the profile v2 page, but doesn't spend any time making things look good there, as that's deprecated and not visible to users anymore.
Screenshot (if adding a client-side feature)

Checklists

[x] Author checked latest in IE - Student Profile v3
[x] Author checked latest in IE - Student Profile v2
[x] Author included specs for new code

selfie


os: Binary data written to os.Stdout gets corrupted on Windows 8.1

What version of Go are you using (go version)?

$ go version
go version go1.13.8 linux/amd64

Does this issue reproduce with the latest release?
Yes (namely go version go1.15.3 linux/amd64)
What operating system and processor architecture are you using (go env)?
I'm cross-compiling on Linux/amd64 (see the following go env output), and running on Windows 8.1/amd64:
go env Output
Build environment:
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/twoi/.cache/go-build"
GOENV="/home/twoi/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/twoi/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.13"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.13/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build855756322=/tmp/go-build -gno-record-gcc-switches"

What did you do?
Using
GOOS=windows GOARCH=amd64 go build main.go

I built the following program
package main

import "os"

func main() {
	os.Stdout.Write([]byte{65})
}

Then I ran it on cmd.exe on Windows 8.1 like so:
.\main > file

What did you expect to see?
A file containing the single byte 65 (i.e. 'A')
(On Unix and Windows 10 - running the very same main.exe build - this is indeed what I am seeing)
N.B. Both tested Windows 8.1. and Windows 10 were Japanese language Windows installations.
What did you see instead?
file was 8 bytes in size, containing the following bytes (hex dump):
00000000  ff fe 41 00 0d 00 0a 00                           |..A.....|

Here's the issue on Stackoverflow: https://stackoverflow.com/q/64639265/709537

/cc @alexbrainman

cc @mattn

It turns out my problem description was wrong.
So I get the same results for Windows 8.1. and Windows 10 in cmd.exe, namely the expected 1 byte output. The problem is powershell, which gives the 8 byte output.
My bad - I am closing this and will see what I can do about powershell.


1.21 Crash when joining world (with other mods?)

Crash report
Works fine when testing in IntelliJ, but crashes in my modpack.

This is due to an incompatibility between the newest Nixenaut Version and the newest Your_Time version. This does not occur in a dev environment, as it automatically downloads the correct version.
Working on a patch.

Can you try the artifact produced by with the newest version of Nixienaut: https://github.com/rotgruengelb/your_time/actions/runs/10819490739

Fix will be merged into for 1.21 and later older versions too.


[html5] multiple elements with own mask under one common mask not working.

example: object (thumbnail) has its own mask[s] inside and was added to another object(lets call it folder) with mask applied- inner mask(in thumbnail) will stop working if there is more than one object added into main object(folder). I need to use masks inside my thumbnail for various reasons.
Issue only on openfl 6.5.0
Here is code and last line causes everything.
package;

import openfl.display.Sprite;

class Main extends Sprite
{

	public function new()
	{
		super();
		init();
	}

	function init():Void
	{
		var thumb:Thumb;
		var folder:Sprite = new Sprite();
		var num:Int = 3; // number of thumbs
		for (i in 0 ... num)
		{
			thumb = new Thumb();
			folder.addChild(thumb);
			thumb.x = (i % 4) * 225;  //just arranging as grid
			thumb.y = Std.int(i / 4) * 185; //just arranging as grid
		}

		addChild(folder);

		var _mask:Sprite = new Sprite();
		_mask.graphics.beginFill(0x07A703);
		_mask.graphics.drawRect(0, 0, 600, 150);
		addChild(_mask); // not necessary to add ?
		folder.mask = _mask;

		// adding frame to show mask boundaries
		var frame:Sprite = new Sprite();
		frame.graphics.lineStyle(3, 0xFFFFFF);
		frame.graphics.drawRect(0, 0, 600, 150);
		addChild(frame);
	}
}

class Thumb extends Sprite
{
	public function new()
	{
		super();
		var frame:Sprite;
		var W1:Int = 221;
		var H1:Int = 181;
		frame = new Sprite();
		frame.graphics.beginFill(0x6B9CE4);
		frame.graphics.drawRect( 0, 0, W1, H1);
		frame.graphics.endFill();
		addChild( frame );

		var imageMask:Sprite;
		imageMask = new Sprite();
		imageMask.graphics.beginFill( 0xFFFFFF, 0.3 );
		imageMask.graphics.drawRect( 0, 0, Std.int(W1 - 20), Std.int(H1 - 20));
		imageMask.graphics.endFill();
		addChild( imageMask ); // if i remove this line - other elements wont show up at all

		frame.mask = imageMask; // this line is causing malfunction
	}

}

Thanks!


Get attribute from props or remove it - how to do this?

Hi
Svg example
<svg xmlns="http://www.w3.org/2000/svg" width="44" height="20" fill="none" viewBox="0 0 44 20">...</svg>

I wan't to use it in a list, and I know that height needed to be 30px, and I don't need to manually see for each icon their viewbox (aspect ratio) for the manual calculation for width value.
So, I want for this situations to pass only single prop (width or height).
I searched how to autogenerate opposite prop value, but is no way for this without use some hacky babel configurations
like this one
So, instead, I want to

keep all default svg attributes (width, height) if none of this is not passed from props
remove opposite svg attribute if at least one of is passed from props

My first try to doing this:
{
        issuer: /\.tsx?$/,
        use: [
          {
            loader: '@svgr/webpack',
            options: {
              prettier: false,
              svgo: false,
              expandProps: 'start',
              // dimensions: 'false',
              svgProps: {
                preserveAspectRatio: 'xMidYMid meet',
                // fill: 'currentColor',
                className: `{['c-icon', props.className].filter(Boolean).join(' ')}`,
                width: `{props.width || props.height ? props.width : undefined}`,
                height: `{props.width || props.height ? props.height : undefined}`,
              },
            },
          },
        ],
      },

This one works well when I pass only one prop value, but it also remove width and height if none of it not passed.
Why? Because of fallback value that is undefined.
Anyone tell me if I'm on a right way and how to get the same result as man from issue https://github.com/gregberge/svgr/issues/303 ?
Second question is: Are there any drawbacks or display problems in some browsers if the one of attributes (width or height) is not set?
Thanks

For svg-sprite-loader i made needed behavior with it runtimeGenerator option like this
const SvgSpriteIcon = function SvgSpriteIcon(props = {}) {
  const { viewBox: propViewBox, width: propWidth, height: propHeight, ...restProps } = props;
  const viewBox = propViewBox || symbol.viewBox;
  const vParts = viewBox.split(' ');
  const vW = vParts[2];
  const vH = vParts[3];

  const aspectRatio = React.useMemo(() => Number(vW) / Number(vH), [vW, vH]);

  const width = propWidth || propHeight * aspectRatio || symbol.width || '1em';
  const height = propHeight || propWidth / aspectRatio || symbol.height || '1em';

  return React.createElement(
    'svg',
    {
      viewBox,
      width,
      height,
      ...restProps,
    },
    React.createElement('use', {
      xlinkHref: '#' + symbol.id,
    }),
  );
};

I need to get the same result with svgr.
Thanks!

Actual workaround
https://github.com/gregberge/svgr/issues/303#issuecomment-1030869476


Add ROUGE Metric Card

Add ROUGE metric card.
I've left the 'Values from popular papers' section empty for the time being because I don't know the summarization literature very well and am therefore not sure which paper(s) to pull from (note that the original rouge paper does not seem to present specific values, just correlations with human judgements). Any suggestions on which paper(s) to pull from would be helpful! :)

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


ssh session timeout

Hi All,
My ssh session generally timeout for every ~10 min
Updated below 2 lines of code in my sshd_config file
vi /etc/ssh/sshd_config
ClientAliveInterval 10m          # 10 minutes
ClientAliveCountMax 0            # 0 times
Now the session getting time out for 10 seconds...
Can someone help in fixing this issue
Thanks in Advance.

please read this document Refer Here


Remove version: DominikReichl.KeePass version 2.55.0

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

Close with reason: Package still available.;


External scripts don't work

Describe the bug
Adding an external script to a lesson doesn't work.
Steps to reproduce

Add <script src="landing.js"></script> to content.md of a lesson.
Create landing.js file with console.log(1) in the lesson folder.
Run npm run dev and open the lesson in the browser.
The script is not executed - the corresponding HTTP request returns 404.

Expected behavior
Astro supports scripts. I'd expect the script to work.
Platform

TutorialKit version: github latest.
OS: macOS
Browser: Chrome

P.S.
Adding image.png to the lesson and including it as ![](ball.png) works.
Perhaps, there's an alternative "proper" way to add an external script?

I'm not sure if Astro supports this out-of-the-box (cc. @Nemikolh, ???). But one thing you could do is convert the content.md into content.mdx, import the script contents as raw string and add it to markup:
---
type: lesson
title: Example
---

import landing from "./_files/landing.js?raw"

# Test

<script>
  {landing}
</script>

With Astro's content collection, I would advise against having code that is part of your app inside your content/* folders (the only exception being content/config.ts which is treated specially by Astro).
I find that it's usually nicer to have the following structure:
src
├── utils
│   └── script.ts
├── content
│   ├── config.ts
│   └── tutorial
│       ├── meta.md
│       └── ...
...

Which then let you import your script like this in any content.mdx:
---
type: lesson
title: Example
---

import "@utils/script";

# My lesson

The nice thing is that you can import your script using the same import in any lesson.
If you want to add a ui element, you might find it useful to have a component instead which let's you access and modify the state of the tutorial:
With a src/components/UpdateFileButton.tsx file:
import tutorialStore from 'tutorialkit:store';

interface Props {
  path: string;
  content: string;
}

export function UpdateFileButton({ path, content }: Props) {
  function writeFile() {
    tutorialStore.updateFile(path, content);
  }

  return (
    <button onClick={writeFile}>
      Update file
    </button>
  );
}

You can then use it like this in your lessons:
---
type: lesson
title: Example
---

import { UpdateFileButton } from "@components/UpdateFileButton";

# My lesson

<UpdateFileButton client:load path="/index.js" content="console.log('Hello world!')" />

So it's ok to put images in content, but not scripts? Even if the script is for a particular lesson only?

@iliakan could you describe your use case a bit more? What does the landing.js do? Why does it need to be run outside webcontainer as well?

@AriPerkkio for this particular case I was going to create a landing script to nicely show callouts above the webcontainer and the editor, to introduce the environment in a clear visual style.

So it's ok to put images in content, but not scripts? Even if the script is for a particular lesson only?

I mean you can also have them there but it won't be as nice as having them outside in terms of DX with vite because we exclude those folders from vite's optimizeDeps entry:
https://github.com/stackblitz/tutorialkit/blob/1eb41fd917ceff08fe9484bd07e732cb673a7a1f/packages/astro/src/index.ts#L87-L89
So IIRC, it means that vite won't be scanning those scripts and they might load a bit slower in development.


chore(v2): Define type for markdown right table of contents

Motivation
Add typing for the rightTOC in @docusaurus/types, so that it can be reference by all plugins and themes.
After this diff, the blog plugin's theme API is fully typed, and we can now correctly type the @theme/TOC component. In a future diff, I plan to use the type for the theme API in docs plugin.
Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan
This PR only adds type annotations, so there should be no runtime change.
To test the accuracy of the type, I added types to @docusaurus/mdx-loader which first produces the TOC. Then I added the type annotation to @theme/TOC component. Both can pass the type checker.
Related PRs
(If this PR adds or changes functionality, please take some time to update the docs at https://github.com/facebook/docusaurus, and link to your PR here.)

thanks :)


Unable to load multible Assemblies to Swagger

I have a Solution that holds 3 Azure function Project
One Project (Main) References the other 2 (HR and Inventory). The referenced projects hold functions for parts of the functionality and are used to structure the the logic.
The Main Project uses <FunctionsInDependencies>true</FunctionsInDependencies> so that the Azure Functions defined in the referenced projects are started and found when running the main project.
The Main project also holds swagger and should render all referenced projects into one unified swagger documentation. For this the referenced projects are in different groups.
The first try was to call "builder.AddSwashBuckle" during Startup multible times but it turns out only the last assembly added and the contained functions will be rendered.
After a quick review it turns out that "SwashBuckleStartupConfig" can only hold one assembly and that it got overwritten every time "builder.AddSwashBuckle" was called.
So I made these Quick changes and they seem to work bit I do not know if there are any side effects for other functions that we do not use.
SwashBuckleStartupConfig.cs
using System.Reflection;
namespace AzureFunctions.Extensions.Swashbuckle.SwashBuckle
{
    internal class SwashBuckleStartupConfig
    {
        public Assembly[] Assembies { get; set; }
    }
}

I added an new method to SwashBuckleStartupExtension.cs
public static IFunctionsHostBuilder  AddSwashBuckle(
            this IFunctionsHostBuilder  builder,
            Assembly[] assemblies,
            Action<SwaggerDocOptions> configureDocOptionsAction = null)
{
    var wbBuilder = builder.Services.AddWebJobs(x => { return; });

    wbBuilder.AddExtension<SwashbuckleConfig>()
            .BindOptions<SwaggerDocOptions>()
            .ConfigureOptions<SwaggerDocOptions>((configuration, section, options) =>
            {
                configureDocOptionsAction?.Invoke(options);
            });

    builder.Services.AddSingleton<IModelMetadataProvider>(new EmptyModelMetadataProvider());
    builder.Services.AddSingleton(new SwashBuckleStartupConfig
    {
        Assembies = assemblies
    });

    var formatter = new SystemTextJsonOutputFormatter(new JsonSerializerOptions());
    builder.Services.AddSingleton<IOutputFormatter>(formatter);
    builder.Services.AddSingleton<IApiDescriptionGroupCollectionProvider, FunctionApiDescriptionProvider>();

    return builder;
}

and changed the other methods in this class to generate an Array with the given assembly
Assembies = new []{assembly}

In SwashbuckleConfig.cs I changed the usage to use the first assembly in the array
var binPath = Path.GetDirectoryName(startupConfig.Assembies[0].Location);

since it is used to get the path and all assemblies should be in the same path this should be ok.
Finaly the last change in FunctionApiDescriptionProvider.cs
public FunctionApiDescriptionProvider(
            IOptions<SwaggerDocOptions> functionsOptions,
            SwashBuckleStartupConfig startupConfig,
            IModelMetadataProvider modelMetadataProvider,
            ICompositeMetadataDetailsProvider compositeMetadataDetailsProvider,
            IOutputFormatter outputFormatter,
            IOptions<HttpOptions> httpOptions)
        {
            _swaggerDocOptions = functionsOptions.Value;
            _modelMetadataProvider = modelMetadataProvider;
            _compositeMetadataDetailsProvider = compositeMetadataDetailsProvider;
            _outputFormatter = outputFormatter;

            var apiDescGroups = new Dictionary<string, List<ApiDescription>>();
            foreach (var assembly in startupConfig.Assembies)
            {
	            var methods = assembly.GetTypes()
	                                  .SelectMany(t => t.GetMethods())
	                                  .Where(m => m.GetCustomAttributes(typeof(FunctionNameAttribute), false).Any())
	                                  .ToArray();

	            foreach (var methodInfo in methods)
	            {
		            if (!TryGetHttpTrigger(methodInfo, out var triggerAttribute))
			            continue;

		            var functionAttr = (FunctionNameAttribute) methodInfo.GetCustomAttribute(typeof(FunctionNameAttribute), false);
		            var apiExplorerSettingsAttribute = (ApiExplorerSettingsAttribute) methodInfo.GetCustomAttribute(typeof(ApiExplorerSettingsAttribute), false) ??
		                                               (ApiExplorerSettingsAttribute) methodInfo.DeclaringType.GetCustomAttribute(typeof(ApiExplorerSettingsAttribute), false);

		            var prefix = string.IsNullOrWhiteSpace(httpOptions.Value.RoutePrefix) ? string.Empty : $"{httpOptions.Value.RoutePrefix.TrimEnd('/')}/";

		            string route;

		            if (_swaggerDocOptions.PrependOperationWithRoutePrefix)
		            {
			            var routePart = !string.IsNullOrWhiteSpace(triggerAttribute.Route) ? triggerAttribute.Route : functionAttr.Name;

			            route = $"{prefix}{(routePart)}";
		            }
		            else
		            {
			            route = !string.IsNullOrWhiteSpace(triggerAttribute.Route) ? triggerAttribute.Route : functionAttr.Name;
		            }

		            var routes = new List<(string Route, string RemoveParamName)>();

		            var regex = new Regex("/\\{(?<paramName>\\w+)\\?\\}$");
		            var match = regex.Match(route);

		            var routeParamRemoveRegex = new Regex(":[a-zA-Z]+(\\(.*\\))?");
		            route = routeParamRemoveRegex.Replace(route, "");

		            if (match.Success && match.Captures.Count == 1)
		            {
			            routes.Add((route.Replace(match.Value, "").Replace("//", "/"), match.Groups["paramName"].ToString()));
			            routes.Add((route.Replace(match.Value, match.Value.Replace("?", "")), ""));
		            }
		            else
		            {
			            routes.Add((route, ""));
		            }

		            var verbs = triggerAttribute.Methods ?? new[] {"get", "post", "delete", "head", "patch", "put", "options"};

		            for (var index = 0; index < routes.Count; index++)
		            {
			            var routeTuple = routes[index];
			            var apiName = functionAttr.Name + (index == 0 ? "" : $"-{index}");
			            var items = verbs.Select(verb => CreateDescription(methodInfo,
			                                                               routeTuple.Route,
			                                                               index,
			                                                               functionAttr,
			                                                               apiExplorerSettingsAttribute,
			                                                               verb,
			                                                               triggerAttribute.AuthLevel,
			                                                               routeTuple.RemoveParamName,
			                                                               verbs.Length > 1))
			                             .ToArray();

			            var groupName = (items.FirstOrDefault()?.ActionDescriptor as ControllerActionDescriptor)?.ControllerName ?? apiName;
			            if (!apiDescGroups.ContainsKey(groupName))
			            {
				            apiDescGroups[groupName] = new List<ApiDescription>();
			            }

			            apiDescGroups[groupName].AddRange(items);
		            }
	            }
            }

            ApiDescriptionGroups =
                new ApiDescriptionGroupCollection(
                    new ReadOnlyCollection<ApiDescriptionGroup>(
                        apiDescGroups.Select(kv => new ApiDescriptionGroup(kv.Key, kv.Value)).ToList()
                    ), 1);
        }

It would be great if that after a review especially of SwashbuckleConfig.cs which sould be the only real critical part, this would be added to the library.

Hi, @OmegaToken
Normally, this functionality is added by using some kind of API Management/Gateway.
For instance, you can easily do the same with uploading all your function documents to Azure API Management SAAS.
Another option here is to use Gateway, such as Ocelot or Kong.
Normally, in functions it's not the best way to cross-reference other projects and solutions, due to the increasing size.
So I don't think it will be an optimal way of doing it.
What do you think?

Closed, if you have any related questions, please create a new one, or mention me here


Weird bugs in working with ParameterAggregate

OS: macOS Mojava Beta
Toolchain: Xcode 10/7-24
I found if I:
1). use update method of ParameterAggregate;
2). print some tensor related information in each iteration；
the program will frozen after 1 or 2 times iteration during the execution, or sometime will cause to compilation error  which message is "#tfop invalid: FIXME: cannot lower a Host->TF tensor transfer in a loop header"
Remove print or update fix the problem. (matmul or •  works fine with print )

sometimes print(loss) works fine but print("\(loss)") will trigger compilation error above.

Test code:
struct Weights : ParameterAggregate {
    var w = Tensor<Float>([[1,2],[3,4]])
}

func train_test(_ x : inout Weights)
{
    for i in 0...3
    {
        let grad = Weights()
        x.update(withGradients : grad){ (p,g) in
            p = g * 0.0001
        }
        print(i)
        let result = x.w.mean()
        print(result)
    }
}

var weight = Weights()
train_test(&weight)

Sorry, maybe I mistakenly removed the key code which can reproduce the error, after simplify the code.
My swift version:
Swift version 4.2-dev (LLVM 748f9878c8, Clang 6c624d9bad, Swift 15f32c49ee)
Target: x86_64-apple-darwin18.0.0
The following code can reproduce the error (added a tensor parameter to train function)
import TensorFlow
struct Weights : ParameterAggregate {
    var w = Tensor<Float>([[1,2],[3,4]])
}

func train_test(_ x : inout Weights, _ temp : Tensor<Float>)
{
    for i in 0...3
    {
        let grad = Weights()
        x.update(withGradients : grad){ (p,g) in
            p = g * 0.0001
        }
        print(i)
        let result = x.w.mean()
        print(result)
    }
}

var weight = Weights()
train_test(&weight, Tensor<Float>(randomNormal:[2,2]))

Thanks, I'm able to reproduce the error on the 7/24 toolchain now.
This bug has been fixed upstream (tested with https://github.com/apple/swift/commit/6da8496ed830fe8bc07f1916960529562fdc4d03): the program terminates without hanging:
0
0.00024999998
1
0.00024999998
2
0.00024999998
3
0.00024999998

I'll leave this bug open until a new toolchain is released.

New toolchains have been released.
https://github.com/tensorflow/swift/blob/master/Installation.md


Work out why settings don't persist

At present, whenever the ESP8266 is reset/powered off, it forgets its WiFi and LED settings (and presumably anything else).
That's not too useful.  It's probably something that I'm missing in how I program the board.

Seemed to be a problem with the filesystem size.  Programming the board with the "Generic ESP8266 Module" settings rather than Node MCU 1.0 seems to fix things.
Failure to write the EEPROM is what generates the "WLED Error 2" messages that were showing up in the web UI.


chore: bdd-spring-1572876038 to 0.0.1

chore: Promote bdd-spring-1572876038 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


[Feature request] support for Rmarkdown files

Markdown is extended by pandoc and RStudio's Rmarkdown, whose extension is .Rmd. I wonder how these files can count words automatically.

@psychelzh Sorry for the late response. The newest version could be activated by .editorconfig file if it exists in the project folder. You can try it.

@holmescn Thanks very much. It is of huge improvement now!


pbix files for tutorial and "demo" don't open message says they are corrupt

I am trying to share/show and tell with the demo files but PBI Desktop won't open saying they are corrupt.  This file location is the link from abovehttps://github.com/Microsoft/powerbi-desktop-samples/blob/master/2019/customerfeedback.pbix

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4200dd45-15ab-ee4a-9818-10cc81475f64
Version Independent ID: 0b96ac28-e2a2-41c9-eff0-efd864fc0529
Content: Key influencers visualizations tutorial - Power BI
Content Source: powerbi-docs/visuals/power-bi-visualization-influencers.md
Service: powerbi
GitHub Login: @mihart
Microsoft Alias: mihart

Please close.  Working from Github repo just fine.

@clarkmac, Thanks for your comment.
We’ve forwarded your feedback to the writer, who should respond to this thread.
Thanks!

Thanks @clarkmac, closing now.