Add arguments to quilkin run

Closes #555, closes #572
This adds the basic options to run quilkin as a reverse proxy, allowing you to set the listen port and the destination address. Additionally I added some admin related options for setting admin port and disabling admin entirely.

Can we get some docs, probably here https://googleforgames.github.io/quilkin/main/book/using.html please?
Re-running CI to see if that's a flaky test error or not.

It's not, I have the fix locally. The change is that the admin spawning is pulled out of the proxy to let the admin logic be shared amongst all commands, and these are tests that are testing the admin server.

@markmandel Rather than duplicating documentation, I figured it was easiest and best to expose the CLI in the rust documentation and just link to that. That way we're maintaining a single source of truth as our documentation now matchces what the CLI outputs in help.
I'm having trouble figuring out what these Agones tests want me to do, do you have any idea?

@markmandel Rather than duplicating documentation, I figured it was easiest and best to expose the CLI in the rust documentation and just link to that. That way we're maintaining a single source of truth as our documentation now matches what the CLI outputs in help.

Makes sense.

I'm having trouble figuring out what these Agones tests want me to do, do you have any idea?

I'm at an event at the moment, but will take a look as soon as I can 👍🏻

Looks like a clippy failure:
error[E0308]: mismatched types
  --> src/cli/run.rs:56:61
   |
56 |                 clusters.default_cluster_mut().localities = vec![self.to.clone().into()];
   |                 -----------------------------------------   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected struct `cluster::LocalitySet`, found struct `std::vec::Vec`
   |                 |
   |                 expected due to the type of this binding
   |
   = note: expected struct `cluster::LocalitySet`
              found struct `std::vec::Vec<_>`
   = note: this error originates in the macro `vec` (in Nightly builds, run with -Z macro-backtrace for more info)

For more information about this error, try `rustc --explain E0308`.

Ah, on my mobile, so difficult to see past build output.
Have you been able to replicate the test failure by running the integration test locally? Either directly, or through the Makefile?
Then you could introspect the Pods and see what is failing.
If not, I'll be able to take the PR for a spin on Friday and see what is failing.

Have you been able to replicate the test failure by running the integration test locally? Either directly, or through the Makefile?

I'm unable to use the Makefile, it just gets killed. I haven't tried directly yet.
#11 147.3 <jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
#11 147.3 <jemalloc>: (This is the expected behaviour if you are running under QEMU)
#11 147.4 <jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
#11 147.4 <jemalloc>: (This is the expected behaviour if you are running under QEMU)
#11 147.4 <jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
#11 147.4 <jemalloc>: (This is the expected behaviour if you are running under QEMU)
#11 147.5 <jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
#11 147.5 <jemalloc>: (This is the expected behaviour if you are running under QEMU)
#11 149.5    Compiling quote v1.0.21
#11 150.0 <jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
#11 150.0 <jemalloc>: (This is the expected behaviour if you are running under QEMU)
#11 152.6 Killed
------
executor failed running [/bin/sh -c wget --quiet https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init &&     chmod +x rustup-init &&     ./rustup-init -y --no-modify-path --default-toolchain $RUST_TOOLCHAIN &&     rm rustup-init &&     chmod -R a+w $RUSTUP_HOME $CARGO_HOME &&     rustup component add rustfmt clippy &&     rustup target add x86_64-pc-windows-gnu &&     cargo install cargo-watch mdbook &&     cargo install cargo-about &&     cargo install --locked cargo-deny &&     rustup --version &&     cargo --version &&     rustc --version]: exit code: 137
make: *** [ensure-build-image] Error 1

🤔 are you on a M1 mac?

Aaah, that old chestnut. Trying to install the x86 version of rust on a aarch64 chip is going to fall over every time.
We should put some OS checks in the makefile to manage that. I don' t have access to a M1 mac, so will be tricky for me to test, but I can steal some stuff from the Agones project, because we do that there.

I'm not too familiar, but when I was reading through it, all that should be required is having a step that runs uname -m and then matches on the string (because uname -m returns arm64 while it needs to be aarch64)

That sound exactly right 👍🏻, then we need to pass that through as a Docker arg, and use that in appropriate places in the build Docker file.
I'm sure other fun bits will come up, but that is the general gist.

The other option, as arch issues can get tricky fast, is to force Docker to Linux amd64 for all things:
export DOCKER_DEFAULT_PLATFORM=linux/amd64
Might actually be the easiest option (we could also bake this into the Makefile in some capacity).

TIL, I’ll try it with that tomorrow

Just tried to take it for a spin, but got stuck at the compilation issue I referenced above 😢
I was going to note as well, if you have a local kubectl config that has access to an Agones cluster (although if you have occasionally access issues, try running a kubectl command to refresh the token), you should be able to run the e2e test directly, of the IMAGE_TAG env var is set.

Decided to just to revert removing the quilkin.yaml and changing the entrypoint for now, as I'd like to get these changes in and we can make those changes separately.

The other option, as arch issues can get tricky fast, is to force Docker to Linux amd64 for all things:

This doesn't work because the build-image is hardcoded to use x86_64 in multiple places, so the build-image needs to be reworked before it's usable on ARM machines.

Decided to just to revert removing the quilkin.yaml and changing the entrypoint for now, as I'd like to get these changes in and we can make those changes separately.

That makes sense 👍🏻

The other option, as arch issues can get tricky fast, is to force Docker to Linux amd64 for all things:

This doesn't work because the build-image is hardcoded to use x86_64 in multiple places, so the build-image needs to be reworked before it's usable on ARM machines.

Huh. That's strange, because if you set DOCKER_DEFAULT_PLATFORM is affects both build and run operations, so the image itself will be x86, so having it install x86 binaries is what should work in that environment. 🤔 Prune the local image cache maybe, if there are old versions lying around?
I almost wish I had a ARM machine so I could test this out. If you are keen, could also try a GVC sometime. It's entirely possible I'm missing something, since i'm shooting in the dark a bit.
In the meantime, I'll take on finding what has broken in the integration tests for this PR, and update you with my results (probably send a PR to your PR.)
Also I'll prioritise #593 such that you can use the same image from CI to locally test an integration test.

Digging into what's failing in the integration tests - not sure what's causing it, but looks like there is something trying to bind to 9091 more than once. I'm seeing this in a few of the tests.
This is in agones management mode:
─➤  kubectl logs -n<PHONE_NUMBER> quilkin-manage-agones-5fb9bf9df7-6rqm9
{"timestamp":"2022-09-20T02:09:39.317682Z","level":"INFO","fields":{"message":"Starting Quilkin","version":"0.4.0-dev","commit":"f45d971388ce356229cc922c36376f267d1e6d40"},"target":"quilkin::cli"}
{"timestamp":"2022-09-20T02:09:39.317882Z","level":"DEBUG","fields":{"message":"provided path not found","path":"quilkin.yaml"},"target":"quilkin::cli"}
{"timestamp":"2022-09-20T02:09:39.318230Z","level":"INFO","fields":{"message":"Starting admin endpoint","address":"<IP_ADDRESS>:9091"},"target":"quilkin::admin"}
{"timestamp":"2022-09-20T02:09:39.318298Z","level":"TRACE","fields":{"message":"registering event source with poller: token=Token(1), interests=READABLE | WRITABLE","log.target":"mio::poll","log.module_path":"mio::poll","log.file":"/usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/mio-0.8.4/src/poll.rs","log.line":521},"target":"mio::poll"}
{"timestamp":"2022-09-20T02:09:39.319482Z","level":"INFO","fields":{"message":"Starting admin endpoint","address":"<IP_ADDRESS>:9091"},"target":"quilkin::admin","span":{"name":"spawn"},"spans":[{"name":"spawn"}]}
thread 'main' panicked at 'error binding to <IP_ADDRESS>:9091: error creating server listener: Address already in use (os error 98)', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.20/src/server/server.rs:77:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
{"timestamp":"2022-09-20T02:09:39.319629Z","level":"ERROR","fields":{"message":"Panic has occurred. Moving to Unhealthy","panic_info":"panicked at 'error binding to <IP_ADDRESS>:9091: error creating server listener: Address already in use (os error 98)', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.20/src/server/server.rs:77:13"},"target":"quilkin::admin::health","span":{"name":"spawn"},"spans":[{"name":"spawn"}]}
{"timestamp":"2022-09-20T02:09:39.319646Z","level":"ERROR","fields":{"message":"Panic has occurred. Moving to Unhealthy","panic_info":"panicked at 'error binding to <IP_ADDRESS>:9091: error creating server listener: Address already in use (os error 98)', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.20/src/server/server.rs:77:13"},"target":"quilkin::admin::health","span":{"name":"spawn"},"spans":[{"name":"spawn"}]}
{"timestamp":"2022-09-20T02:09:39.319634Z","level":"TRACE","fields":{"message":"no local config found, falling back to local in-cluster config","error":"failed to read kubeconfig from '\"/home/nonroot/.kube/config\"': No such file or directory (os error 2)"},"target":"kube_client::config"}
{"timestamp":"2022-09-20T02:09:39.320006Z","level":"TRACE","fields":{"message":"deregistering event source from poller","log.target":"mio::poll","log.module_path":"mio::poll","log.file":"/usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/mio-0.8.4/src/poll.rs","log.line":652},"target":"mio::poll"}
{"timestamp":"2022-09-20T02:09:39.320187Z","level":"DEBUG","fields":{"message":"buffer closing; waking pending tasks"},"target":"tower::buffer::worker"}
{"timestamp":"2022-09-20T02:09:39.320399Z","level":"TRACE","fields":{"message":"deregistering event source from poller","log.target":"mio::poll","log.module_path":"mio::poll","log.file":"/usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/mio-0.8.4/src/poll.rs","log.line":652},"target":"mio::poll"}

(Side note - looks like the check in create_quilkin_pod shouldn't just check if it's running, because the pod is in crashloop backoff in the integration test).
Looks like all the tests are failing for this reason.

Aha! I can replicate. Running a local binary with a config file reproduces the error:
➜  agones-xonotic git:(pr/ep/run-args) ✗ ../../target/build-image/x86_64-unknown-linux-gnu/release/quilkin -c ./client-compress.yaml run
{"timestamp":"2022-09-20T02:26:50.508501Z","level":"INFO","fields":{"message":"Starting Quilkin","version":"0.4.0-dev","commit":"f45d971388ce356229cc922c36376f267d1e6d40"},"target":"quilkin::cli"}
{"timestamp":"2022-09-20T02:26:50.508851Z","level":"INFO","fields":{"message":"Starting admin endpoint","address":"<IP_ADDRESS>:9091"},"target":"quilkin::admin"}
{"timestamp":"2022-09-20T02:26:50.508956Z","level":"INFO","fields":{"message":"Starting","port":7000,"proxy_id":"markmandel"},"target":"quilkin::proxy"}
{"timestamp":"2022-09-20T02:26:50.508973Z","level":"INFO","fields":{"message":"Starting admin endpoint","address":"<IP_ADDRESS>:9091"},"target":"quilkin::admin"}
{"timestamp":"2022-09-20T02:26:50.509022Z","level":"ERROR","fields":{"message":"Panic has occurred. Moving to Unhealthy","panic_info":"panicked at 'error binding to <IP_ADDRESS>:9091: error creating server listener: Address already in use (os error 98)', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.20/src/server/server.rs:77:13"},"target":"quilkin::admin::health"}
{"timestamp":"2022-09-20T02:26:50.509039Z","level":"ERROR","fields":{"message":"Panic has occurred. Moving to Unhealthy","panic_info":"panicked at 'error binding to <IP_ADDRESS>:9091: error creating server listener: Address already in use (os error 98)', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.20/src/server/server.rs:77:13"},"target":"quilkin::admin::health"}
thread 'main' panicked at 'error binding to <IP_ADDRESS>:9091: error creating server listener: Address already in use (os error 98)', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.20/src/server/server.rs:77:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtr

Looks like it's trying to recreate the admin server twice.

Re-running, so that the new CI failure messages can give us a build image to debug against.


:recycle: chore: Migrate resolutions to Effect

Description
Migrate resolutions to using effect.ts
Testing
Explain the quality checks that have been done on the code changes
Additional Information

[ ] I read the contributing docs (if this is your first contribution)

Your ENS/address:

Current dependencies on/for this PR:

main

PR #615   👈

This comment was auto-generated by Graphite.


[Niconico] Merging niconico extractor from animelover1984/youtube-dl

Since yt-dlp no longer supports python2, is it possible to merge https://github.com/animelover1984/youtube-dl 's niconico live implementation now? I think your original reason for not wanting to was python2 support.

I think your original reason for not wanting to was python2 support.

Yes, that was part of the reason. But even with py3.6 being the minimum version, there are a few issues

The danmaku to subs code is under GPL3 and thus cannot be copy-pasted. It can perhaps be added as an optional dependancy
There must be no required dependancies, only optional ones. This can be achieved by wrapping the import in try...except and checking for its availability before use.
A lot of the niconico extractor was modified in yt-dlp independently (See #49). We will need to compare and fix conflicts for these
There seems to be a lot of obsolete code (like the rtmp download)

I am not familiar enough with the extractor to fix these issues myself

So the remaining is conversion and Niconico Live?

Yeah
Idk if there is any point in adding a dependency for danmaku conversion. Someone can easily implement https://github.com/m13253/danmaku2ass as a custom PP (or even just use it with --exec)
The main thing remaining is nico live


Errors after session expiry

If while searching/listing entities the session expires due to inactivity a request to http://localhost:8080/login is made according to Chrome Dev Tools but this page is not displayed (only seen in the Network tab), Console shows...
gnOperator.service.ts:82 JSON content could not be parsed
    GnOperatorService.handleError @ gnOperator.service.ts:82
    CatchSubscriber.error @ catch.ts:55
    MapSubscriber._next @ map.ts:77
    Subscriber.next @ Subscriber.ts:95
    onLoad @ http.umd.js:1168
    ZoneDelegate.invokeTask @ zone.js:275
    onInvokeTask @ core.umd.js:4382
    ZoneDelegate.invokeTask @ zone.js:274
    Zone.runTask @ zone.js:151
    ZoneTask.invoke @ zone.js:345

message.service.ts:25 ERROR: Could not get the results DETAIL: JSON content could not be parsed

The website is still on e.g. http://localhost:8080/gnOperator-list and to continue I have to explicitly log out and back in.
Might this be due in some way to my earlier report on ts compilation problems?

Fixed, could you please test it again?

Yes, looks good now - thanks.


Export current plugin name and Configuration

Experimental - see https://forum.buildkite.community/t/native-support-for-golang-plugins/776/5
tl;dr:
Exposes two new environment variables to plugins:

BUILDKITE_PLUGIN_NAME - The "env key compatible" version of the current plugin name - that's what's being used to construct the BUILDKITE_PLUGIN_<name>_<param> variables
BUILDKITE_PLUGIN_CONFIGURATION - JSON string holding the current plugin's configuration (as opposed to all the plugin configs as found in BUILDKITE_PLUGINS

This PR is not tackling the underlying question in the community post regarding native Golang plugins as first class citizens.
In a multi-OS build environment, I would imagine a Golang plugin to be wrapped in something like this:
# file: pre-command
function get_ostype() {
  if [[ "$OSTYPE" == "linux"* ]]; then
    echo "linux"
  elif [[ "$OSTYPE" == "darwin"* ]]; then
    echo "darwin"
  else
    >&2 echo -e "${RED}ERROR: Cannot determine os type from '$OSTYPE'${NC}"
    return 1
  fi
}

DIR="$(cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd)"

# call <hook>-ostype binary and pass requisite values
BUILDKITE_PLUGIN_CONFIGURATION=$BUILDKITE_PLUGIN_CONFIGURATION BUILDKITE_PLUGIN_NAME=$BUILDKITE_PLUGIN_NAME ./pre-command-$(get_ostype)

I'm just copying a useful bit of info from the forum reply into here:

The reason we need the current plugin’s configuration in BUILDKITE_PLUGIN_CONFIGURATION_<name> and cannot just rely on parsing the full BUILDKITE_PLUGINS json is that within the context of the plugin, I don’t actually know which entry in the plugin list is the appropriate one to look at, especially if the same plugin is called more than once in the same step (which is technically possible today I think).

Thanks @moensch, we like this idea and are keen to hear how it goes for you in the real world :sparkles:
One caveat to be aware of is that plugins implemented in bash also have magic that ensures exported environment variables in the plugin become available in subsequent hooks and the command. This magic obviously won't apply within the golang binary, so if your plugin needs to influence the environment for later hooks or the command you'll need a way to pass them back to the bash shim and re-export them.

Oh  yer, I should note that I built a version of the agent from this branch and ran a silly debug plugin to double-check it works the way it said it would on the tin :+1:


Unable to create link that works for kURL and existing clusters

I've defined application resources as
apiVersion: kots.io/v1beta1
kind: Application
metadata:
  name: myapp
spec:
  title: MyApp
  ports:
  - serviceName: myapp
    servicePort: 8080
    localPort: 8080
    applicationUrl: http://myapp
---
apiVersion: app.k8s.io/v1beta1
kind: Application
metadata:
  name: myapp
spec:
  descriptor:
    links:
    - description: MyApp
      url: http://myapp

This link (sometimes) works when I run kubectl kots admin-console, but I don't see any way to use it with kURL. Having broken links there is worse than no links, as users will try to follow them and assume something's broken rather than look at the next step in install documentation.
I'd like to be able to provide links for

the ingresses my application has configured
when connected via kots admin-console (i.e. over a forwarded port), also add the current port-forwarded link

Templating seems to get partway towards this goal.

Internal replicated issues: https://app.clubhouse.io/replicated/story/32482/unable-to-create-link-that-works-for-kurl-and-existing-clusters


wrong id generated by yii\grid\ActionColumn

hi,
doc has a example which shows how to use ActiveDataProvider without ActiveRecord.
but generated gridview action url has wrong id started with 0 when query is created by query builder. everything is ok when query passed to dataproviader with ActiveRecord.
i guess problem is with $key (key of model) in yii\grid\ActionColumn which is not defined in query builder.
there is another report on forum which i found.

you have to configure the key property if you need a key column: http://www.yiiframework.com/doc-2.0/yii-data-activedataprovider.html#$key-detail

I have 2 related table, event and participant
Event Model:
    public function getParticipants() {
        return $this->hasMany(Participant::className(), ['event_id' => 'id']);
    }

EventController:
    public function actionView($id) {
        $model = $this->findModel($id);
        $participants = new ArrayDataProvider([
            'allModels' => $model->participants,
            'pagination' => false,
        ]);
        return $this->render('view', ['model' => $model, 'participants' => $participants]);
    }

views/event/view.php:
<?= $this->renderAjax('/participant/index', [
        'dataProvider' => $participants,
        'searchModel' => false
    ]) ?>

view/participant/index.php generated by yii
The Question is:

Why renderAjax gridview action url has wrong id? There's a workaround (adding a bit of code to views/participant/index.php, ActionColumn template and buttons. But can we have it automated? When we call ActiveDataProvider or ArrayDataProvider, it will check and get the ActiveRecord automatically.

Thanks!


Testnet faucet link not working

The testnet faucet link mentioned in Readme is not working https://testnet-faucet.mempool.co/. Do we have an alternative to this faucet or a new link? If there is one we should update it.

Yep let's update those. Here are two new ones:

https://coinfaucet.eu/en/btc-testnet/
https://bitcoinfaucet.uo1.net/

We can close this now


After activation, I don't see anything in logs

I've followed https://github.com/kercre123/wire-pod/wiki/Installation to the letter with a production Vector on a Linux machine.
I've got the "Vector setup is complete!"

However, the logs page doesn't say anything

Even after I talk with Vector.
I also noticed that the dropdown for the configuration page is blank.

With the http://escapepod.local:8080/api-sdk/get_sdk_info returning:

I also noticed my OpenAI key says it's never been used.
This leads me to believe that the setup didn't work? Is it somehow still defaulting to using DDL servers?
How can I verify this?

I found the issue. When I setup initially, I didn't put Vector in recovery mode properly. Once doing that, works fine.


Adding support for ecdsa ssh key types

This PR adds support for ecdsa ssh key types, along with tests for the same.
Checklist

[X] Affected Issues have been mentioned in the Closing issues section
[ ] Documentation has been written/updated.
[X] Changelog entry has been written

Changelog Entry
Improvement - Add ecdsa ssh key support (#217)
Closing issues
Closes #217

@shreddedbacon Now that we've released Lagoon version 2.7.1 which supports these additional key types, can we get this PR merged in so we'll have support for those key types via the CLI as well? :)

Just built and used against a 2.7.1 lagoon and 🎉  will merge


Add nodes to CreateNewConfigFile

Users report not being able to get connections after a fresh install with no config.
When using the new user wizard it adds 7 nodes to the config however if ran the other route and not using the wizard then only 3 are added via CreateNewConfigFile. This will give more nodes in the initial config to allow users to connect as node.gridcoin.us grcmagnitude.com and gridcoin.asia nodes are down.
shout out to @barton2526 for this

aparently these ndoes might be gone as well


[Bug]: The source table or target table is deleted during the archiving task, but the task does not stop.

ODC version
ODC424
OB version
independent
What happened?
The source table or target table is deleted during the archiving task, but the task does not stop.

What did you expect to happen?
。。
How can we reproduce it (as minimally and precisely as possible)?
。。
Anything else we need to know?
No response
Cloud
No response

task log suspended on 6/10, time from 2024-03-05 21:45:33 to 2024-03-05 21:55:33, and still be suspend

expect: when target table deleted,  dml final status is failed @guowl3


Throwing string in coverage check, results in strange confusing error print-out

In lib/cli.js, there's this function:
function exit(ex, code) {
  // flush output for Node.js Windows pipe bug
  // https://github.com/joyent/node/issues/6247 is just one bug example
  // https://github.com/visionmedia/mocha/issues/333 has a good discussion
  var streams = [process.stdout, process.stderr];
  async.forEach(streams, function (stream, done) {
    // submit a write request and wait until it's written
    stream.write('', done);
  }, function () {
    if (ex) {
        throw ex; // turn it into an uncaught exception
    } else {
        exitProcess(code);
    }
  });
}

That throw ex; usually ends up getting a string, and so the output looks like this:
/Users/isaacs/dev/js/tap/node_modules/nyc/node_modules/istanbul/lib/cli.js:39
        throw ex; // turn it into an uncaught exception
        ^
ERROR: Coverage for lines (42.23%) does not meet global threshold (100%)

The path to lib/cli.js, and the turn it into an uncaught exception comment, is confusing to users, and makes them think that something is broken (rather than the program running correctly, and telling them that there is a problem that they need to fix).
I think if it's a string, and not a bug in the program, simply outputting the string to stderr and then exiting with status code 1 is the right thing to do.
Happy to send a PR to do this if you all agree.
diff --git a/lib/cli.js b/lib/cli.js
index 19b3b27..5a8f1bf 100755
--- a/lib/cli.js
+++ b/lib/cli.js
@@ -35,7 +35,12 @@ function exit(ex, code) {
     stream.write('', done);
   }, function () {
     if (ex) {
-        throw ex; // turn it into an uncaught exception
+        if (typeof ex === 'string') {
+            console.error(ex);
+            exitProcess(1);
+        } else {
+            throw ex; // turn it into an uncaught exception
+        }
     } else {
         exitProcess(code);
     }

Sounds good.

@gotwarlost #519 awaits your attention :)  Thanks.


Handle omnicorp 500

In principle a 500 from omnicorp is recoverable.   The scoring won't be as good (we assert) but there will at least be scores.  Currently though, a 500 from any subservice causes aragorn/robokop to return immediately (with a 200 in this case since it does return results, even if they are unscored).
Modify this so that if omnicorp fails, we put an error in the logs, and continue on to weight / scoring.

fixed in 2.3.10a


Implement FSM Driver Logic

Implemented Four Block Structure & Driver Logic but simulation seems to be caught in endless loop and isn't producing any results now. @Alexanderallenbrown I just uploaded it to week #5 updates branch. Can you take a look?

@PostMatthew which file is this in? Notebook or all-in-one script?


[bug] Edit project: view focus

Hi @randy3k , awesome plugin!
When trying to navigate to the .sublime-project file using Project Manager -> Edit Project
the .sublime-project is opened without focused.
I tried calling focus_view with the view returned by window.open_file in edit_project but seems to have no effect.
I'm on latest ST3 and OSX yosemite.

I am still having the issue. Is there any way to check to see if the fix has been pulled in? Package Control says that there isn't any available upgrades for any packages.

May be, your problem is related to #2? If that is the case, I am sorry to tell you that it is still not fixed yet.


Added MEG user documentation

Codecov Report

Merging #40 (4c3db3f) into main (c6cd979) will decrease coverage by 1.47%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main      #40      +/-   ##
==========================================
- Coverage   82.95%   81.47%   -1.48%
==========================================
  Files          13       13
  Lines        2962     2677     -285
==========================================
- Hits         2457     2181     -276
+ Misses        505      496       -9

Impacted Files
Coverage Δ

ancpbids/model_v1_7_1.py
81.18% <0.00%> (-3.13%)
:arrow_down:

ancpbids/model_v1_7_0.py
90.00% <0.00%> (-1.64%)
:arrow_down:

ancpbids/_version.py
39.46% <0.00%> (+2.67%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c6cd979...4c3db3f. Read the comment docs.


Fix multiGet's value for unknown keys to null

Similar to #18, multiGet should return null for unknown keys.

✨ Thank you 😄 I love it :) Tonight it will be released. After work :)


gollm fixes and improvements

Description

changes to the taskhanlder now that it is using a new event source library
bugfix to gollm controller
model config improvements

Let's wait until this ASYNC/async parameter is resolve.

I was wrong, my update didn't fix the enum casings. Lets merge this we can fix the casing stuff another day


feat(server): add a --replace option

Closes #529 which seemed to be a common complaint among teachers.

learn-ocaml serve --replace will kill an existing server (running on the
same port) just before starting

learn-ocaml build serve with an existing server on the same port will fail
fast (before actually doing the build)

learn-ocaml build serve --replace is more clever:

it will do the build in a temporary directory
then, only if everything is ok, kill the older server
swap the files and start the new server

This is all done in order to minimise downtime and be convenient for server
updates.
Note that this PR sits on top of #481 and should be rebased once it's
merged.

Thanks @AltGr, this enhancement is a must-have !!
I didn't finish my tests yet, but as I told you by e-mail I expect this to be done on this Monday Nov 20.


Derive 'node_number' field from MAC addresses

The node_number was previously derived from a node's management IP.
In a routed rack configuration, this could cause the node_number to
wrap around and overlap inside a single cluster.
This commit derives the node_number from the MAC address on the
management interface.  The MAC is converted to an integer.  The node
number is this integer modulo Integer.MAX_VALUE.
This gives us a ~31 bit positive integer that is guaranteed to be
unique inside any single-vendor customer.  It is very likely to be
unique across any two NICs, but not certainly so, since only the
last 7 bits of a 24 bit OUI are present.
Zookeeper and keepalived take use (node_number % 8) to derive an 8 bit
value from the 31 bit node number.  In a sufficiently large cluster
this may overlap.

This should not be a breaking change, since the current node_number in code has precedence "default" while the old node_number implementation used "set".
Existing clusters will have an already-existing node number in the node definition.

The numbering scheme is also used to assign IDs to resource manager hosts in /etc/hadoop/conf/yarn-site.xml and /etc/hadoop/conf/hdfs-site.xml, for uniformity's sake you may want to suggest we make a change there as well

How do you see this number wrapping around?

If we still agree to go ahead with this, perhaps to ease the transition a switch can be used to control how the node number is generated, this way a massive reconfiguration should not be a problem.  However it would be interesting to see if this can be handled gracefully.  I suspect neither hdfs nor yarn would be problematic.

BREAKING: This may require some coordination since the node numbers will change in an existing cluster. The following technologies use this:

Zookeeper (should not matter if different across different nodes)
YARN RM (only the node number is in the Zookeeper znode for the active RM)
HDFS Namenode (the node number and hostname is in the Zookeeper znode for the active NN)

One will need to be cautious about rolling out the change. I suspect the following will be needed:
Start with all heads except the active RM and NN (ensure at least one in-active RM and NN)
Then, Chef all clients to provide the mapping for the recently updated nodes
Next, ensure the previously chef'd in-active RM and NN are made active
Chef the previously-active RM and NN's
Then, re-Chef all clients to ensure consistent mappings

I have reduced the precedence on the "new" node number attribute to default.
Existing clusters all have node numbers hard-coded in the node object with "node.set".  If we want to migrate those clusters, we can simply remove the override from the node object.
New clusters and hosts who have had their node objects cleared will derive node numbers and ZK IDs from the MAC address.

For reference the VRRP Virtual Router ID is 8 bit per section 5 of RFC 3768 can you also truncate the keepalived router ID to 8 bits?

Clay, the router ID is generated with "rand(1000)," it doesn't use the node_number.
The VRRP router priorities are determined by node_number, but they are already mod 255.

Wow; that rand(1000) craziness needs to stop; please use the 8-bit node_number. Also, please raise if you find another node has the same 8-bit node_number. This new 8-bit node_number will be more opaque to an operator should it collide now than when it was an IP address last octet or offset.

Andrew are you planning to squash commits into node_number changes and syntax fix-ups or did you want to leave this in its current incarnation?
If you do plan to leave as is, I am confused about this comment:
Zookeeper and keepalived take use (node_number % 8) to derive an 8 bit
value from the 31 bit node number.  In a sufficiently large cluster
this may overlap.

In testing, this seems to work so far for adding to an existing cluster. I ran the following for validation:

before running new code

for i in 1 2 3; do EDITOR=cat knife node edit -a bcpc-vm${i}.bcpc.example.com > /tmp/pre-bcpc-vm${i}.bcpc.example.com.json; done

updated and ran new code, via

git pull <new code URL>
knife cookbook upload -a
./cluster-assign-roles.sh Test-Laptop hadoop

then re-dumped the node objects with for i in 1 2 3; do EDITOR=cat knife node edit -a bcpc-vm${i}.bcpc.example.com > /tmp/pre-bcpc-vm${i}.bcpc.example.com.json; done
finally differed the pre- and post- node objects, ensuring node[:bcpc][:node_number] was the same

To test this with nodes that did not have a node_number, I cleared the attribute via a knife node edit <fqdn> command. However, this was not repopulated via a chef-client -o 'role[Basic]' while a normal Chef run of any of the head node roles would fail since there was a node object without a node number.
While running chef-shell -z I could puts node[:bcpc][:node_number] and see it was being set correctly but since I was running chef-client with the -o option the node object was not being written back to the server. (When running a full Chef run to failure it did update the node with the node_number.)

For reference, Andrew has asserted he was able to build a test cluster from scratch with this addressing my concerns, so I verbally gave my accent to a self-merge for this PR.


Fix: Elasticsearch settings in docker-compose

This PR aims to fix some Elasticsearch container settings to avoid the error of memory allocation in elasticsearch, and remove redundant file.
Type of change

[x ] Bug fix (non-breaking change which fixes an issue)

Modified files

docker-compose.yaml: Changed the image of Argilla to our own newly deployed image, increased the cluster.max_shards_per_node to 10000, changed the ES_JAVA_OPTS to 5GB to avoid memory insufficient error
demo_task_with_context.ipynb: deleted this redundant file. It is unintentionally added in a PR and it's not needed so I removed it.

Hello, 1GB could be fine but I reduced it to 2GB because we don't know how the future data looks like so for avoiding future risk I will reserve 1gb for that case. And also 4GB is too much for you since it's on WSL but here we have the server so I think 2GB won't be a problem ;)


[Comment if interested] Grib Indexes

From @0x1mason on August 6, 2017 16:47
Makes searching much faster
Copied from original issue: 0x1mason/GribApi.NET#74

From @ArnimSchinz on September 14, 2017 15:26
We would love to have this feature! At the moment we are using the GribApi to convert files, map them on postal codes afterwards and save them to a database. If we could use indexes we maybe will not have to use the database anymore to access the data fast enough. In fact it would be awesome if we could just read them from flat files, because the grib2 format itself is really nice.

From @eriove on August 6, 2017 17:50
This is also a "nice to have" for us

From @IdahoSixString on August 7, 2017 16:39
Grib indexes IE dictionary<Key, value>? if so, yes, else no.


Validate generated certificates to contain the correct CN and SANs entries

Addresses #153

I don't think validation at validation is the right place, it gets complicated with add-worker workflow.
There is also code that would need to be duplicated.
I propose printing the warn messages and exiting in this part of the code, below is what it would look like:
Generating cluster Certificate Authority                                        [OK]
Generating certificates for host "etcd"                                         [OK]
Generating certificates for host "master"                                       [OK]
Generating certificates for host "worker"                                       [OK]
Generating certificates for service accounts                                    [OK]
Generating certificates for user "admin"                                        [OK]
Regenerating Certs
Found valid key and certificate for node "etcd"                                 [OK]
Found valid key and certificate for node "master"                               [OK]
Found valid key and certificate for node "worker"                               [OK]
Found key and certificate for service accounts                                  [OK]
Found key and certificate for user "user" but it is not valid                   [ERROR]
- error validating CN, expected "user", instead got "admin"
- error validating SANs, expected:
	[user]
  instead got:
	[admin]

Generating cluster Certificate Authority                                        [OK]
Generating certificates for host "etcd"                                         [OK]
Generating certificates for host "master"                                       [OK]
Generating certificates for host "worker"                                       [OK]
Generating certificates for service accounts                                    [OK]
Generating certificates for user "admin"                                        [OK]
Regenerating Certs
Found valid key and certificate for node "etcd"                                 [OK]
Found valid key and certificate for node "master"                               [OK]
Found valid key and certificate for node "worker"                               [OK]
Found key and certificate for service account but it is not valid               [ERROR]
- error parsing cert service-account: {"code":1002,"message":"Failed to decode certificate"}

Generating cluster Certificate Authority                                        [OK]
Generating certificates for host "etcd"                                         [OK]
Generating certificates for host "master"                                       [OK]
Generating certificates for host "worker"                                       [OK]
Generating certificates for docker registry                                     [OK]
Generating certificates for service accounts                                    [OK]
Generating certificates for user "admin"                                        [OK]
Found key and certificate for docker registry but it is not valid               [ERROR]
- error validating SANs, expected:
	[master <IP_ADDRESS> <IP_ADDRESS>]
  instead got:
	[<IP_ADDRESS> <IP_ADDRESS> master]

We already do something similar in "add-worker", where we validate and exit.

Is the above output from test runs? Just want to clarify

Yes it is


Improve description of NUS Classes in UG

Give explanation about Task formatting: Marked + overdue, unmarked + overdue, marked + undue, unmarked + undue

Done


Rename skip-koji-whitelist-check to skip-koji-tag-check

See https://github.com/theforeman/obal/pull/293

We can either coordinate this merge change with the change in Obal or pin Obal ahead of time to a commit or version we check out.

We no longer have Koji, so that's another approach I guess.


How to confirm that the model has converged? Using YTB data training, the first stage loss basically varies between [0.06, 0,07, 0.08], with no convergence trend

I download the pretrained ckpt, the pose decoder's state dict is missing.


Filename in Content-Disposition

In the same feature we are serving the content-disposition in nginx via GET, add another one to put the filename:
"Content-Disposition: attachment; filename="mallorca.jpg"
https://developer.mozilla.org/es/docs/Web/HTTP/Headers/Content-Disposition

Added another case in nginx.conf, to solve this
            if ($args ~* "download_filename"){
                add_header 'Content-Disposition' 'attachment; filename="$arg_download_filename"';
            }

Available in 6.2.0b version


Modularity

Modularity++ ? If so we can move others as well ;)
//cc @tj

Coverage remained the same when pulling 1fc5903edd73beefaf9cf7e061285e4c63b852e1 on hemanth:modularity into ef0de40e0b1581118e4eac9accfc7ef52798df0b on tj:master.

Why not? :+1:

keep co no deps should be better. And thunk-to-promise is no need to be a module.

:( if at any, which are those eligible candidates?
On Sun, Jan 11, 2015, 12:06 AM fengmk2<EMAIL_ADDRESS>wrote:

keep co no deps should be better. And thunk-to-promise is no need to be a
module.
—
Reply to this email directly or view it on GitHub
https://github.com/tj/co/pull/186#issuecomment-69466556.

yeah i've generally tried to avoid adding dependencies because people use this in the browser in non-commonjs environments. also, thunks shouldn't even be a thing, so i agree that thunk-to-promise shouldn't be a thing either :P

Number of modules that can go in to this:

is-promise

is-generator-function

is-generator

isObject

array-to-promise, yieldables-to-promise.

Modularity is tough to define...

In my opinion, "co" should be simple, especially that it may be used in browsers as well. Those "helpers" are quite small. Yeah, this PR is kind of an overkill.

Agree with @fengmk2 @jonathanong :smile: closing this PR.


CI issue

the issue with CI is starport related, let's discuss on tomorrow's call

CI seems to be working now


[Request] Higher mbit 1080p Recording for NEX-5R?

Hello,
A while back I saw someone hack or presumably hack the bitrate for NEX-5N camera. Here is the article for that http://www.eoshd.com/2012/08/sony-nex-5n-hack-is-go-46mbit-1080p-tested/
They stated that 46mbit 1080p would have been achieved. I wonder if this is still possible for other Sony cameras like the old NEX-5R?

Just for the record: There was never a working bitrate hack. "Someone1.00" found the decryption keys for the firmware updates (nobody knows how) and managed to reverse engineer the firmware file format. They then raised some money and disappeared.

@ma1co Ah yes this is what I found, but I didn't know about the fundraising scam. I'm sorry to hear that.

I wouldn't accuse them of scamming, apparently some money was indeed used to buy cameras. But the whole story was way over-hyped. Obtaining the firmware is an important first step. But understanding it, and then extending it is a whole different story.


The resource type 'operations' could not be found in the namespace 'Microsoft.Resources' for api version '2021-01-01'

@azure/arm-resources v4.1.0
Windows 10
NodeJS v14.17.0

Bug is related to the example shown here: https://azuresdkdocs.blob.core.windows.net/$web/javascript/azure-arm-resources/2.1.0/index.html
I am trying to test out the above example. The device authentication is successfull but the listing of Resources fails.
response: {
    body: `{"error":{"code":"InvalidResourceType","message":"The resource type 'operations' could not be found in the namespace 'Microsoft.Resources' for api version '2021-01-01'. The supported api-versions are '2015-01-01'."}}`,
    headers: HttpHeaders { _headersMap: [Object] },
    status: 404
  },
  body: {
    code: 'InvalidResourceType',
    message: "The resource type 'operations' could not be found in the namespace 'Microsoft.Resources' for api version '2021-01-01'. The supported api-versions are '2015-01-01'."
  }

My code is:
import * as msRestNodeAuth from "@azure/ms-rest-nodeauth";
import {
  ResourceManagementClient,
} from "@azure/arm-resources";

dotenv.config();

const port = process.env.PORT;

const temp = async () => {
  try {
    const credits = await msRestNodeAuth.interactiveLogin();
    console.log(credits);
    const client = new ResourceManagementClient(credits, subscriptionId);
    const data = await client.operations.list();
    console.log(data);
  } catch (err) {
    console.log(err);
  }
};

temp();

Am I missing something?

Hi @gerasimos1995 Thanks for reporting this issue.
This issue is caused by service, which returns 404 Not Found with Error Message:
{
    "error": {
        "code": "InvalidResourceType",
        "message": "The resource type 'operations' could not be found in the namespace 'Microsoft.Resources' for api version '2021-01-01'. The supported api-versions are '2015-01-01'."
    }
}

We need service team to confirm why we cannot list operations with tag 2021-01-01.
In Swagger, it's a valid request. There should be something wrong with the service backend or swagger definiton.
Thanks

Dear @dw511214992 should I close the issue? Do we have any updates?

Hi @gerasimos1995 we haven't get response from service team for long time. As this issue is in swagger/service side, I will close this issue and track it in https://github.com/Azure/azure-rest-api-specs/issues/14807
Thanks

Any update? I come across the same issue.

I aimed to list deployment operations. It works fine when through azure-cli or portal:

But when using azure-sdk will come across this error:
(node:24120) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `Code --trace-deprecation ...` to show where the warning was created)
rejected promise not handled within 1 second: Error: The resource type 'operations' could not be found in the namespace 'Microsoft.Resources' for api version '2021-01-01'. The supported api-versions are '2015-01-01'.
stack trace: Error: The resource type 'operations' could not be found in the namespace 'Microsoft.Resources' for api version '2021-01-01'. The supported api-versions are '2015-01-01'.
	at new RestError (C:\Users\dilin\workspace\TeamsDev\TeamsFx\packages\fx-core\node_modules\@azure\ms-rest-js\dist\msRest.node.js:2480:28)
	at C:\Users\dilin\workspace\TeamsDev\TeamsFx\packages\fx-core\node_modules\@azure\ms-rest-js\dist\msRest.node.js:3687:37
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async Timeout._onTimeout (C:\Users\dilin\workspace\TeamsDev\TeamsFx\packages\fx-core\build\plugins\solution\fx-solution\arm.js:137:36)

After discussed with @dilin-MS offline, the operation she is looking for is client.deploymentOperations.list() which should works fine in the current SDK package.
cc @lirenhe

Provide the workaround for the new ask and leverage https://github.com/Azure/azure-rest-api-specs/issues/14807 to track the service issue,  close this issue.


Respect hidden forum

Respect when discussion forum panel is hidden in Studio. This could be an extensive change. Based on ALLOW_HIDING_DISCUSSION_TAB feature in edxapp.

FYI The method currently used is not feasible in the Android app, plus not terribly happy about purposefully causing 404s.  Will look into properly updating the course data API to compensate.


get master accounts and sub account for enterprises

one of the request we have got from El Salvador is the possibility to have multiple accounts for enterprises.
Think of a chain of restaurant. The owner would like to have each restaurant with a wallet, and be able to know easily the total balances of all the wallet. Currently, an owner would have to open different wallet and each wallet would need its own access. Maybe at the end of day the restaurant would sent the balance back the "main" wallet.
I don't think the "meta" enterprise account would be manage through our current mobile wallet necessarily, but maybe the individual wallet would.
Note: this could be a paid functionality for the enterprise

We could solve this via introducing a more sophisticated RBAC model.
Roles like wallet-viewer, wallet-admin... or something like that.
Do you know if https://github.com/maticzav/graphql-shield supports this kind of use case?

We could solve this via introducing a more sophisticated RBAC model.
Roles like wallet-viewer, wallet-admin... or something like that.
Do you know if https://github.com/maticzav/graphql-shield supports this kind of use case?

or maybe just a solution in the top of galoy core (backend)? something that offer account management, payment reports by wallet/whole account, accounting reports, taxes, POS integration, etc...

Also I think that if we will add enterprise accounts to the core must have a different approach.. Ex:

for transactions: create/approve/release transactions (invoice payments, payroll, btc/usd conversion). This is important for large companies
Batch payments (integrated with invoice electronic invoice systems, ERPs, or just allowing access to the API )
Add multiple accounts for accounting issues (but not for employees)
etc

I think re should certainly do some model discovery around this feature set before implementing anything to get a better understanding of what we do / don't want to be. I think implementing a whole accounting system within Galoy is probably going a bit for. Integration with existing enterprise systems is a better approach. But certainly need to be aware of the trade offs and boundaries

I think think like tax and accounting for indivual/companies should not be done within our core system, but could be built on top of our API by us or some other companies when we have a mature API. I don't think those element should be part of a core banking solution. (we still need to do accounting obviously, but this is to maintain the ledgers of the customer deposits)
But things like having multiple account per user and delegation of account access I think could be part of our core banking system.

this is mostly done. will add more granular issue as needed


refactor method names to spring team conventions

Checklist

[ ] I tested all changes and all related features with different users (student, tutor, instructor, admin) on the test server https://artemistest.ase.in.tum.de.
[ ] Server: I added multiple integration tests (Spring) related to the features
[ ] Server: I added @PreAuthorize and check the course groups for all new REST Calls (security)
[ ] Server: I implemented the changes with a good performance and prevented too many database calls
[ ] Server: I documented the Java code using JavaDoc style.
[x] Client: I added multiple integration tests (Jest) related to the features
[x] Client: I added authorities to all new routes and check the course groups for displaying navigation elements (links, buttons)
[x] Client: I documented the TypeScript code using JSDoc style.
[x] Client: I added multiple screenshots/screencasts of my UI changes
[x] Client: I translated all the newly inserted strings into German and English

Motivation and Context

Functions in our JPA Repositories do not follow the official conventions for naming methods like findBySomething or findAllBySomething. This PR unifies the naming schema.

Issue number: https://github.com/ls1intum/Artemis/issues/760
Description

The methods of the repositories in the repository folder are refactored so they comply to the conventions by spring team. Also, since most of the methods have an @Query annotation and the SQL commands were sometimes lowercase and sometimes uppercase, now they are all refactored to uppercase for better readability.
Steps for Testing

Screenshots

What is the status of this PR?

It would be nice to have consistency here, but without a review that no functionality is broken, the risk is too high that something does not work after the refactoring any more


Show clear widget on prefilled state

When I pushed up #92 I had not yet added any tests over the functionality, and so during the refactoring one bit of functionality was lost without being noticed. When I load an edit form where a combobox field already has a value, I expect to see the clear handle instead of the open handle. This adds the simplest test scenario and corresponding fix that I could think of.

Great catch, and an elegant solution + test. Love to see it! Thanks!


Changing Size of Integrated Terminal Does Not Change text rows

Issue Type: Bug
Open an integrated terminal. Try to resize the pane larger, either by dragging the splitter, or clicking the Expand icon. The pane resizes, but the number of rows in the terminal window does not change. (ie, if the window started with 6 rows of text, expanding the pane will grow the pane, but the "terminal" will still only take 6 rows at the bottom of the window.
VS Code version: Code - Insiders 1.22.0-insider (a918e7b3ef2feaf2c9fbf003e936084eaaf53f76, 2018-03-26T05:16:34.852Z)
OS version: Windows_NT x64 10.0.16299

System Info

Item
Value

CPUs
Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz (2 x 3691)

Memory (System)
12.00GB (6.47GB free)

Process Argv
C:\Program Files\Microsoft VS Code Insiders\Code - Insiders.exe

Screen Reader
no

VM
100%

Extensions (19)

Extension
Author (truncated)
Version

ng-template
Ang
0.1.9

path-intellisense
chr
1.4.2

bracket-pair-colorizer
Coe
1.0.27

vscode-svgviewer
css
1.4.3

githistory
don
0.4.0

tslint
eg2
1.0.28

auto-rename-tag
for
0.0.15

Angular2
joh
2.15.0

json-to-ts
Mar
1.5.4

markdown-shortcuts
mdi
0.8.1

code-beautifier
mic
1.7.2

prettify-json
moh
0.0.3

csharp
ms-
1.14.0

vsliveshare
ms-
0.2.260

debugger-for-chrome
msj
4.2.2

material-icon-theme
PKi
3.2.6

polacode
pnp
0.2.2

view-in-browser
qin
0.0.5

MDTools
sea
1.0.1

(3 theme extensions excluded)

Reproduces without extensions

Update: This happens on all platforms - Linux, Windows and OS X.

I see this was reported in #46577.


TST: Coveralls

Changes Unknown when pulling ac6a45a8a06c58c781a4ff28c431bf3e895dd817 on sinhrks:coveralls into ** on pandas-ml:master**.


New contemporary change and future climate figs

at some point we are going to need to move these data to SB so they aren't in the repo, and use the fetch parts of vizlab: https://github.com/USGS-VIZLAB/vizlab/pull/17


Isolate.spawnUri() should default to the same language version as the parent isolate

Currently, if you run Isolate.spawnUri() on Dart 2.12.0-dev, it will always spawn the isolate in null-safety mode, even if the parent isolate is running in an older language version mode. To reproduce:
import 'dart:isolate';

main() async {
  await Isolate.spawnUri(Uri.dataFromString('int a; main() {}'), [], null);
}

This breaks the test package's hybrid testing APIs, since they're now always spawning in null-safety mode, which in turn breaks Dart Sass's CI. This is a high-severity issue for us, as it prevents us from testing on Dart 2.12.

Duplicate of #44181

See also https://github.com/dart-lang/test/issues/1363
I thought @jakemac53 has fixed it in test package somehow, but I have not tested it.

You will need to add language comments (// @dart=2.10) etc if spawning code from a uri that is not under the current package, which is not opted in. This includes data uris as well as uris under some temp directory (such as files created with test_descriptor).
This is a known breaking change with the null safety release. The default language version is the latest sdk, and this behaves identically to dart <some-uri>, which is the intention.

You will need to add language comments (// @dart=2.9) etc if spawning non-null-safe code from a uri that is not under the current package. This includes data uris as well as uris under some temp directory (such as files created with test_descriptor).

In gRPC test I am spawning a file from test/ directory and it does not seem to pick up opt-out even though other files (tests) are considered opted out.

In gRPC test I am spawning a file from test/ directory and it does not seem to pick up opt-out even though other files (tests) are considered opted out.

It looks like you are using Isolate.spawn? I don't know how we intend that to behave. Their is no uri to go off of, so inheriting the current isolates mode does seem reasonable in that case to me.


Keras BatchNormalization causes "failed to allocate XX bytes from device" error

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Linux ubuntu 16.04 ( a1.vega64 instance from https://www.gpueater.com )
TensorFlow version (use command below):
1.12.0
Python version:
3.5.2
ROCm/MIOpen version:

=====================
HSA System Attributes
=====================
Runtime Version:         1.1
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:<PHONE_NUMBER>3709551615 (number of timestamp)
Machine Model:           LARGE
System Endianness:       LITTLE

==========
HSA Agents
==========
*******
Agent 1
*******
  Name:                    Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz
  Vendor Name:             CPU
  Feature:                 None specified
  Profile:                 FULL_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        0
  Queue Min Size:          0
  Queue Max Size:          0
  Queue Type:              MULTI
  Node:                    0
  Device Type:             CPU
  Cache Info:
    L1:                      32768KB
  Chip ID:                 0
  Cacheline Size:          64
  Max Clock Frequency (MHz):3900
  BDFID:                   0
  Compute Unit:            4
  Features:                None
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    16362668KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Acessible by all:        TRUE
    Pool 2
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    16362668KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Acessible by all:        TRUE
  ISA Info:
    N/A
*******
Agent 2
*******
  Name:                    gfx900
  Vendor Name:             AMD
  Feature:                 KERNEL_DISPATCH
  Profile:                 BASE_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        128
  Queue Min Size:          4096
  Queue Max Size:          131072
  Queue Type:              MULTI
  Node:                    1
  Device Type:             GPU
  Cache Info:
    L1:                      16KB
  Chip ID:                 26751
  Cacheline Size:          64
  Max Clock Frequency (MHz):1630
  BDFID:                   768
  Compute Unit:            64
  Features:                KERNEL_DISPATCH
  Fast F16 Operation:      FALSE
  Wavefront Size:          64
  Workgroup Max Size:      1024
  Workgroup Max Size Per Dimension:
    Dim[0]:                  67109888
    Dim[1]:                  50332672
    Dim[2]:                  0
  Grid Max Size:<PHONE_NUMBER>
  Waves Per CU:            40
  Max Work-item Per CU:    2560
  Grid Max Size per Dimension:
    Dim[0]:<PHONE_NUMBER>
    Dim[1]:<PHONE_NUMBER>
    Dim[2]:<PHONE_NUMBER>
  Max number Of fbarriers Per Workgroup:32
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    8372224KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Acessible by all:        FALSE
    Pool 2
      Segment:                 GROUP
      Size:                    64KB
      Allocatable:             FALSE
      Alloc Granule:           0KB
      Alloc Alignment:         0KB
      Acessible by all:        FALSE
  ISA Info:
    ISA 1
      Name:                    amdgcn-amd-amdhsa--gfx900
      Machine Models:          HSA_MACHINE_MODEL_LARGE
      Profiles:                HSA_PROFILE_BASE
      Default Rounding Mode:   NEAR
      Default Rounding Mode:   NEAR
      Fast f16:                TRUE
      Workgroup Max Dimension:
        Dim[0]:                  67109888
        Dim[1]:                  1024
        Dim[2]:                  16777217
      Workgroup Max Size:      1024
      Grid Max Dimension:
        x<PHONE_NUMBER>
        y<PHONE_NUMBER>
        z<PHONE_NUMBER>
      Grid Max Size:<PHONE_NUMBER>
      FBarrier Max Size:       32
*** Done ***

I was trying to train the this Keras model using ROCm platform. But it failed with following error
> /root/pottan-ocr/misc/keras_model.py(15)KerasCrnn()
-> ks = [3, 3, 3, 3, 3, 3, 2]
(Pdb) c
> /root/pottan-ocr/misc/keras_model.py(33)convRelu()
-> cnn.add( BatchNormalization(axis=3, momentum=0.1, epsilon=1e-4, name='batchnorm{0}'.format(i) ))
(Pdb)
(Pdb) n
2019-03-19 22:01:57.723296: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2019-03-19 22:01:57.723696: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1530] Found device 0 with properties:
name: Vega 10 XT [Radeon RX Vega 64]
AMDGPU ISA: gfx900
memoryClockRate (GHz) 1.63
pciBusID 0000:03:00.0
Total memory: 7.98GiB
Free memory: 7.73GiB
2019-03-19 22:01:57.723721: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Adding visible gpu devices: 0
2019-03-19 22:01:57.723866: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1051] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-03-19 22:01:57.723877: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1057]      0
2019-03-19 22:01:57.723883: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1070] 0:   N
2019-03-19 22:01:57.723913: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1189] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7524 MB memory) -> physical GPU (device: 0, name: Vega 10 XT [Radeon RX Vega 64], pci bus id: 0000:03:00.0)
2019-03-19 22:01:57.769754: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 7.35G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.769826: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 6.61G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.769851: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 5.95G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.769872: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 5.36G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.769894: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 4.82G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.769915: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 4.34G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.769936: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 3.90G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.769957: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 3.51G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.769978: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 3.16G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770000: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 2.85G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770025: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 2.56G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770049: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 2.31G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770070: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 2.07G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770092: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 1.87G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770113: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 1.68G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770134: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 1.51G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770155: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 1.36G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770177: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 1.22G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770201: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 1.10G<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770224: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 1016.38M<PHONE_NUMBER> bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770246: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 914.74M (959177216 bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770267: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 823.27M (863259648 bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770289: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 740.94M (776933632 bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770311: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 666.85M (699240448 bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770332: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 600.16M (629316608 bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770357: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 540.15M (566385152 bytes) from device: hipError_t(1002)
2019-03-19 22:01:57.770380: E tensorflow/stream_executor/rocm/rocm_driver.cc:601] failed to allocate 486.13M (509746688 bytes) from device: hipError_t(1002)
terminate called after throwing an instance of 'std::runtime_error'
  what():  No device code available for function: _ZN10tensorflow7functor28FillPhiloxRandomKernelLaunchINS_6random19UniformDistributionINS2_12PhiloxRandomEfEEEEvS4_PNT_17ResultElementTypeExS6_
Aborted (core dumped)

Code to reproduce the issue
python3 misc/run_keras.py   --traindata ../datasets/validate.txt.gz  --valdata ../datasets/validate.txt.gz   --batchSize 64 --traindata_limit $(( 1999*64 )) --valdata_limit 128 --nh 64

Complete shell script to automate the process: https://github.com/harish2704/harish2704.github.io/blob/master/for-pottan-ocr/boot_gpueater.sh

The line which caused this bug* : https://github.com/harish2704/pottan-ocr/blob/108234c589533608b9a7ced7f95906cd1c6cf19d/misc/keras_model.py#L31

@harish2704 The error message looks like related to your local configuration.
Could you try to run the HIP unit tests? Link is here:
https://github.com/ROCm-Developer-Tools/HIP/tree/master/tests

Besides, could you also review the following comment from @jlgreathouse ?
https://github.com/RadeonOpenCompute/ROCm/issues/622#issuecomment-441941594
The failure signature seems related to your case.

Hi @sunway513 : You are right. It seems like a configuration problem of the system I am using.
The issue disappeared when I tried to replicate the same on a different machine.
I was not expecting an hardware configuration issue from a service provider who officially provides ROCm enabled GPU instances . That is why I didn't checked for the possibility of a configuration issue in the first time.
Thanks!!


Persisting data in Kubernetes

I've been using this image for a while no with no problems, except that I don't think it's persisting information in the disks.
I'm using managed Kubernetes on Digital Ocean, everything runs great. Here's my setup:
Postgres:
apiVersion: v1
kind: ConfigMap
metadata:
  name: gigs-psql-config-map
  namespace: gigsnet
  labels:
    owner: gigs-psql
data:
  PG_DATA: "/var/lib/postgresql/pgdata"
  POSTGRES_HOST: "gigs-psql-service"
  POSTGRES_PORT: "5432"
  POSTGRES_USER: "gigs_master"
  POSTGRES_DB: "gigs_master"
  POSTGRES_PASSWORD: "myPasswordHere"
---
apiVersion: v1
kind: Service
metadata:
  name: gigs-psql-service
  namespace: gigsnet
  labels:
    app: gigs-psql
spec:
  type: NodePort
  ports:
  - name: tcp
    protocol: TCP
    port: 5432
  selector:
    app: gigs-psql
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gigs-psql-deployment
  namespace: gigsnet
  labels:
    app: gigs-psql
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gigs-psql
  template:
    metadata:
      name: gigs-psql-template
      labels:
        app: gigs-psql
    spec:
      containers:
        - image: postgres:12-alpine
          imagePullPolicy: Always
          name: gigs-psql-container
          livenessProbe:
            initialDelaySeconds: 180
            tcpSocket:
              port: 5432
          envFrom:
            - configMapRef:
                name: gigs-psql-config-map
          volumeMounts:
            - mountPath: /var/lib/postgresql/pgdata
              name: gigs-psql-pgdata-pvc
      volumes:
        - name: gigs-psql-pgdata-pvc
          persistentVolumeClaim:
            claimName: gigs-psql-pgdata-pvc

PVC:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: gigs-psql-pgdata-pvc
  namespace: gigsnet
  labels:
    app: gigs-psql
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 4Gi
  storageClassName: do-block-storage

It works just fine until me or someone else deletes the Pod/Deployment. Deleting the Pod or Deployment will shut down the container but will leave the PVC there with the data written in it. The expected behavior is that the Postgres image will be able to recover the database state after creating the Pod, mounting the volume again, as happens with the MariaDB image, for instance.
But it's not happening. The container gets recreated, mounts the PVC but the database behaves like it was empty.
Any tip of what may be happening here or how to solve that?
Thank you!

Does the PV have persistentVolumeReclaimPolicy set to reclaim?
Might also be that if the pod is being rescheduled on a different node then its ReadWriteOnce access mode won't mount to a different node
Your PG_DATA env var is wrong, it should be PGDATA

Thanks for spotting this, you're right.


Won't show events fired

I'm having issues w/ the UI - when I choose the Omnibug tab within Firebug, and take an action on the page to fire an event, Firebug defaults to the HTML tab.
Even switching back to Omnibug, it doesn't display a list of events fired. It shows HTML.
How can I get this to show me actions taken within an environment with omniture tagging.

Hi @Queststjacobs  - What version of Firefox & Omnibug are you running?

The latest. 37
Quest St. Jacobs
On Thu, Apr 23, 2015 at 5:37 PM, Philip Lawrence<EMAIL_ADDRESS>wrote:

Hi @Queststjacobs https://github.com/Queststjacobs - What version of
Firefox & Omnibug are you running?
—
Reply to this email directly or view it on GitHub
https://github.com/simpsora/omnibug/issues/21#issuecomment-95748366.

What version for Omnibug? If it is older than 0.5.509, you'll need to update it as there was a fix for a similar sounding issue. I believe Mozilla still hasn't approved the updated version in the store, but you can still download the latest at http://omnibug.rosssimpson.com/

Hi @Queststjacobs, as @MisterPhilip says, please try downloading a new version from the Omnibug site -- there was a bug in a previous release, and the auto-update feature is not working for that release.  Please let us know if you continue to have problems!


Support for other media players (e.g. VLC)

I'd like this program to one day add support for other media players such as VLC, in that it passes a radio stream to that app. For example, it might call "vlc XXX", where XXX is the URL of the requested stream.

Or choosing mpv as player would be already good alternative as mpv used IPC, so it can be controlled remotely.

I suppose I could just take the path to the stream -- i.e. https://path.to/stream -- and append it to an mpv command, such that mpv https://path.to/stream, and that's that. Of course, not all of us are as savvy with the command line as I am, so maybe we ought to consider adding support for mpv and other command-line audio players natively, such that this program is a frontend.

@robinhood2014: radioactive allows adding a station to favorite list, so you will have the stream URL. With that I am using my mpv.sh script to play it and control it with another script.
#!/bin/sh
$(which mpv) --input-ipc-server=/tmp/soc_mpv "$@"

Then I use my media buttons mapped to other script which allows play/pause operations:
#!/bin/sh
# play
echo '{ "command": ["set", "pause", "no"] }' | socat - /tmp/soc_mpv

#!/bin/sh
# pause
echo '{ "command": ["set", "pause", "yes"] }' | socat - /tmp/soc_mpv

The best would be to have some way to configure radioactive to allow external player command and also play and pause commands.

I'll work on this next week.

What about command line parameters for the autio players? Is there a way to pass them to the radioactive?


Paginator not working with query "*"

Method getPage() ignore a page number and return all objects.
May be offset() or limit() problem?

fixed in https://github.com/javalite/activejdbc/issues/883


Links to Genome, CDS and Browser on jobs page are broken

You click on the 3 links at the top right of the page (before the green board) and nothing works.

We have a webinar on genome annotation tomorrow (11/20).  It would be swell if this worked.

@olsonanl i'm looking at Rebecca's latest annotation run (job id: f9b254c1-1972-454e-b277-17ea0a654890) and it looks like the usual autometa from this run is missing.  Any idea why?
"{
    "id": "9DDE1AD2-EC10-11E8-B24C-E6B0682E0674",
    "path"<EMAIL_ADDRESS>folder/.Klebsiella pneumoniae 704SK6_complete/Klebsiella pneumoniae 704SK6_complete.genome",
    "name": "Klebsiella pneumoniae 704SK6_complete.genome",
    "type": "genome",
    "creation_time": "2018-11-19T15:34:34Z",
    "link_reference": "https://p3.theseed.org/services/shock_api/node/1724ce9e-2989-4088-a443-c60977dbfc8a",
    "owner_id"<EMAIL_ADDRESS>    "size": 21066834,
    "userMeta": {},
    "autoMeta": {
        "inspection_started": "2018-11-19T15:36:18Z",
        "is_folder": 0
    },
    "user_permission": "o",
    "global_permission": "n",
    "timestamp":<PHONE_NUMBER>000
}"

Earlier run:
"{
    "id": "6A55A50A-E5CA-11E8-9514-9CF5682E0674",
    "path"<EMAIL_ADDRESS>folder/.Bacteria saeb test 4/.annotation/annotation.genome",
    "name": "annotation.genome",
    "type": "genome",
    "creation_time": "2018-11-11T15:56:56Z",
    "link_reference": "https://p3.theseed.org/services/shock_api/node/a0277e42-fb46-4ee0-859e-c0ae3a0e987f",
    "owner_id"<EMAIL_ADDRESS>    "size": 22468022,
    "userMeta": {},
    "autoMeta": {
        "N90": 32141,
        "is_folder": 0,
        "gc_content": 30.3036670091033,
        "taxonomy": null,
        "dna_size": 7353834,
        "ncbi_taxonomy_id": "2",
        "num_features": 7297,
        "scientific_name": "Bacteria saeb test 4",
        "domain": "Bacteria",
        "N50": 100514,
        "N70": 60789,
        "genome_id": "2.1969",
        "num_contigs": 174
    },
    "user_permission": "o",
    "global_permission": "n",
    "timestamp":<PHONE_NUMBER>000
}"

I will check; I updated workspace code and the autometa stuff there is delicate.
--bob

From: nconrad<EMAIL_ADDRESS>Sent: Monday, November 19, 2018 10:34 AM
To: PATRIC3/patric3_website
Cc: Olson, Robert D.; Mention
Subject: Re: [PATRIC3/patric3_website] Links to Genome, CDS and Browser on jobs page are broken (#2152)
@olsonanlhttps://github.com/olsonanl i'm looking at Rebecca's latest annotation run (job id: f9b254c1-1972-454e-b277-17ea0a654890) and it looks like the usual autometa from this run is missing. Any idea why?
"{
"id": "9DDE1AD2-EC10-11E8-B24C-E6B0682E0674",
"path"<EMAIL_ADDRESS>folder/.Klebsiella pneumoniae 704SK6_complete/Klebsiella pneumoniae 704SK6_complete.genome",
"name": "Klebsiella pneumoniae 704SK6_complete.genome",
"type": "genome",
"creation_time": "2018-11-19T15:34:34Z",
"link_reference": "https://p3.theseed.org/services/shock_api/node/1724ce9e-2989-4088-a443-c60977dbfc8a",
"owner_id"<EMAIL_ADDRESS>"size": 21066834,
"userMeta": {},
"autoMeta": {
"inspection_started": "2018-11-19T15:36:18Z",
"is_folder": 0
},
"user_permission": "o",
"global_permission": "n",
"timestamp":<PHONE_NUMBER>000
}"
Earlier run:
"{
"id": "6A55A50A-E5CA-11E8-9514-9CF5682E0674",
"path"<EMAIL_ADDRESS>folder/.Bacteria saeb test 4/.annotation/annotation.genome",
"name": "annotation.genome",
"type": "genome",
"creation_time": "2018-11-11T15:56:56Z",
"link_reference": "https://p3.theseed.org/services/shock_api/node/a0277e42-fb46-4ee0-859e-c0ae3a0e987f",
"owner_id"<EMAIL_ADDRESS>"size": 22468022,
"userMeta": {},
"autoMeta": {
"N90": 32141,
"is_folder": 0,
"gc_content": 30.3036670091033,
"taxonomy": null,
"dna_size": 7353834,
"ncbi_taxonomy_id": "2",
"num_features": 7297,
"scientific_name": "Bacteria saeb test 4",
"domain": "Bacteria",
"N50": 100514,
"N70": 60789,
"genome_id": "2.1969",
"num_contigs": 174
},
"user_permission": "o",
"global_permission": "n",
"timestamp":<PHONE_NUMBER>000
}"
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/PATRIC3/patric3_website/issues/2152#issuecomment-439956422, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACDXIppdK8-ANaTFYvr4s-8xEKANBaH4ks5uwt15gaJpZM4YpZe3.

hmmm... I also see the latest one there is .Klebsiella pneumoniae 704SK6_complete/Klebsiella pneumoniae 704SK6_complete.genome while the previous is .annotation/annotation.genome.    I'm just looking at what the UI is filtering out and using for the id:
https://github.com/PATRIC3/p3_web/blob/d6567eabfff24399ff82f12ce358bda5ce1455c8/public/js/p3/widget/viewer/GenomeAnnotation.js#L8-L20

the autometa script was invoked:
Invoke /disks/p3/deployment/services/Workspace/bin/ws-autometa-job_result on /disks/p3/workspace/wsjobs/avz1JCHGMn/object.txt for<EMAIL_ADDRESS>folder/Klebsiella pneumoniae 704SK6_complete

Errors are logged, and there wasn't anything in the log.
I just instrumented the metadata scripts and it is being generated, and the genome link works properly.
It's not clear what is going on .

The problem is that mongo is dumb. You can't have dots in keys in metadata. We were using contig ids for metadata keys (what a radical concept) and they break the mongo insert.
I've hacked the problem for now by not saving that bit of metadata


middleware: CSRF

change validate and checkCSRFToken to add new tokens
on request attributes.

Motivation for this is to be able to use CSRF with forms
as well.

@rossabaker @jmcardon WDYT?

I am not sure what I was thinking wrt. changing checkCSRFToken, because a form only needs the validate change. Will change this tomorrow

@rossabaker Ah, now I remember. It is needed for checkCSRFToken in scenarios where a POST is invalid and a form is needed, hence a token.

The middleware type for CSRF in http4s is
Middleware[F, Request[G], Response[G], Request[G], Response[G]]

whereas in TSecCSRF it is:
Middleware[OptionT[F, ?], Request[F], Response[F], Request[F], Response[F]]

Why is this so?

@ahjohannessen because tsec is before 0.19 changes.

What is the goal with adding CSRF tokens to request attributes? I don't think I understand.

@jmcardon such that they are available for hidden form fields in twirl

If you're using twirl to render a form server side, you can still extract the csrf token as usual. This is what this function is for:
   def refreshOrCreate[M[_]](r: Request[G])(
       implicit F: Sync[M]): EitherT[M, CSRFCheckFailed, CSRFToken] =	       implicit F: Sync[M]): EitherT[M, CSRFCheckFailed, CSRFToken] =
     CSRF.cookieFromHeaders(r, cookieName) match {	     CSRF.cookieFromHeaders(r, cookieName) match {
       case Some(c) =>	       case Some(c) =>
         EitherT(F.pure(extractRaw(c.content)))	         EitherT(F.pure(extractRaw(c.content)))
           .semiflatMap(signToken[M])	           .semiflatMap(signToken[M])
       case None =>	       case None =>
         EitherT.liftF(generateToken[M])	         EitherT.liftF(generateToken[M])
     }	     }

@jmcardon Reading a bit up on this I think I have misunderstood how this works. I thought that the raw token was changed on every request, it is just validated and re-signed.
So, in case of twirl forms I can do this:

On login POST route after authentication I use embedNewInResponseCookie.
Use refreshOrCreate for subsequent twirl forms.

I should not need to wrap login (GET/POST) routes with csrf.validate(), because the user has to authenticate anyway.
So an example of how I imagine how to use this is:
val csrf: CSRF[OptionT[F, ?], F] = ???
...
val asset: HttpRoutes[F] = ???
val login: HttpRoutes[F] = ???

val serv1: HttpRoutes[F] = ???
val serv2: HttpRoutes[F] = ???
val serv3: HttpRoutes[F] = ???
...
val services: HttpRoutes[F] = auth.liftService(serv1 <+> serv2 <+> serv3)
val protect: HttpRoutes[F]  = csrf.validate()(services)

val app = Router("/" → protect, "/auth" → login, "/assets" → asset).orNotFound

Should the user of csrf not be able to specify response cookie path, to say /?

Closing this as it was based on wrong assumptions from my side.


New filter option: Add a filter for Legendary Creatures

It would be great to filter the monster list to only show legendary creatures for solo encounter building.

Do you mean show only creatures with legendary actions?
On Thu, Nov 16, 2017 at 1:41 PM Cybertroll-xx<EMAIL_ADDRESS>wrote:

It would be great to filter the monster list to only show legendary
creatures for solo encounter building.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Asmor/5e-monsters/issues/87, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAb_N2Qu9vkJwWlc9_PULZ3R9h_FuRfFks5s3Kv3gaJpZM4QhLr-
.

Good idea. Although what I may end up doing is just making it so you can search for "legendary"

That'll work :)
Sent via the Samsung Galaxy S7 edge, an AT&T 4G LTE smartphone
-------- Original message --------From: Ian Toltz<EMAIL_ADDRESS>Date: 11/16/17  7:55 PM  (GMT-05:00) To: Asmor/5e-monsters<EMAIL_ADDRESS>Cc: Cybertroll-xx<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [Asmor/5e-monsters] New filter option: Add a filter for Legendary
  Creatures (#87)
Good idea. Although what I may end up doing is just making it so you can search for "legendary"
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.
{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/Asmor/5e-monsters","title":"Asmor/5e-monsters","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/Asmor/5e-monsters"}},"updates":{"snippets":[{"icon":"PERSON","message":"@Asmor in #87: Good idea. Although what I may end up doing is just making it so you can search for "legendary""}],"action":{"name":"View Issue","url":"https://github.com/Asmor/5e-monsters/issues/87#issuecomment-345112096"}}}


clusterlogging pkg added

namespace.go still in commit

it is not 'still', it's just back when I did refactoring for the Pull method name. fixed


Vertical harmony for headings(and other elements)?

https://getbootstrap.com/docs/4.0/content/typography
Could you tell me why you use same margin(margin-bottom: .5rem;) in every heading? Have you ever heard about vertical rhythm?
https://www.smashingmagazine.com/2012/12/css-baseline-the-good-the-bad-and-the-ugly
https://jhildenbiddle.github.io/vertical-rhythm-reset
How come the so called perfect Bootstrap doesn't know about this?

How come the so called perfect Bootstrap doesn't know about this?

Who said Bootstrap was perfect? And who says we don't know about it? Please consider checking the attitude at the door—we're just maintaining an open source project.

Could you tell me why you use same margin(margin-bottom: .5rem;) in every heading? Have you ever heard about vertical rhythm?

We're optimizing for a few things with our heading styles, chief among them Bootstrap's use in more complex sites and apps where vertical rhythm across all components and columns isn't exactly as important. We reset the styles to provide a consistent baseline and provide utilities for folks to customize as needed.
By the way, one of the examples you included use a similar approach to singular margin values on all elements:
blockquote, dl, form, h2, h3, h4, h5, h6, ol, p, pre, table, ul {
  margin-top: 1.50001rem;
  margin-bottom: 1.50001rem;
}

It's unlikely we'll make significant changes here for every heading and other elements, but we do have some issues open for creating a "content component" where we apply more typographically sound spacing and sizing specifically designed for longer content. That might be where we start for something like this, but I'm not 100% sure this would help you.
Lastly, Issues are for bug reports and feature requests. Please consider using Discussions next time for open ended questions :).

Please consider checking the attitude at the door—we're just maintaining an open source project.
Ok maybe my attitude was not proper but it just annoys me that something so basic/important thing is not taken into account in Bootstrap.

Who said Bootstrap was perfect?

Nobody said but i think most of developers use it. So this is why Bootstrap should use every modern and important technology and Bootstrap shouldn't be just a "reset" framework.

We're optimizing for a few things with our heading styles, chief among them Bootstrap's use in more complex sites and apps where vertical rhythm across all components and columns isn't exactly as important.
The vertical rythm should be applied to the headings at least. Not important? Yes it is not because nobody(neither the developers) knows about it.

By the way, one of the examples you included use a similar approach to singular margin values on all elements:

If you looked closely style of all headings is overriden and other elements like p, ul, table so on can be set globally.

but we do have some issues open for creating a "content component" where we apply more typographically sound spacing and sizing specifically designed for longer content.

At least this is a good news and I hope it will be implemented soon.

Lastly, Issues are for bug reports and feature requests. Please consider using Discussions next time for open ended questions :).
I didn't know where i should put this, because it is both bug report and feature request i guess.


[identity] Expires on timestamp should be in seconds not ms

Packages impacted by this PR
@azure/identity
Issues associated with this PR
Fixes https://github.com/Azure/azure-sdk-for-js/issues/23903
Describe the problem that is addressed by this PR
The "expiresOnTimestamp" field in Managed Identity should be in seconds and was assigned milliseconds. The PR fixes the bug
What are the possible designs available to address the problem? If there are more than one possible design, why was the one in this PR chosen?
Are there test cases added in this PR? (If not, why?)
Provide a list of related PRs (if any)
Command used to generate this PR:**(Applicable only to SDK release request PRs)
Checklists

[ ] Added impacted package name to the issue description
[ ] Does this PR needs any fixes in the SDK Generator?** (If so, create an Issue in the Autorest/typescript repository and link it here)
[ ] Added a changelog (if necessary)

API change check
API changes are not detected in this pull request.


load_from_disk takes a long time to load local dataset

I have an extremely large tokenized dataset (24M examples) that loads in a few minutes. However, after adding a column similar to input_ids (basically a list of integers) and saving the dataset to disk, the load time goes to >1 hour. I've even tried using np.uint8 after seeing #1985 but it doesn't seem to be helping (the total size seems to be smaller though).
Does anyone know what could be the issue? Or does the casting of that column to int8 need to happen in the function that writes the arrow table instead of in the map where I create the list of integers?
Tagging @lhoestq since you seem to be working on these issues and PRs :)

Hi !
Can you share more information about the features of your dataset ? You can get them by printing my_dataset.features
Can you also share the code of your map function ?

It is actually just the tokenized wikipedia dataset with input_ids, attention_mask, etc, with one extra column which is a list of integers. The text column is removed during tokenization.
def add_len_and_seq(example):
    example['actual_len'] = example['input_ids'].index(SEP)-1
    seq_len = len(example['input_ids'])
    end_idx = example['actual_len']+1

    example['seq'] = [PAD_ID] + [np.uint8(example['some_integer'])]*(end_idx-1) + [PAD_ID]*(seq_len-end_idx)

    return example

Is PAD_ID a python integer ? You need all the integers in example['seq'] to have the same type.
Does this work if you remove the np.uint8 and use python integers instead ?

yup I casted it outside the function where it was defined. It was originally using python integers.

Strangely, even when I manually created np.arrays of specific dtypes, the types in the final dataset_info.json that gets written are still int64.

Yes this is a known issue: #625
We're working on making the precision kept for numpy :)
To specify the precision of the integers, currently one needs to specify the output features with .map(..., features=output_features)

Do you know what step is taking forever in the code ?
What happens if you interrupt the execution of the dataset loading ?

After a synchronous discussion, we found that the cache file sizes have an enormous effect on the loading speed: smaller cache files result in faster load times. num_proc controls the number of cache files that are being written and is inversely proportional to the individual file size. In other words, increase num_proc for smaller cache files :)
Maybe this can be highlighted somewhere in the docs.


Optional stuff

I think we need to add some options for different cases.
http://cssmojo.com/the-very-latest-clearfix-reloaded/

Yeah could certainly add an option in the JS api for block vs table display of the clearfix. I probably don't have time right now to do it, but would happily accept a PR.


NEXT: Toast Component

Zag Integration

React: https://zagjs.com/components/react/toast
Svelte: https://github.com/chakra-ui/zag/tree/main/website/data/snippets/svelte/toast

Maintainer Requests
The following requests are coming straight from the Skeleton team. These are highly likely be implemented:

[ ] Should implement all relevant Zag features
[ ] Should aim for feature parity with v2 component
[ ] Consider support for dynamic components for rich media display
[ ] Consider support for progress bars showing display time remaining
[ ] Consider a fixed option to use as a GDPR cookies banner

Community Requests
The following requests have come from the community and are under consideration:

https://github.com/skeletonlabs/skeleton/issues/2398
https://github.com/skeletonlabs/skeleton/pull/2673

Feedback
If you have additional updates or requests for this feature, please do so in the comments section below.

Note that I've followed up with the Zag team regarding a few issues with this component:

Currently there' no obvious and/or available type for <Toast> and <ToastProvider> components.
We can set a action button label and callback function - unfortunately the template API doesn't provide label
Finally, the overlap: true option is not working. I don't think their docs provide the correct CSS styles.

Depending on how quickly they respond and what they say, we may have to pause and return to this feature a later time. If they can at least provide the proper types, we can likely build an initial version using a subset of features.
Keep an eye on this post and the PR for further updates.

I've talked with the Zag team and they have noted the issues at hand with the Toast component, but my assumption is it won't be a quick turnaround. After reviewing with the team, I'm going to pivot to a third party integration using the following:

React: https://react-hot-toast.com/
Svelte: https://svelte-french-toast.com/

These seems like fully featured solutions, with tons of options, and support for Tailwind styling.
I'll begin by creating proofs of concept. If that goes well I'll proceed with the full integration guides. The goal being to have these completed before the Beta release next Tuesday.
In the future, we're happy to revisit the Zag component once improvements are in place. If we feel it meets our expectations we may replace our integration guide with this.

On more quick update - the react-hot-toast library is working great. It's super mature and well documented. Feature-wise it seems to do everything we need it to do.
Unfortunately svelte-french-toast isn't quite there. The docs are lacking, it's currently using Svelte 4, and the project itself seems all but abandoned. It's not received updates in nearly a year, even though there appears to be a community-led effort to update and maintain it.
For now, we're going to roll our own custom Toast component specifically for Svelte users. This will mirror the Zag APIs in the hopes we can swap it out with the Zag version in a future release. When that happens, it should remain backwards compatible, but receive a plethora of new features. As such, we'll label the current version with a [beta] tag to indicate this one is still in flux and subject to breaking changes.


Implementing a catch all route

How would Klein be able to setup a catch all route? So I can go through a list of rules and if none of them matched, use this last one?

Just an adjustment to @Rican7 's answer for Klein 2.1 dev
// Klein current master (2.1 dev) style:
$klein->respond(function($request, $response, $service, $app, $klein, $matched) {
if (count($matched->all()) > 0) {
// We must have matched a route earlier so do nothing
} else {
// I guess we didn't match anything earlier, let's go home..., do what you like
}
});
$matched > 0 alone did not work for me.


Update ipn.rst

Explained how 'custom' parameter is used in the example. Also verified that only 'return_url' requires @csrf_exempt and only in the case of single payments. (Fixed indentation errors)

Thanks!