Primary key and its effect on data size

Does primary key composition have influence on data size in the same manner as in other databases like MySQL or Postgres?
If I understood correctly, it influences how data are sorted in tables so it shouldn't create any significant data on top but only influence the sorting, right? Eg. in Postgres sometimes happen that indexes are larger in size than actual data but that shouldn't be the case here or am I mistaken?
Thanks!

The right PK order allows to reduce data size by column compression.
PK it's not an index it just a sort order of data in columns.
so you have data
field0  field1  field3
---------
1 1 longstring1
2 2 anotherstring1
3 3 thirtdstring1
4 1 longstring2
5 2 anotherstring2
6 3 thirtdstring2

case of PK = field1, field0 the data on the will take less space because of higher compression.
field0  field1  field3
---------
1 1 longstring1
4 1 longstring2
2 2 anotherstring1
5 2 anotherstring2
3 3 thirtdstring1
6 3 thirtdstring2

Thank you very much!


azcopy fails with the message of abort

Which version of the AzCopy was used?
azcopy 7.2.0-netcore
Note: The version is visible when running AzCopy without any argument
Which platform are you using? (ex: Windows, Mac, Linux)
Ubuntu 16.04
What command did you run?
azcopy --source folderA --destination https://xxxx/folderB --exclude-older --dest-key xxxxxxxx== --quiet --parallel-level 32 --resume /tmp/azure.1639be1.jnl --recursive
Note: Please remove the SAS to avoid exposing your credentials. If you cannot remember the exact command, please retrieve it from the beginning of the log file.
What problem was encountered?
/usr/bin/azcopy: line 8: 29564 Aborted                 $DIR/../lib/azcopy/azcopy "$@"
How can we reproduce the problem in the simplest way?
Have you found a mitigation/solution?
re-install it and it works for the first time. But if i upload another data, it fails again. Then, i have to re-install it again.

@amsword
Would you have a try with AzCopy 7.3.0? You can download and install it with following commands:
wget -O azcopy.tar.gz https://aka.ms/downloadazcopylinux64
tar -xf azcopy.tar.gz
sudo ./install.sh

Closing since V7 is no longer being actively maintained. Please give V10 a try here, it's been GA for a bit of time now. Please don't hesitate if you encounter any issue.


[v6] RouteProps type is not exported by react-router-dom package

Version
<EMAIL_ADDRESS>Test Case
codesandbox.io/s/react-router-zkmp8
Steps to reproduce
Try import "RouteProps" from "react-router-dom" library
Expected Behavior
Can be imported like other types.
Actual Behavior
"Module '"../../node_modules/react-router-dom"' has no exported member 'RouteProps'.ts(2305)"

Those are exported by react-router.

@timdorr Shouldn't everything exported by react-router be re-exported by react-router-dom?


How can I set a longer prefix key?

It says "it's not a prefix key"
I'm trying to set
  (persp-mode-prefix-key (kbd "C-M-w p"))

I need to type it in a bit different?

That works for me in the :custom block of use-package. You need to use either that, the Customize UI, or (customize-set-variable 'persp-mode-prefix-key (kbd "C-M-w p")). How are you trying to set it?


SAK-31466 - Following menu if tools do not overflow view port and auto adjustable width tabs

Video Demo
JIRA

@jmarianolujan Mariano, adjusts based on your testing.

It's a great work, it looks nice and useful but if you can share this behaviour changes on Slack channel it would be great in order to avoid some bugs if more people can watch modifications.

Hey Eduardo. Thanks, sure. Is there a Slack ch.? What's the domain?
On Jul 5, 2016 8:42 AM, "Eduardo Rey Jara"<EMAIL_ADDRESS>wrote:

It's a great work, it looks nice and useful but if you can share this
behaviour changes on Slack channel it would be great in order to avoid some
bugs if more people can watch modifications.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/sakaiproject/sakai/pull/2987#issuecomment-230467267,
or mute the thread
https://github.com/notifications/unsubscribe/AEnF-CVaq_4syLheeswvcFCs-e_t64L3ks5qSlFDgaJpZM4JEquB
.

You can find us on https://apereo.slack.com/archives/sakaimorpheusdev


Installing with both homebrew and fink installed

.keyboard%:script/setup
/sw/bin/which: illegal option -- s
Usage: /sw/bin/which [-a] args
Homebrew is required: http://brew.sh/

How to I point the installation script to the Homebrew installation?

How to I point the installation script to the Homebrew installation?

:wave: Hi @dnstarostin: I've never seen this problem before, so I'm not exactly sure how to resolve it, but I can offer a potential workaround. I think you can probably get around this issue as follows:

Using your existing Homebrew installation, manually run the brew commands that are present in Brewfile
Comment out the Homebrew-related lines in script/setup
Run script/setup

I think that will do the trick. I hope this helps.

Thank you!!!
This (is the workaround I found: instead of standalone Hammerspoon I
installed it via 'brew install cask'. This way some problems disappeared. I
have not tested further since it is working now.
On Sun, Sep 1, 2019 at 5:39 PM Jason Rudolph<EMAIL_ADDRESS>wrote:

Closed #58 https://github.com/jasonrudolph/keyboard/issues/58.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jasonrudolph/keyboard/issues/58?email_source=notifications&email_token=AKHI46Z3K5JJBV5CWD3UCSDQHPHYRA5CNFSM4ISWGLKKYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOTLX33LI#event-2599402925,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKHI466ULNVVNHPKQZYND4DQHPHYRANCNFSM4ISWGLKA
.


st2actionrunner not logging Mistral workflow "Liveaction completed"

Normal actions,core.local for example, executed by st2actionrunner post a log message when they finish:
$ st2 run core.local cmd='uname -a'
$ grep "Liveaction completed" /var/log/st2/st2actionrunner*.log | grep "core.local" | tail
...
2017-09-13 16:47:01,978 79551376 AUDIT base [-] Liveaction completed (liveaction_db={'status': 'succeeded', 'runner_info': {u'hostname': u'hostname.domain.tld', u'pid': 15696}, 'parameters': {u'cmd': u'uname -a'}, 'action_is_workflow': False, 'start_timestamp': '2017-09-13 20:47:01.415255+00:00', 'callback': {}, 'notify': None, 'result': {'succeeded': True, 'failed': False, 'return_code': 0, 'stderr': '', 'stdout': 'Linux hostname.domain.tld 3.10.0-693.el7.x86_64 #1 SMP Thu Jul 6 19:56:57 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux'}, 'context': {u'user': u'st2admin', u'pack': u'core'}, 'action': u'core.local', 'id': '59b99945032fb6245bb873c4', 'end_timestamp': '2017-09-13 20:47:01.913602+00:00'})

You can see in this log message says liveaction_db={'status': 'succeeded',  so we know it is done and has succeeded.
If we force a failure the log looks like this:
$ st2 run core.local cmd='exit 1'
$ grep "Liveaction completed" /var/log/st2/st2actionrunner*.log | grep "core.local" | tail
2017-09-13 16:52:39,912 70421392 AUDIT base [-] Liveaction completed (liveaction_db={'status': 'failed', 'runner_info': {u'hostname': u'hostname.domain.tld', u'pid': 15690}, 'parameters': {u'cmd': u'exit 1'}, 'action_is_workflow': False, 'start_timestamp': '2017-09-13 20:52:39.280128+00:00', 'callback': {}, 'notify': None, 'result': {'succeeded': False, 'failed': True, 'return_code': 1, 'stderr': '', 'stdout': ''}, 'context': {u'user': u'st2admin', u'pack': u'core'}, 'action': u'core.local', 'id': '59b99a97032fb6245bb873c7', 'end_timestamp': '2017-09-13 20:52:39.857110+00:00'})

Now, if we do the same thing for a workflow examples.mistral-basic:
$ st2 run examples.mistral-basic cmd='uname -a'
$ grep "Liveaction completed" /var/log/st2/st2actionrunner*.log | grep "examples.mistral-basicl" | tail
2017-09-13 16:54:29,416 49003568 AUDIT base [-] Liveaction completed (liveaction_db={'status': 'running', 'runner_info': {u'hostname': u'hostname.domain.tld', u'pid': 15692}, 'parameters': {u'cmd': u'uname -a'}, 'action_is_workflow': True, 'start_timestamp': '2017-09-13 20:54:28.605902+00:00', 'callback': {}, 'notify': None, 'result': {'tasks': []}, 'context': {u'pack': u'examples', u'user': u'st2admin', 'mistral': {'workflow_name': u'examples.mistral-basic', 'execution_id': '78762aa4-0623-46d2-a1c0-c7a216b8022a'}}, 'action': u'examples.mistral-basic', 'id': '59b99b04032fb6245bb873ca', 'end_timestamp': None})
......
2017-09-13 16:54:30,376 69381008 AUDIT base [-] Liveaction completed (liveaction_db={'status': 'succeeded', 'runner_info': {u'hostname': u'hostname.domain.tld', u'pid': 15700}, 'parameters': {u'cmd': u'uname -a', u'timeout': 60}, 'action_is_workflow': False, 'start_timestamp': '2017-09-13 20:54:29.408990+00:00', 'callback': {u'url': u'http://<IP_ADDRESS>:8989/v2/action_executions/e336c965-5680-4037-a5ec-5640bed3c069', u'source': u'mistral_v2'}, 'notify': None, 'result': {'succeeded': True, 'failed': False, 'return_code': 0, 'stderr': '', 'stdout': 'Linux hostname.domain.tld 3.10.0-693.el7.x86_64 #1 SMP Thu Jul 6 19:56:57 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux'}, 'context': {u'mistral': {u'action_execution_id': u'e336c965-5680-4037-a5ec-5640bed3c069', u'task_id': u'9cc7c95e-384b-4d4e-af81-6c9d86c2a44d', u'workflow_name': u'examples.mistral-basic', u'workflow_execution_id': u'78762aa4-0623-46d2-a1c0-c7a216b8022a', u'task_tags': None, u'task_name': u'task1', u'callback_url': u'/v2/action_executions/e336c965-5680-4037-a5ec-5640bed3c069'}, u'user': u'st2admin', u'parent': {u'user': u'st2admin', u'execution_id': u'59b99b04032fb6245bb873cb', u'pack': u'examples'}, u'pack': u'core'}, 'action': u'core.local', 'id': '59b99b05032fb6245bb873cd', 'end_timestamp': '2017-09-13 20:54:30.110157+00:00'})

You can see that the workflow's action goes into the 'running' state, but we never got a log for the 'succeeded' state. The last log message is the state suceeded for the core.local action.

Dug into this a bit - this isn't ideal from a UX perspective but it is expected, due to the asynchronous nature of how Mistral workflows complete and report their final status (i.e. in the callback handler).
I'll see if I can come up with a solution that accomplishes what you're looking for - likely just some additional logging in the mistral callback module.

@Mierdin my usecase for this is around metrics collection based on logs discussed here: https://docs.stackstorm.com/reference/monitoring.html?highlight=metrics#completed-actions
We use fluentd to ship the audit logs to Graylog. We're trying to create metrics for: number of actions run, number of successful actions, number of failed actions, number of workflows, number of successful workflows, number of failed workflows.
The action queries were fairly easier to track, however the workflow ones were difficult because of the problem discussed in the first post.

Have you looked in st2resultstracker.log? Mistral workflows are inherently updated within st2 in an asynchronous fashion.
The problem is that the log message in question that says Liveaction completed runs well before st2resultstracker has had a chance to query Mistral and get an updated workflow status. It will always report running, because the function doesn't block until workflow completion, only until the workflow request is sent to Mistral
So if you are trying to track the number of workflow completions, st2resultstracker.log seems like a better candidate to me. Thoughts?

Note that eventually st2resultstracker will go away once Mistral has native callbacks but I haven't stayed up to date on that conversation so that may be a ways away still

Per our slack conversation, the best solution for now will be to add additional metadata to existing logging statements in the results tracker. The nature of how mistral workflows are executed make it impractical to try to display this information in a timely manner in the actionrunner logs. So I'll follow up with a PR to the results tracker code that exposes the liveaction_db data you are looking for.


some fixes for input

add spellcheck false
do not set input value if it is the same to prevent jumping to end of input

thx


Remove last unwrap or else and only use SigstoreError

thanks :). I would prefer the commits squashed into 2 or 2, as the follow-up commits are trivial fixes.

I use the github squash and merge feature.


Add postgres backend for NBXplorer / BTCPayServer

So far we have an older version of NBXplorer pinned: #3055 #3217
Help would be appreciated to have postgres set up automatically.
Keep in mind that postgres can potentially serve the lightning instances as well.

just bump nbxplorer, and add those env variables: (replace with your own connection string if database is different)
      NBXPLORER_POSTGRES: User ID=postgres;Host=postgres;Port=5432;Application Name=nbxplorer;MaxPoolSize=20;Database=nbxplorer${NBITCOIN_NETWORK:-regtest}
      NBXPLORER_AUTOMIGRATE: 1
      NBXPLORER_NOMIGRATEEVTS: 1

Note that AUTOMIGRATE will just be ignored for new installs, so you can safely set that up for all.
In BTCPay, you need to link the database with the EXPLORERPOSTGRES.
BTCPAY_EXPLORERPOSTGRES: User ID=postgres;Host=postgres;Port=5432;Application Name=btcpayserver;MaxPoolSize=80;Database=nbxplorer${NBITCOIN_NETWORK:-regtest}

You know it's working if you can see btcpayserver wallet graph.

version is nicolasdorier/nbxplorer:2.3.28

Ping me if you have any issues. Though I didn't have any issues since I auto migrated people. Got a few perf issues on some big instances, but this has been fixed, so I doubt you'll have any issue.

version is nicolasdorier/nbxplorer:2.3.28

we are using https://github.com/dgarage/NBXplorer. That's the right repo, right?
https://github.com/dgarage/NBXplorer/releases/tag/v2.3.28

yes, sorry I was linking the docker image

The command line arguments for these (using them in the systemd service):
NBXPLORER_POSTGRES: User ID=postgres;Host=postgres;Port=5432;Application Name=nbxplorer;MaxPoolSize=20;Database=nbxplorer${NBITCOIN_NETWORK:-regtest}
NBXPLORER_AUTOMIGRATE: 1
NBXPLORER_NOMIGRATEEVTS: 1

--postgres User ID=postgres;Host=postgres;Port=5432;Application Name=nbxplorer;MaxPoolSize=20;Database=nbxplorermainnet
--automigrate 1
--nomigrateevts 1

yes. Note systemd can also set environment variable.  (be careful there are spaces in the connection string)

@NicolasDorier are you planning continue to PGP sign the BTCpay releases as well (had some signatures form Kukks before)?

you mean for signing the release? I don't pay attention to it, git commit is signing automatically. Sometimes Kukks can make releases.

you mean for signing the release? I don't pay attention to it, git commit is signing automatically. Sometimes Kukks can make releases.

yes, we verify the tags - which is really just the signature of the last git commit. The issue is this being a random person (from a PR maybe) is that we cannot verify the updates.

I used the setting for postgres in the /home/btcpay/.nbxplorer/Main/settings.config for NBXplorer:
postgres='User ID=postgres;Host=postgres;Port=5432;Application Name=nbxplorer;MaxPoolSize=20;Database=nbxplorermainnet'
automigrate=1
nomigrateevts=1

Getting the error:
info: NBXplorer.DatabaseSetup: Postgres services activated
dotnet[3263134]: Unhandled exception. System.ArgumentException: Format of the initialization s
tring does not conform to specification starting at index 84.

Will need to initialize the postgres database. Do you have a script for that already?

no I guess the problem come from the quote '

@openoms if you use settings.config, remove quote.

ok, works without quotes..
to fix
Unhandled exception. System.Net.Internals.SocketExceptionFactory+ExtendedSocketException (00000005, 0xFFFDFFFF): Name or service not known

needed to set the
Host=postgres; to Host=localhost;
since not dockerised.
but now expects a password:
Unhandled exception. Npgsql.NpgsqlException (0x80004005): No password has been provided but the backend requires one (in MD5)

It would be ideal to password protect the btcpay database.
will also need to:

[ ] symlink the postgres data directory (/var/lib/postgresql/13/main/) to the disk

What worked with a hardcoded password raspiblitz so far:
Install postgres:
sudo apt install -y postgresql
sudo systemctl enable postgresql
sudo systemctl start postgresql

Autogenerate the database:
sudo -u postgres psql -c "create database nbxplorermainnet;"
sudo -u postgres psql -c "create user btcpay with encrypted password 'raspiblitz';"
sudo -u postgres psql -c "alter user btcpay with encrypted password 'raspiblitz';"
sudo -u postgres psql -c "grant all privileges on database nbxplorermainnet to btcpay;"

NBXplorer config:
echo "\
network=mainnet
btc.rpc.user=raspibolt
btc.rpc.password=PASSWORD_B
postgres=User ID=btcpay;Host=localhost;Port=5432;Application Name=nbxplorer;MaxPoolSize=20;Database=nbxplorermainnet;Password='raspiblitz';
automigrate=1
nomigrateevts=1
" | sudo -u btcpay tee  /home/btcpay/.nbxplorer/Main/settings.config

₿ sudo journalctl -fu nbxplorer -n 100
...
info: NBXplorer.PostgresMigration: The migration completed in 0 minutes.
...


Message key deserialization issue since upgrading Kafka-ui versions across all environments

Issue submitter TODO list

[X] I've looked up my issue in FAQ
[X] I've searched for an already existing issues here
[X] I've tried running master-labeled docker image and the issue still persists there
[X] I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)
We are seeing a  message key deserialization issue since upgrading Kafka-ui versions, across all environments, on all topics.
Expected behavior
Message key should deserialize when opening a topic.
Your installation details
b0c367c
v0.6.2
Version- v0.6.2
Steps to reproduce

Open Kafka UI
Select a topic with messages on it
You will see the message key is not deserialized, and when clicking on the actual message key, it still shows the special chars.

Screenshots

Logs
No response
Additional context
No response

Hi,
when you expand the message, which serdes are displayed there?
Also, have you tried different serdes (key serde, value serde fields)?

First off, I just want to thank everyone involved in this wonderful project!!!
@zachgius9 and I work together. Adding the following for context. Hope this helps.

We recently upgraded from v0.4.0 to v0.6.2

Prior to the upgrade keys and values were able to deserialize

The schema registry is configured and schemas are visible when on the "Schema Registry" tab.
Both the Key and Value Serdes are listed as SchemaRegistry, see screenshot below
We are using the Confluent Schema Registry
We are using Confluent's schema contexts, e.g. https://psrc-id.region.provider.confluent.cloud/contexts/.my-context as the REST URL in the config
Keys seem to be having issues, but values deserialize without a problem
It is possible that we missed something when upgrading the config for Kafka-UI. We are seeing some messages in the bootup logs (attached)

Thank you!!!

kafka-ui-66966458db-h2gtb.log

After some additional research, here are the findings

The previous info is not fully accurate as the instance of Kafka-UI was passing through a proxy
Kafka-UI fails to list schemas when on the "Schema Registry" tab when using a Schema Registy URI containing a Confluent Schema Context

example Schema Registry URI with a schema context https://psrc-id.region.provider.confluent.cloud/contexts/.my-context

Kafka-UI fails to deserialize keys when using a Schema Registry URI containing a Confluent Schema Context

example Schema Registry URI with NO schema context https://psrc-id.region.provider.confluent.cloud

Kafka-UI has no issues when using a Confluent Schema Registry URI with NO schema context
This seems like a parsing issue for the URL due to schema contexts containing the following characters, /.:
It is desirable that Kafka-UI support schema context URI's so that a user's view is limited to only schemas within a particular context rather than schemas in all contexts

Hope this helps clarify the root issue. Kafka-UI does not correctly support Schema Registry URI's using a Confluent Schema Context.
Thank you!

@immutablesemaphore thank you for deep investigation, we will take a look and provide fix soon!

@immutablesemaphore we discussed your issue internally - we will provide contexts support with UI in next versions.
For now we suggest you

Set schemaRegistry url to default context to see all subjects in UI table
For each context configure separate serde like this:

      kafka.clusters.0.serde.0.name: SchemaRegistry (ctx 1)
      kafka.clusters.0.serde.0.className: com.provectus.kafka.ui.serdes.builtin.sr.SchemaRegistrySerde
      kafka.clusters.0.serde.0.properties.url: http://schema-registry0:8085/contexts/.myctx1

      kafka.clusters.0.serde.1.name: SchemaRegistry (cxt 2)
      kafka.clusters.0.serde.1.className: com.provectus.kafka.ui.serdes.builtin.sr.SchemaRegistrySerde
      kafka.clusters.0.serde.1.properties.url: http://schema-registry0:8085/contexts/.myctx2

for other serdes configs you can check serialization configuration doc

@iliax Thank you!


Export Benchmark Results

Export the benchmark results from a project to a predefined location.
Tasks:

[ ] write an exporter which receives an assembly as parameter and finds the corresponding benchmark results.
[ ] configure exporter output format via appsettings: graphs, html, json, txt
[ ] configure the output location via appsettings

Also, I'll update the design documents with architecture decisions taken while implementing this story.


cutoff of recommendations result?

I use the fellow command to output the predicted result.

python recommend.py ../examples/rbps_new.h5  ../examples/utrs.h5 ../examples/trained_from_AURA_HT.pkl --output ../examples/recommendations.txt --standardize-Fr

RBP     target  y_hat   serendipity
Pcbp2      uc008jda.1_3UTR_Tor1b   0.992   1.00
Fus           uc011wjf.1_3UTR_Fam168b 0.979   1.00
Fus           uc007apj.2_3UTR_Fam168b 0.979   1.00
Fus           uc011wje.1_3UTR_Fam168b 0.979   1.00
.....
Fus           uc009ouj.1_5UTR_Tmem218 0.000   1.00
Fus           uc007yxk.2_5UTR_Bdh1    0.000   1.00
Fus           uc008qoy.2_5UTR_Fmo5    0.000   1.00
Pcbp2      uc009fqd.1_5UTR_Phldb3  0.000   1.00
Rbm3       uc009qqk.3_3UTR_Ccpg1os 0.000   1.00

So, how to set the cutoff to select the high-credible interactions of RBPs and  RNA?

RNAcommender outputs a ranking. The higher the prediction score, the more accurate the prediction should be. Anyway the prediction score is in the range [0,1] (obtained with a sigmoid).

The threshold value may vary depending on the considered dataset. In the paper we did not analyse the relationship between a selected threshold and the corresponding accuracy of the method, especially because it is not straightforward to set a threshold. If you need cast the ranking problem analysed in RNAcommender into a strict classification task where you identify positives and negatives you may try two options:

Select a fixed threshold (e.g. 0.5), or a different value depending on your needs regarding the precision/recall tradeoff.
Find the threshold that gives an acceptable accuracy on a validation set extracted from the dataset

Cheers.


[ITensors][Bug] Broadcast function in ITensors fails to compile on GPU

Description
Aleix has found an issue which arises from the use of VectorInterface with GPU based ITensors.  The problem is a broadcast function
 a.= a .* l .+ b .* k
is being called and the anonymous function being generated is not able to be parsed on GPU. One way I could see to fix this is to call permute_dims!! alternatively I could split the function into two broadcasts.  Right now I have a patch in VectorInterfaces. Below I have a pseudocode that can be added to ITensors and tested using JLArrays when we have that integrated.
Simple Example
using ITensors: Index, random_itensor
using Metal: mtl
using NDTensors: tensor

elt = Float32
A = random_itensor(elt, Index(5))
A .= A .* elt(1.0)
A .= A .* elt(1.0) .+ A .* elt(2.0)

mA = mtl(A)
mA .= mA .* elt(1.0) ## This works fine
mA .= mA .* elt(1.0) .+ mA .* elt(2.0) ## This fails with scalar indexing
mt = tensor(mA);
mt = mt .* elt(1.0)

Checklist:

[x] Update the broadcast function
[ ] Add tests to ITensorVectorInterfaceExt library which can be tested later with JLArrays

@emstoudenmire Hi Miles, I was able to make a minimal example of the issue. The code that is causing an issue is here https://github.com/ITensor/ITensors.jl/blob/1ef12d6bd1b6a3161a9d3308efc8b976e2389de6/src/broadcast.jl#L417

@kmp5VT I remember we previously faced issues with the GPU compiler struggling with certain ITensor broadcast operations that were written in terms of anonymous functions, for example see the changes to broadcast.jl in this PR: #1194.
It seems like you were able to do pretty modest rewrites of some of the ITensor broadcast functions to circumvent those issues but I don't remember the details, perhaps something similar can be done here.
One of my suggestions at the time of #1994 was that you can always rewrite a closure as a callable struct (see the Julia docs on closures: https://docs.julialang.org/en/v1/devdocs/functions/#Closures), which may be easier for the GPU compiler to compile, if there isn't a simpler way to refactor the code to help the GPU compiler.

@mtfishman  Thank you for the suggestion, I know we had made a PR to fix broadcasts before and it was helpful to have that as a reference. I tried the pattern we used in #1194 but the GPU compiler still complains that the values \alpha and \beta are not bits type. If I split the line into 2 broadcast calls that seems to work with minimal overhead.  Though if you think it would better, I can modify the code and make a closure.
Thanks again!

I don't think we should split it into two lines because that is less performant, so you will have to find another approach, whether that is making a struct that implements the closure or something else.

Worst case, we can have two code branches, one for CPU that leaves the code as it is and one for GPU that splits the broadcast call into two calls, but hopefully there is a way to rewrite it so that it involves just one broadcast call and works on CPU and GPU.

Here is a demonstration that your new version is slower (by roughly a factor of 1.5 in this case):
function f1!(T, A, α, β)
  T .= β .* T .+ α .* A
  return T
end

function f2!(T, A, α, β)
  T .= β .* T
  T .+= α .* A
  return T
end

T = randn(10^8)
A = randn(10^8)
α = 2.0
β = 3.0

using BenchmarkTools: @btime
T1 = copy(T)
@btime f1!($T1, $A, $α, $β)
T2 = copy(T)
@btime f2!($T2, $A, $α, $β)

which outputs:
  27.130 ms (0 allocations: 0 bytes)
  42.336 ms (0 allocations: 0 bytes)

It doesn't use more memory since it reuses the existing allocated memory, but does require more FLOPS since it isn't fusing all of the operations into a single loop.

@mtfishman This makes sense as every element in T will have to be modified twice and would be especially bad if \beta = 1 . I ran btime but didn't think to look at the actual time difference between the two calls on CPU. I will work on code that works on CPU and GPU without splitting the line up. Thanks!

@kmp5VT it looks like the test failures are caused by Julia 1.11, which was just released and our tests automatically started testing against that.
Let's fix those issues in a separate PR from this one, and this and other open PRs can be based on those fixes once it is merged.
The ITensorMPS failure looks like it is caused by a change to the random number generator, in which case it can be fixed by switching to using StableRNGs.jl in the tests.
The NDTensors failure looks like some kind of slicing issue in NDTensors.BlockSparseArrays. That's not being used by end users anyway so could be ignored for now, maybe we could just disable those tests for now (mark them as broken) so they don't produce so much noise in the CI log and confusing failures in PRs. Though potentially that failure in in BlockSparseArrays caused by Julia 1.11 could cause issues for @ogauthe, as far as I know he is the only one really using that library right now (he's using it for his non-abelian fusion tensor work).
Julia 1.10 became the LTS (https://discourse.julialang.org/t/julia-v1-11-0-has-been-released-and-v1-10-is-now-lts/121064) so we can also change that to the oldest Julia version we support and just test against Julia 1.10 and 1.11, but that should probably be done in a PR of its own since that is something we will want to announce to users (I don't think it needs a breaking release of ITensor packages to change that but I have to think about that...).

@kmp5VT it looks like the test failures are caused by Julia 1.11, which was just released and our tests automatically started testing against that.
Oh I see, I was running on my mac with metal and see that the tests are passing. I am making an issue as a reminder to update our tests.

The ITensorMPS failure looks like it is caused by a change to the random number generator, in which case it can be fixed by switching to using StableRNGs.jl in the tests.
Okay I can try to do this as well in another PR. I can also open an issue as a reminder.

The NDTensors failure looks like some kind of slicing issue in NDTensors.BlockSparseArrays. That's not being used by end users anyway so could be ignored for now, maybe we could just disable those tests for now (mark them as broken) so they don't produce so much noise in the CI log and confusing failures in PRs. Though potentially that failure in in BlockSparseArrays caused by Julia 1.11 could cause issues for @ogauthe, as far as I know he is the only one really using that library right now (he's using it for his non-abelian fusion tensor work).
This is very useful information, thanks!

@kmp5VT all of the test failures in this PR are in Julia 1.11, I see the Julia 1.6 tests pass. So I would be fine merging this once it is ready, say before #1539 is merged.
Is the original issue you were trying to address fixed? If so, could you add a test?

@mtfishman yes the original issue is fixed now. It looks like there is a test here in the ITensor test suite but the issue was related to using GPU's with ITensor. Currently we are using GPUs in the ITensors suite so I am not really sure where to put a test. Do you have a suggestion?

Thanks for the reminder that we're not running the ITensors.jl tests on GPU right now.
map!(axpby(α, β), T, T, A) gets lowered to some call in NDTensors.jl, probably a call to permutedims!. Can you add a test of that permutedims! call in the NDTensors.jl tests?

Note that the original issue may be related to more general issues that closures can have in Julia:

https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-captured
https://github.com/JuliaLang/julia/issues/15276
https://github.com/c42f/FastClosures.jl

i.e. closures can have some type stability issues in certain cases, and certain GPU backends may have trouble compiling functions with type instabilities. Just pointing that out to potentially give some explanation and broader context for the original issue. Using a let block or FastClosures.jl could be alternative solutions for future issues like this, since they may be more general/automated than the current solution of making a custom callable object.


containerd+kata shim-v2: shim not killed, artifacts left on host

Description of problem
With kubernetes 1.14, containerd 1.2.5 and Kata artifacts from the 1.6.0 release, kubectl delete -f doesn't result in the containerd-shim-kata-v2 process being stopped, and artifacts are left on the host in /var/run/vc/sbs, /var/run/vc/vm, etc.
example, starting with a clean containerd+k8s setup, install kata (1.6.0):
install kata 1.6
kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/kata-rbac.yaml
kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/kata-deploy.yaml

setup appropriate runtimeclass:
kubectl apply -f https://raw.githubusercontent.com/clearlinux/cloud-native-setup/master/clr-k8s-examples/8-kata/kata-qemu-runtimeClass.yaml

in seperate windows, watch for qemu, v2-shim, and monitor the filesystem where Kata stores state:
watch -d 'ls /var/run/vc/sbs/ /var/lib/vc/sbs /var/run/vc/vm/ '
watch -d 'ps -aef | grep -E "qem|containerd-shim-kata-v2" | grep -v grep'

start a pod, then remove after started:
kubectl apply -f https://gist.githubusercontent.com/egernst/9cbbb3d6ba566afca98be2a4e29328d6/raw/0f6e332ba1b7306ed895e5ff91b2939ca3f0bb42/two-pod.yaml

kubectl delete -f https://gist.githubusercontent.com/egernst/9cbbb3d6ba566afca98be2a4e29328d6/raw/0f6e332ba1b7306ed895e5ff91b2939ca3f0bb42/two-pod.yaml

You'll see QEMU come and go as expected, but the v2-shim keep running and storage stay on the filesystem.

/cc @lifupan @mcastelino @sboeuf @bergwolf

Some logs around time of deletion:
containerd[5344]:  level=debug msg="sending request" ID=53ffff95b545f254cc36058df22f2442f584a286393b1115fc408023cf5609fd name=grpc.StatsContainerRequest req="container_id:\"6261346f506314999a6d4e94b84dd09d050d7763502827eca9ae334103bdd67a\" " source=virtcontainers subsystem=kata_agent
containerd[5344]:  level=info msg="ExecSync for "7b19ae263fc1870052876ec315542d907f8fbf73a3e7d42bf256fff6c7ac0df0" with command [/bin/sh -ec ETCDCTL_API=3 etcdctl --endpoints=https://[<IP_ADDRESS>]:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt --key=/etc/kubernetes/pki/etcd/healthcheck-client.key get foo] and timeout 15 (s)"
containerd[5344]:  level=info msg="Exec process "141f5299e65d21a02c1e80bf1625c4cc78f4c32c29284cbd78ca705baec9632f" exits with exit code 0 and error <nil>"
containerd[5344]:  level=info msg="Finish piping "stderr" of container exec "141f5299e65d21a02c1e80bf1625c4cc78f4c32c29284cbd78ca705baec9632f""
containerd[5344]:  level=info msg="Finish piping "stdout" of container exec "141f5299e65d21a02c1e80bf1625c4cc78f4c32c29284cbd78ca705baec9632f""
containerd[5344]:  level=info msg="ExecSync for "7b19ae263fc1870052876ec315542d907f8fbf73a3e7d42bf256fff6c7ac0df0" returns with exit code 0"
containerd[5344]:  level=error msg="StopPodSandbox for "eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f" failed" error="failed to stop sandbox container "eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f" in '\x01' state: failed to kill sandbox container: all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing rpc error: code = DeadlineExceeded desc = timed out connecting to unix socket ////run/vc/vm/eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f/kata.sock": unavailable"
kubelet[32262]: E0403 22:14:26.962281   32262 remote_runtime.go:132] StopPodSandbox "eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f" from runtime service failed: rpc error: code = Unknown desc = failed to stop sandbox container "eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f" in '\x01' state: failed to kill sandbox container: all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing rpc error: code = DeadlineExceeded desc = timed out connecting to unix socket ////run/vc/vm/eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f/kata.sock": unavailable
kubelet[32262]: E0403 22:14:26.962348   32262 kuberuntime_manager.go:846] Failed to stop sandbox {"containerd" "eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f"}
kubelet[32262]: E0403 22:14:26.962397   32262 kubelet_pods.go:1093] Failed killing the pod "two-containers-kata-guaranteed": failed to "KillPodSandbox" for "f817d5cf-565a-11e9-bdb5-000d3a6d0876" with KillPodSandboxError: "rpc error: code = Unknown desc = failed to stop sandbox container \"eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f\" in '\\x01' state: failed to kill sandbox container: all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing rpc error: code = DeadlineExceeded desc = timed out connecting to unix socket ////run/vc/vm/eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f/kata.sock\": unavailable"
containerd[5344]:  level=info msg="StopPodSandbox for "eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f""
containerd[5344]: 2019-04-03 22:14:28.728 [INFO][27671] calico.go 442: Extracted identifiers ContainerID="eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f" Node="kata-k8s-containerd" Orchestrator="k8s" WorkloadEndpoint="kata--k8s--containerd-k8s-two--containers--kata--guaranteed-eth0"
containerd[5344]: 2019-04-03 22:14:28.751 [WARNING][27671] workloadendpoint.go 70: Operation Delete is not supported on WorkloadEndpoint type
containerd[5344]: 2019-04-03 22:14:28.751 [INFO][27671] k8s.go 474: Endpoint deletion will be handled by Kubernetes deletion of the Pod. ContainerID="eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"kata--k8s--containerd-k8s-two--containers--kata--guaranteed-eth0", GenerateName:"", Namespace:"default", SelfLink:"", UID:"cc1fcdf1-565d-11e9-bdb5-000d3a6d0876", ResourceVersion:"2657", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63689926446, loc:(*time.Location)(0x1edd720)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"projectcalico.org/serviceaccount":"default", "projectcalico.org/namespace":"default", "projectcalico.org/orchestrator":"k8s"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"kata-k8s-containerd", ContainerID:"", Pod:"two-containers-kata-guaranteed", Endpoint:"eth0", IPNetworks:[]string{"<IP_ADDRESS>/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.default", "ksa.default.default"}, InterfaceName:"cali0232257877e", MAC:"", Ports:[]v3.EndpointPort(nil)}}
containerd[5344]: 2019-04-03 22:14:28.751 [INFO][27671] k8s.go 481: Releasing IP address(es) ContainerID="eaad9157b27429205bfbd5eab91c8a4953899e5790e6e821ad931328859e3a1f"

more complete logs here: https://gist.githubusercontent.com/egernst/9b01743a1f4cf84f84689c5a7df297ed/raw/7df24992bcbb18fb9ae50da67942fcd40f71b9be/issue-1480-logs.md

@egernst could you try with Kata 1.5 or 1.4 to check if you can reproduce? Let's try to identify if this is coming from something that would have changed in Kata.
If that's not the case, then I suspect we need to look into containerd. The same way, you could try this with different version of containerd to find the culprit.

Already tried with containerd 1.2.4; can reproduce.
It is slightly different terrible behavior in 1.5.2:
The QEMU and v2 shim process remain, as well as items on filesystem, after deleting.
However...  after deleting:

If I re-apply the same workload again, I see that the 'dead' one is cleaned up during the starting of the new one.
If I re-apply a different workload, the original is still left on the system, and a second QEMU/v2-shim process is started, and both the deleted and new one are stored on the filesystem.

log for scenario w/ 1.5.y and 1.4.y: https://gist.githubusercontent.com/egernst/0deaf8542b5617b7e0210b04d789ba91/raw/fa92014abdd39740bb56dba17f447f251fedc0eb/log%2520using%25201.4

@bergwolf @teawater - curious if either of you can reproduce this?

This only occurs with macvlan.  Not an issue with tc-mirror.
/cc @mcastelino

@egernst this sounds like something is going wrong with the network deletion then. Make sure we check the traces related to this, it'll help narrowing down faster the issue.
/cc @lifupan

I am now using kata 1.6 and have the same issue.
There are kata-v2 shim process left after pod deleted.
The network mode is macvtap, then I changed to bridged, the issue still the exists.

Hi @wenlxie what's network plugin you used for containerd? standard cni or ?

@egernst this sounds like something is going wrong with the network deletion then. Make sure we check the traces related to this, it'll help narrowing down faster the issue.
/cc @lifupan

I agree with @sboeuf , It seems this issue is related with calico network plugin, I extracted the key log as below:
containerd[5344]: 2019-04-03 22:25:13.882 [INFO][23861] k8s.go 485: Cleaning up netns ContainerID="ac5a7b5ac24812c11253594e1bed082a668d2f3fed829b6d56fe33701e3b1c4b"
containerd[5344]: Calico CNI deleting device in netns /var/run/netns/cni-6637d21d-3031-921d-6467-e0e66692a8f2
containerd[5344]:  level=error msg="StopPodSandbox for "ac5a7b5ac24812c11253594e1bed082a668d2f3fed829b6d56fe33701e3b1c4b" failed" error="failed to destroy network for sandbox "ac5a7b5ac24812c11253594e1bed082a668d2f3fed829b6d56fe33701e3b1c4b": failed to get IP addresses for "eth0": <nil>"
kubelet[32262]: E0403 22:25:13.918205   32262 remote_runtime.go:132] StopPodSandbox "ac5a7b5ac24812c11253594e1bed082a668d2f3fed829b6d56fe33701e3b1c4b" from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "ac5a7b5ac24812c11253594e1bed082a668d2f3fed829b6d56fe33701e3b1c4b": failed to get IP addresses for "eth0": <nil>
kubelet[32262]: E0403 22:25:13.918283   32262 kuberuntime_manager.go:846] Failed to stop sandbox {"containerd" "ac5a7b5ac24812c11253594e1bed082a668d2f3fed829b6d56fe33701e3b1c4b"}

From the log, we can see that calico would like to destroy a network namespace by get the interface "eth0" IP first, but for kata's macvtap and bridge mode, the interface "eth0" IP has been passed into qemu, thus the calico failed to get the IP which will cause containerd failed to stop the sandbox, thus
containerd will not send Shutdown() cmd which responsible fro shutdown shimv2 process.

@lifupan Our own cni network plugin
We have the same logical to teardown the pods network , which get the ip addr from eth0
But we will get from the pod first. If the pod not exit, then will try to get it from eth0 in the netns.

@wenlxie -- tc-mirror is the ideal configuration.  Can you test with this?

@amshinde the issue seems to be that we may not be recovering all the interface information on release. With tc mirror we leave the host side interface and network namespace completely untouched. Hence there is no need to recover any namespace or interface programming.
We should make tc mode the default going forward as it reduces the scope of issues such as this.

@mcastelino @egernst @lifupan I recall that cri-containerd was caching the IP address of the eth0 interface, while looking at the cri code long back.
Here is the issue for it:
https://github.com/containerd/cri/issues/524
And this is the fix they had in place for the above issue"
https://github.com/containerd/cri/blob/b23b406fedd2009f813c36206865ae2a8c51d5bb/pkg/server/sandbox_run.go#L128
They had a workaround to cache the IP address which they say led to a subtle performance improvement as well.
I guess the above workaround is no longer in place for the shimv2 path (havent traced the shimv2 flow).
@lifupan If you are already taking a look at this, can you confirm this.
I wonder if it is worth fixing this, to benefit from slightly better performance with macvtap.
Yes, but I agree we should move to tc as default model. I'll raise a PR to do so.

@amshinde @mcastelino @egernst
Tc mirroring should be the only configuration, hence the default one.
That being said, I think this is one of the items that need to go to the list of things to make happen in 2.0 release.
I don't think we care enough about the very small performance difference between macvtap and tc-mirroring to keep this in the long term.
This is going to be part of reducing Kata complexity, but I don't see this happening before 2.0, if we care about backward compatibility (and I think we do).

Is there a difference from user perspective, aside from improved stability?

Is there a difference from user perspective, aside from improved stability?

@egernst the switch should be invisible from a user POV. So this is not a breaking change. IMO we should push this early vs waiting for 2.0

@egernst Seems there is no docs for the details of these network modes.

@WeiZhang555 need your input here please.

@egernst Could you update your calico cni plugin and see if it still reproduces? I looked a bit at it and I think the issue is fixed by https://github.com/projectcalico/cni-plugin/commit/6fef469abf670574464dcbbf6f593d294927f109#diff-cd575165f84eed10d10eb1460b0927b4L120 -- DelLinkByNameAddr does check the endpoint ip address where as DelLinkByName doesn't. calico releases contain it since v3.4.0 (latest is v3.6.1).
That said, I do agree with @sboeuf @mcastelino @amshinde that we should move to tc as default, as it is a very subtle change that even calico didn't mention it in the commit message.

I didn't get to test this yet, my apologies @bergwolf

I believe we are running into this issue with our setup.
It tends to occur when we schedule a large amount of pods on a node, 60+ pods at a time.
We have found that the processes within the pods are completing, but the VM for the pod isn't being cleaned up. The qemu-system processes continue running, as well as the containerd-shim-kata-v2 process continues running. We can also see the sandbox containers are still in containerd.
The containerd-shim-kata-v2 is also consuming alot of CPU when this occurs, each containerd-shim-kata-v2 consumes 100% of a CPU core and with 40+ containerd-shim-kata-v2 processes, 40+ cores are at 100% usage.
This is resulting in the pod CIDR IP addresses being exhausted and new pods aren't able to start on the node.
We are seeing the following log lines when this occurs:
Apr 29 03:39:38 ip-10-155-106-161.ec2.internal containerd[3100]: time="2019-04-29T03:39:38.879351593Z" level=error msg="StopPodSandbox for \"22abc9d98dfa2585d2484d931c6e931dc2ab872081f76926b72908e19cb9771a\" failed" error="failed to stop sandbox container \"22abc9d98dfa2585d2484d931c6e931dc2ab872081f76926b72908e19cb9771a\": failed to kill sandbox container: all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing rpc error: code = DeadlineExceeded desc = timed out connecting to unix socket ////run/vc/vm/22abc9d98dfa2585d2484d931c6e931dc2ab872081f76926b72908e19cb9771a/kata.sock\": unavailable"

We are using the following versions:

Containerd 1.2.6
Kata 1.7.0-alpha1
Kubernetes 1.13.4
Calico 3.5.2

Let me know if you need any more information.

@awprice This sounds like the exact issue.  Can you update your configuration to utilize tcfilter instead of macvtap (update you configuration.toml for Kata)?

@egernst Unfortunately we are already using tcfilter by default with 1.7.0-alpha1 and it hasn't improved the issues.
I've dug up some more logs when this occurs from containerd:
failed to stop sandbox container "fc4781d8206257248df55c4d90fad93e7f50984806b94289459cc2a0130deb64" in '\x01' state: failed to kill sandbox container: Container not ready, running or paused, impossible to signal the container: unknown"

The problem is also prevalent when scheduling a larger amount of pods at once. When scheduling 10 or so at once on a single node, we don't see the problem, but scheduling 100 or more at once brings out the problem straight away.
Could it be a load/scale issue with the Kata components being overwhelmed?


'vm image list' Too many open files

Description
Running $ az vm image list --all -otable fails with Too many open files errors.
I wanted to create a Debian 7 VM and so needed the URN but didn't know what publisher it was under (turns out it's credativ) so I wanted to list all images.
The full command I ran: $ az vm image list --all -otable | grep debian.
This is possibly due to the logic we have to make concurrent API calls.
Error 1:
[Errno 24] Too many open files
Traceback (most recent call last):
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/main.py", line 36, in main
    cmd_result = APPLICATION.execute(args)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/application.py", line 210, in execute
    result = expanded_arg.func(params)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 289, in __call__
    return self.handler(*args, **kwargs)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 488, in _execute_command
    reraise(*sys.exc_info())
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 465, in _execute_command
    result = op(client, **kwargs) if client else op(**kwargs)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/vm/custom.py", line 211, in list_vm_images
    all_images = load_images_thru_services(publisher_name, offer, sku, image_location)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/vm/_actions.py", line 67, in load_images_thru_services
    t.result()  # don't use the result but expose exceptions from the threads
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/concurrent/futures/_base.py", line 422, in result
    return self.__get_result()
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/concurrent/futures/thread.py", line 62, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/vm/_actions.py", line 50, in _load_images_from_publisher
    images = client.virtual_machine_images.list(location, publisher, o.name, s.name)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/mgmt/compute/compute/v2016_04_30_preview/operations/virtual_machine_images_operations.py", line 172, in list
    response = self._client.send(request, header_parameters, **operation_config)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/msrest/service_client.py", line 180, in send
    **kwargs)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/requests/sessions.py", line 513, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/requests/sessions.py", line 623, in send
    r = adapter.send(request, **kwargs)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/connectionpool.py", line 345, in _make_request
    self._validate_conn(conn)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/connectionpool.py", line 844, in _validate_conn
    conn.connect()
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/connection.py", line 326, in connect
    ssl_context=context)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/util/ssl_.py", line 325, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py", line 440, in wrap_socket
    rd = util.wait_for_read(sock, sock.gettimeout())
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/util/wait.py", line 33, in wait_for_read
    return _wait_for_io_events(socks, EVENT_READ, timeout)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/util/wait.py", line 22, in _wait_for_io_events
    with DefaultSelector() as selector:
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/util/selectors.py", line 581, in DefaultSelector
    return _DEFAULT_SELECTOR()
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/util/selectors.py", line 461, in __init__
    self._kqueue = select.kqueue()
IOError: [Errno 24] Too many open files

Error 2:
[('system library', 'fopen', 'Too many open files'), ('BIO routines', 'BIO_new_file', 'system lib'), ('x509 certificate routines', 'X509_load_cert_crl_file', 'system lib')]
Traceback (most recent call last):
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/main.py", line 36, in main
    cmd_result = APPLICATION.execute(args)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/application.py", line 210, in execute
    result = expanded_arg.func(params)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 289, in __call__
    return self.handler(*args, **kwargs)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 488, in _execute_command
    reraise(*sys.exc_info())
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 465, in _execute_command
    result = op(client, **kwargs) if client else op(**kwargs)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/vm/custom.py", line 211, in list_vm_images
    all_images = load_images_thru_services(publisher_name, offer, sku, image_location)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/vm/_actions.py", line 67, in load_images_thru_services
    t.result()  # don't use the result but expose exceptions from the threads
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/concurrent/futures/_base.py", line 422, in result
    return self.__get_result()
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/concurrent/futures/thread.py", line 62, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/vm/_actions.py", line 42, in _load_images_from_publisher
    offers = client.virtual_machine_images.list_offers(location, publisher)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/azure/mgmt/compute/compute/v2016_04_30_preview/operations/virtual_machine_images_operations.py", line 235, in list_offers
    response = self._client.send(request, header_parameters, **operation_config)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/msrest/service_client.py", line 180, in send
    **kwargs)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/requests/sessions.py", line 513, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/requests/sessions.py", line 623, in send
    r = adapter.send(request, **kwargs)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/connectionpool.py", line 345, in _make_request
    self._validate_conn(conn)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/connectionpool.py", line 844, in _validate_conn
    conn.connect()
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/connection.py", line 326, in connect
    ssl_context=context)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/util/ssl_.py", line 309, in ssl_wrap_socket
    context.load_verify_locations(ca_certs, ca_cert_dir)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py", line 413, in load_verify_locations
    self._ctx.load_verify_locations(cafile, capath)
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/OpenSSL/SSL.py", line 669, in load_verify_locations
    _raise_current_error()
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
Error: [('system library', 'fopen', 'Too many open files'), ('BIO routines', 'BIO_new_file', 'system lib'), ('x509 certificate routines', 'X509_load_cert_crl_file', 'system lib')]
Traceback (most recent call last):
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/site.py", line 67, in <module>
    import os
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/os.py", line 49, in <module>
    import posixpath as path
  File "/Users/debekoe/lib/azure-cli/lib/python2.7/posixpath.py", line 15, in <module>
    import stat
ImportError: No module named stat

Environment summary
Install Method: How did you install the CLI? (e.g. pip, interactive script, apt-get, Docker, MSI, nightly)
curl/interactive script
CLI Version: What version of the CLI and modules are installed? (Use az --version)
$ az --version
azure-cli (2.0.7)

acr (2.0.5)
acs (2.0.7)
appservice (0.1.7)
batch (3.0.0)
billing (0.1.0)
cdn (0.0.3)
cloud (2.0.3)
cognitiveservices (0.1.3)
command-modules-nspkg (2.0.0)
component (2.0.5)
configure (2.0.7)
consumption (0.1.0)
core (2.0.7)
cosmosdb (0.1.7)
dla (0.0.7)
dls (0.0.7)
feedback (2.0.3)
find (0.2.3)
interactive (0.3.3)
iot (0.1.6)
keyvault (2.0.5)
lab (0.0.5)
monitor (0.0.5)
network (2.0.7)
nspkg (3.0.0)
profile (2.0.5)
rdbms (0.0.2)
redis (0.2.4)
resource (2.0.7)
role (2.0.5)
sf (1.0.2)
sql (2.0.4)
storage (2.0.7)
vm (2.0.7)

Python (Darwin) 2.7.10 (default, Feb  7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]

Python location '/Users/debekoe/lib/azure-cli/bin/python'

OS Version: What OS and version are you using?
macOS Sierra Version 10.12.5
Shell Type: What shell are you using? (e.g. bash, cmd.exe, Bash on Windows)
bash

The opened socket number exceeded the process limitation. It turned out Azure SDK creates a new session object for each SDK call, so underlined connections (and sockets) are not reused at all. There are tracking issue already at runtime, https://github.com/Azure/msrestazure-for-python/issues/6
For now I will shrink the threads from 40 down to 5. The perf loss is not that much(around 40~ seconds to get back 4300 images). At least this gets thing stabilized. Also I will output a warning if the --all is used w/o any other filters. az image list supports partial matching, so using server filter is highly suggested to get result faster.


gradle always download LATEST-EAP-SNAPSHOT of java-compiler-ant-tasks when use localPath

When I use gradle to build intellij plugin and use localPath config like this:
intellij {
localPath 'D:\Program Files\JetBrains\PyCharm Community Edition 2018.3'
}
The gradle always download the java-compiler-ant-tasks-LATEST-EAP-SNAPSHOT.jar,  not the specific version such as java-compiler-ant-tasks-192.5728.105.jar.
How can let gradle do not download the latest version ?  My maven reposity is in my compony network which can not connect the internet,  and I can only put specific version like java-compiler-ant-tasks-192.5728.105.jar in my maven repository, because the maven repository only accepte release version.
Thanks

PyCharm Community Edition 2018.3 cannot have 192.5728.105 build number, your expectations are wrong.
We do not have compiler build for 183.* builds

How can let gradle do not download the latest version ?

see the documentation: https://github.com/jetbrains/gradle-intellij-plugin#instrumenting-dsl

@zolotov sorry,  the version number in my question is just an example.   Actually, I use 2019.2 not 2018.3
intellij {
localPath 'D:\Program Files\JetBrains\PyCharm Community Edition 2019.2'
}
And the question is :  the gradle always download the java-compiler-ant-tasks-LATEST-EAP-SNAPSHOT.jar, not the specific version such as java-compiler-ant-tasks-192.5728.105.jar
When I use config like this:
intellij {
version 'PC-2019.2'
type 'PC'
}
the gradle will download the java-compiler-ant-tasks-192.5728.105.jar
So why this two kind of config download different jar ?  Thanks

should be fixed in the next 0.5-snapshot, please check it in few hours


Old color scheme?

Hi! Thanks a lot for the awesome theme. I am using it for like 4 years now.
Dotfiles: https://github.com/sobolevn/dotfiles#vs-code
My main question is: after a recent update (1-2 days ago) my color scheme has changed a lot. And I don't like it, sorry 🙂
Let's discuss new problems I see:

From top to bottom:

[x] New green highlight of (), which was not used before. It now has too many "attention" attracted to it
[ ] Purple function arguments and constants, it just does not fit the color scheme
[ ] New color of message panel on hover is too light

Related https://github.com/dempfi/ayu/issues/250
Is there something you can suggest here?
Or should I just fork it?

4 years is really a lot of time. I'm happy to hear people loving and using ayu ❤
Thank for detailed report, it seems all of this can be addressed:

I'll look into toning down a notch bracket highlight. I can see how that can be annoying;
Purple was used for arguments for a couple years now. What has changed tho is that I enabled semantic highlighting which causes this arguments color to be used in places of usage too. For now, I can recommend to disable semantic highlighting
Notifications color prominence is linked to list selection color and since I aimed to make list selections brighter and more readable, that also got applied to notifications; I'll check if it's possible to reduce contrast of those notifications;

@emlautarom1 when you say

Text used to be yellow, and now is pink-ish.

Do you mean the false constants or something else?

I've toned down bracket highlights.

TYSM for this theme. I use Ayu Dark for pretty much every IDE I use.
Please reconsider these recent changes, especially the purple accent. I haven't tried other languages with them yet, but currently, my Dart code looks like this (v1.0.2):

Wow, that's really terrible @xuhaibahmad. Can you share a piece of code for me to test on?

Sure! I think the semantic highlighting will have this issue with languages/frameworks that uses object nesting such as Flutter, SwiftUI, and JetPack Compose, etc.
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';

class ClickableMaskView extends StatelessWidget {
  const ClickableMaskView({
    Key? key,
    required this.rippleColor,
    required this.onClick,
    required this.padding,
    required this.radius,
  }) : super(key: key);

  final Color rippleColor;
  final Function onClick;
  final double padding;
  final double radius;

  @override
  Widget build(BuildContext context) {
    return Positioned.fill(
      child: Container(
        padding: EdgeInsets.symmetric(horizontal: padding),
        child: Material(
          color: rippleColor,
          child: InkWell(
            onTap: () => onClick.call(),
            borderRadius: BorderRadius.circular(radius),
          ),
        ),
      ),
    );
  }
}

The same with python named arguments:

setup(
    name="django-stubs",
    version="1.8.0",
    description="Mypy stubs for Django",
    long_description=readme,
    long_description_content_type="text/markdown",
    license="MIT",
    url="https://github.com/typeddjango/django-stubs",
    py_modules=[],
    python_requires=">=3.6",
    install_requires=dependencies,
)

One more problem: new search is very low in contrast. It is hard to see the highlighted text.

I really hope that it will be possible to restore things back to normal soon 🙂

Can you guys give a go to this version ayu-1.0.2.vsix.zip

@dempfi thanks a lot! Brackets and keyword arguments are back to normal! 👍
However, function arguments, search results, and constants are still purple for some reason 😞

Much better! Though the use of purple for parameters means a lot of my code is still that color. Also, noticed that parameters are not colored when I disable semantic highlighting.
Semantic Highlighting ON:

Semantic Highlighting OFF:

Ok, I found the solution for my problem. Sharing with other people:
You can use older versions of extensions in vscode.
https://stackoverflow.com/questions/42626065/vs-code-rollback-extension-install-specific-extension-version/53755378
Demo (and the final result, code color is back to normal):
https://user-images.githubusercontent.com/4660275/132106037-b87ed8db-7e96-4e98-821c-11ff619e69f2.mov

i had to rollback to v0.20.2. love ayu tho!!

Hope I am not late for the party.
@dempfi I work with Angular/TS. And the yellow accent colors are still purple, the type associations are the same color as the Interface
screens from v.1.0.0 and up:

I think this is the worst part:

screens from same code before v.1.0.0:

If you need snippets or anything else let me know.

@emlautarom1 when you say

Text used to be yellow, and now is pink-ish.

Do you mean the false constants or something else?

Yes, exactly that. Notice that the constant 2 is also pink-ish

Any updates in this issue?


Templates: Add 5 new story templates

Summary
Adds 5 new story templates.

Baking Bread Guide
Sangria Artichoke
Ways to Eat Avocado
Kitchen Stories
Album Release

Relevant Technical Choices

Adds stickers and story template JSON
Adds video poster URL replacement for __WEB_STORIES_TEMPLATE_BASE_URL__
Adds test to check video poster URLs should be replaceable with __WEB_STORIES_TEMPLATE_BASE_URL__

Testing Instructions
New templates should be available under 'Explore Templates' page.
/wp-admin/edit.php?post_type=web-story&page=stories-dashboard#/templates-gallery

NOTE: Existing 8 story templates are there, above these templates, it's just that they are not visible in this screenshot.
Checklist

[x] This PR addresses an existing issue and I have linked this PR to it in ZenHub
[x] I have tested this code to the best of my abilities
[x] I have verified accessibility to the best of my abilities (docs)
[x] I have verified i18n and l10n (translation, right-to-left layout) to the best of my abilities
[x] This PR contains automated tests (unit, integration, and/or e2e) to verify the code works as intended (docs)
[x] I have added documentation where necessary
[x] I have added a matching Type: XYZ label to the PR

Fixes #5902
Companion PR: #7941 ( assets have already been committed, this PR updates the video )

@choumx, @o-fernandez   Will it be possible to get these templates included in the 1.8 release?

Yea, this should get in for 1.8. Adding @spacedmonkey for review since Pascal's out. I'll review this shortly too.

@choumx thank you, for the suggestions on sticker names.
These names were provided in the Figma, and I think your suggestions can be updated in designs as well, I'll forward the same to design team.

@choumx
With spell fix for avocado needed to update the static assets directory path as well in this PR #7959

Updated the PR,

Reset ids to 0,
Adds track details for the Kitchen Stories template as we only have captions for those videos.

For the alt / title updates can be taken up in a separate issue / PR which will address all the templates.
A companion PR which adds subtitles to the static-site #7990

@spacedmonkey
If you are referring to the @choumx's suggestions, those have been addressed. They were to update the name of the SVG stickers. I had previously used the one provided in the figma. And yes I feel as the number of stickers are growing there are, we may want to re-visit the naming a bit, for now I'm using what designers provide and make necessary changes if there are duplicates by adding suffixes.
For the 'ID', I missed the image types. I'll update and also will add a test to make sure when adding a raw template these ids should be 0.
thank you.

on other note @spacedmonkey can we address change of ids in a separate PR, as the same change will be required for some of the existing templates. or for now I can update this for batch 0 templates without adding the test for that. and we create a separate issue to track changes required for consistent JSON output.

@divyarajmasani There are a number of awaiting feedback to change names of files that seem valid to me. If can you review the review comments and either action them or mark them as resolved.

I have created the breakout issue that related to accessibility fields. https://github.com/google/web-stories-wp/issues/7999

@divyarajmasani I suppose you now need to add the caption references to these videos here?

@divyarajmasani I suppose you now need to add the caption references to these videos here?

The references are in place, I had updated the JSON where needed after raising the PR against static-site
https://github.com/rtCamp/web-stories-wp/blob/386a1c861dc02d6b73d2016e37b004968f38b985/packages/templates/src/raw/kitchen-stories/template.json#L449
I suppose the caption files are not deployed yet so they may not be available in the editor just yet.

https://wp.stories.google/static/main/images/templates/kitchen-stories/page02_bg_video_subtitle.vtt

I suppose the caption files are not deployed yet so they may not be available in the editor just yet.

wp.stories.google/static/main/images/templates/kitchen-stories/page02_bg_video_subtitle.vtt

It takes ~15mins or so after merging. Changes are all live now.

I'll update the Docs for adding external templates to reflect the points that were discussed on this PR, like reset the IDs and about the caption files in a separate PR or with the batch 01 PR. :)

Awesome, thanks!
If you can think of any Jest tests to catch ID resets, that would be great too.

If you can think of any Jest tests to catch ID resets, that would be great too.

Yep, I had added it in the same PR but, the test was failing for existing templates as well, so I'll raise another PR today to fix that in current templates and the test.

Moving to QA just to get some extra eyes before merging.


s3ql: 4.0.0 -> 5.1.3

Changelog: https://github.com/s3ql/s3ql/releases/tag/s3ql-5.1.3
Description of changes
Fix build (https://hydra.nixos.org/build/255085966)

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 303073 run on x86_64-linux 1

2 packages built:

s3ql
s3ql.dist


Old School Bond not unlocked in GE (tradeable)

The tradable Old School Bond is locked on the GE. You cannot buy a bond there without first disabling the plugin. The tradeable version of the old school bond should be unlocked since you cannot obtain it in game without first having an untradeable version in your inventory and spending gp to make it tradeable (which you cannot do because the tradeable bond is locked on the GE) . I think the untradeable  bond is unlocked on starting a new account but that doesn't do any good because you cannot by an untradeable bond.

It looks like the old school bond should be unlocked by default on line 677 of AnotherBronzemanModePlugin.java :
queueItemUnlock(ItemID.OLD_SCHOOL_BOND);
But it doesn't work. Maybe use ItemID.OLD_SCHOOL_BOND_UNTRADEABLE ? Perhaps the 2 IDs got transposed?

Aah interesting! I will try to take a look this weekend, thank you for bringing this to my attention!

Closing this due to being unable to reproduce it, and no one else reporting it as an issue.


Setting beginAtZero

Hi,
thanks for all the effort you put in Charba, we just started using it and I alreay like it!
I am trying to get a Y axis to start from zero. The chartjs way seems to be to set a flag beginAtZero on the ticks to true.
I tried this:
      CartesianLinearAxis axis = new CartesianLinearAxis(chart);
      axis.getTicks().setBeginAtZero(true);
      chart.getOptions().getScales().setYAxes(axis);

But I get an exception at runtime:
ConsoleLogger.java:33 Uncaught exception: (TypeError) : this.$init_12554_g$ is not a function
log_33_g$ @ ConsoleLogger.java:33
ConsoleLogger.java:33     at Unknown.JavaScriptObjectContainer_2_g$(MyClient-0.js@8:338027)
    at Unknown.JavaScriptObjectContainer_1_g$(MyClient-0.js@34:338021)
    at Unknown.ChartContainer_1_g$(MyClient-0.js@34:340006)
    at Unknown.BaseTick_1_g$(MyClient-0.js@23:349429)
    at Unknown.Tick_1_g$(MyClient-0.js@17:350247)
    at Unknown.CartesianTick_1_g$(MyClient-0.js@13:349669)
    at Unknown.new CartesianLinearTick_1_g$(MyClient-0.js@22:349742)
    at Unknown.CartesianLinearAxis_2_g$(MyClient-0.js@45:349651)
    at Unknown.new CartesianLinearAxis_1_g$(MyClient-0.js@28:349646)
    at Unknown.lambda$0_545_g$(MyClient-0.js@17:207978)
    at Unknown.execute_135_g$(MyClient-0.js@3:208127)
    at Unknown.$executeScheduled_0_g$(MyClient-0.js@40:15033)
    at Unknown.runScheduledTasks_0_g$(MyClient-0.js@9:14755)
    at Unknown.flushPostEventPumpCommands_0_g$(MyClient-0.js@5:14847)
    at Unknown.execute_5_g$(MyClient-0.js@22:14986)
    at Unknown.execute_4_g$(MyClient-0.js@19:14723)
    at Unknown.apply_23_g$(MyClient-0.js@28:14409)
    at Unknown.entry0_0_g$(MyClient-0.js@16:14465)
    at Unknown.anonymous(MyClient-0.js@14:14445)
    at Unknown.callback_0_g$(MyClient-0.js@45:14774)

Is there a precondition that I missed? Or is there even a better way to get an axis to start at zero

@ak80 Thank you very much! We are doing our best!
About the issue, you are using version 1.7, isn't it?
We have done a test, using setBeginAtZero, this is the code that we have created:
chart.getOptions().setResponsive(true);
chart.getOptions().setMaintainAspectRatio(true);
chart.getOptions().setAspectRatio(3);
chart.getOptions().getLegend().setPosition(Position.top);
chart.getOptions().getTitle().setDisplay(true);
chart.getOptions().getTitle().setText("Charba Line Chart");
chart.getOptions().getTooltips().setMode(InteractionMode.index);
chart.getOptions().getTooltips().setIntersect(false);
chart.getOptions().getHover().setMode(InteractionMode.nearest);
chart.getOptions().getHover().setIntersect(true);

LineDataset dataset1 = chart.newDataset();
dataset1.setLabel("dataset 1");

IsColor color1 = Colors.ALL[0];

dataset1.setBackgroundColor(color1.toHex());
dataset1.setBorderColor(color1.toHex());
double[] values = getRandomDigits(months, false);
dataset1.setData(values);
dataset1.setFill(Fill.nofill);
LineDataset dataset2 = chart.newDataset();
dataset2.setLabel("dataset 2");

IsColor color2 = Colors.ALL[1];

dataset2.setBackgroundColor(color2.toHex());
dataset2.setBorderColor(color2.toHex());
dataset2.setData(getRandomDigits(months, false));
dataset2.setFill(Fill.nofill);
dataset2.setHidden(false);

CartesianCategoryAxis axis1 = new CartesianCategoryAxis(chart);
axis1.setDisplay(true);
axis1.getScaleLabel().setDisplay(true);
axis1.getScaleLabel().setLabelString("Month");

CartesianLinearAxis axis2 = new CartesianLinearAxis(chart);
axis2.setDisplay(true);
axis2.getTicks().setBeginAtZero(true);
axis2.getScaleLabel().setDisplay(true);
axis2.getScaleLabel().setLabelString("Value");

chart.getOptions().getScales().setXAxes(axis1);
chart.getOptions().getScales().setYAxes(axis2);

chart.getData().setLabels(getLabels());
chart.getData().setDatasets(dataset1, dataset2);

Here is the result, without any exception:

Could you put here the complete code in order to recreate the use case?

Thanks for your quick response. Yes I am using Charba 1.7.
I tried with your code sample, and it is not working, so the problem must lie outside. Note that we create the Chart object with new and the configure it, set all the date and after that put it in a panel to be displayed. Could this be the issue.
For completeness here our code
    LineChart chart = new LineChart();
    chart.setWidth("100%");
    chart.getOptions().getLayout().getPadding().setTop(16);
    chart.getOptions().getLayout().getPadding().setBottom(32);

    chart.getOptions().setResponsive(true);
    chart.getOptions().setMaintainAspectRatio(false);
    chart.getOptions().getLegend().setDisplay(true);

    LineDataset dataSet = chart.newDataset();
    dataSet.setBackgroundColor("#4863A0");
    dataSet.setData(new double[]{90.0, 80.0, 88.0});
    chart.getData().setDatasets(dataSet);
    chart.getData().setLabels("one", "two", "three");

    CartesianLinearAxis axis = new CartesianLinearAxis(chart);
    axis.setDisplay(true);
    axis.getTicks().setBeginAtZero(true);
    axis.getScaleLabel().setDisplay(true);
    axis.getScaleLabel().setLabelString("Value");

    chart.getOptions().getScales().setXAxes(axis);

@ak80 Well, I have tried your code as following (in your code you use setXAxes but I think you wanted setYAxes):
@UiField
VerticalPanel panel;

public LineView() {
	initWidget(uiBinder.createAndBindUi(this));

	LineChart chart = new LineChart();
	chart = new LineChart();
	chart.setWidth("100%");
	chart.getOptions().getLayout().getPadding().setTop(16);
	chart.getOptions().getLayout().getPadding().setBottom(32);
	chart.getOptions().setResponsive(true);
	chart.getOptions().setMaintainAspectRatio(false);
	chart.getOptions().getLegend().setDisplay(true);

	LineDataset dataSet = chart.newDataset();
	dataSet.setBackgroundColor("#4863A0");
	dataSet.setData(new double[] { 90.0, 80.0, 88.0 });
	chart.getData().setDatasets(dataSet);
	chart.getData().setLabels("one", "two", "three");

	CartesianLinearAxis axis = new CartesianLinearAxis(chart);
	axis.setDisplay(true);
	axis.getTicks().setBeginAtZero(true);
	axis.getScaleLabel().setDisplay(true);
	axis.getScaleLabel().setLabelString("Value");
	chart.getOptions().getScales().setYAxes(axis);

	panel.add(chart);

}

In UI binder:
<!DOCTYPE ui:UiBinder SYSTEM "http://dl.google.com/gwt/DTD/xhtml.ent">
<ui:UiBinder xmlns:ui="urn:ui:com.google.gwt.uibinder"
	xmlns:g="urn:import:com.google.gwt.user.client.ui"
	xmlns:c="urn:import:org.pepstock.charba.client">

	<g:HTMLPanel  width="100%">
	   <g:VerticalPanel ui:field="panel" width="100%" height="100%" >
       </g:VerticalPanel>
	</g:HTMLPanel>
</ui:UiBinder>

and it works.
I've done also another test, creating the chart but to not add to panel in panel constructor. I've added a button to add the chart and it works as well.

Then, I went to the row of code where you got the exception and it sounds correct (is the base for all Charba objects in version 1.7).
Interesting issue....
Are you using GWT 2.8.2?
Do you have more details about MyClient part?

@stockiNail Yes it is GWT 2.8.2, Java 8.
The application is quite big, maybe I can try to isolate the code and see when it breaks. I am not sure what I can tell you that would be interesting. We use gin/guice and separate our pages into presenter and view. In this case we a have a presenter for charts that determines which type should be displayed (by an enum) and then generates the line chart. Originally it was injected and the data for the series and labels came from a rest call that was done before creating the line chart. But I changed that to be static information. Also I tried adding the chart to an existing panel before rendering it but got some weird exceptions... so I am doing a clean build now. Maybe something in devmode got stuck - it wouldn't be the first time that happened to me.

@ak80 Let me know. FYI, we are using Charba in an application based on GWT Material (where gin is used), version 1.7, and we don't have any issue. I'd ask you if the issue is isolated to setBeginAtZero on ticks and then, removing it, you still get the issue. I don't think so. I'm still investigating to catch the exception. I see you are also using deferred command and I'm gonna try it as well.

@stockiNail: well what do you know... I rebuild the code based on your sample. Did a clean build and threw away all the leftofvers in my temp folder (gwt caches / codeserver leftovers) and now it works. I am not sure  what went wrong but maybe with some of the lambdas and callbacks in the code something got stuck.
Let me thank you again for such quick and helpful response. I am sorry it seems like I messed it up somehow. Anyway I am very happy with the outcome.
For comepleteness sake, here s the code I use now
  private RenderedReport renderLineChart(Report report) {
    LineChart chart = injector.getLineChart();

    chart.setWidth("100%");
    chart.getOptions().getLayout().getPadding().setTop(16);
    chart.getOptions().getLayout().getPadding().setBottom(32);

    chart.getOptions().setResponsive(true);
    chart.getOptions().setMaintainAspectRatio(false);
    chart.getOptions().getLegend().setDisplay(false);
    LineDataset dataSet = chart.newDataset();
    dataSet.setBackgroundColor(getColor(0, 1));
    dataSet.setData(report.getSeries(0));
    dataSet.setFill(Fill.nofill);

    CartesianCategoryAxis axis1 = new CartesianCategoryAxis(chart);
    axis1.setDisplay(true);
    axis1.getScaleLabel().setDisplay(true);
    axis1.getScaleLabel().setLabelString(report.getLabelAxisY());
    chart.getOptions().getScales().setXAxes(axis1);

    CartesianLinearAxis axis2 = new CartesianLinearAxis(chart);
    axis2.setDisplay(true);
    axis2.getTicks().setBeginAtZero(!report.isBestFit());
    axis2.getScaleLabel().setDisplay(true);
    axis2.getScaleLabel().setLabelString(report.getLabelAxisX());

    chart.getOptions().getScales().setYAxes(axis2);

    chart.getData().setLabels(report.getLabels());
    chart.getData().setDatasets(dataSet);

   // this is basically just a wrapper. The callse will pass it to the view
   // which will get the chart out of it and put it into a panel
    return new RenderedReport(chart);
  }

@ak80 I'm very happy too! Thank you! Please start having a look to version 2.0, if you can. Some bugs/features are fixed over there!


Adds yarn lock file

Adds a default yarn lock file for those that don't want to use npm, who want quicker and more deterministic deployments/installs of dependencies etc.

@niallobrien In the future, what would be the best way to update yarn.lock for me with new package versions after running npm-check-updates?

@sahat With yarn you can do yarn upgrade to update all dependencies. It will also update your package.json file and your yarn.lock file. See https://yarnpkg.com/en/docs/cli/upgrade
Also, you could do the following to update specific dependencies:
yarn upgrade [package]
yarn upgrade [package]@[version]
yarn upgrade [package]@[dist-tag]

For further info, see: https://yarnpkg.com/en/docs/managing-dependencies#toc-upgrading-a-dependency
The main advantage to all of this other than speed (yarn can install packages that have already been cached locally, which mean you can even install packages offline), is consistent installation of specific dependencies and their versions, which is quite neat.
If you'd prefer, I could add such information to the Readme.md and update this PR?

Yes please do, I would really appreciate that. Yarn is pretty well known in the the Node.js community now, but might strange for those who are new to Node.js or have been out of the loop for the past week. I've read about the project announcement, its positive benefits, but have yet to give it a try.

@sahat Sure, no problem. I will also clarify that yarn upgrade upgrades packages to the latest version conforming to the version rules set in package.json, instead of the exact versions defined in yarn.lock and will supply some extra reading material (links).

@sahat Can you please review the Readme changes I've add to this PR? Thanks in advance.

Looks good, thank you @niallobrien! 👍


feat(enums): add indonesian locale

Summary
#933
Checklist

[ ] If code changes were made then they have been tested.

[ ] I have updated the documentation to reflect the changes.
[ ] I have run task pyright and fixed the relevant issues.

[ ] This PR fixes an issue.
[x] This PR adds something new (e.g. new method or parameters).
[ ] This PR is a breaking change (e.g. methods or parameters removed/renamed)
[ ] This PR is not a code change (e.g. documentation, README, ...)

Related PR got merged last week.


Plugin simply doesnt install

When I put the .jar into my plugins folder and restart the server nothing happens, nothing new in the console, and the only thing that changes is a folder in the plugins folder that only contains a folder called "External" I've attached the folder it's created.
AdvancedEnchantments.zip

Are there any dependencies or steps that I am missing?

Delete the "external" folder, re-download the plugin, and wait a bit for it to start up.
Make sure that your server has a constant, reliable connection to the internet. AE downloads libs and checks for library updates on startup.

I tried and the same thing happened. What do you mean by library updates?

It wasn't fixed and I've removed and reinstalled multiple times... what am I doing wrong. Would you like to see a server startup log?

Hey, I found an error. In the console it says this: [me.egg82.ae.BukkitBootstrap] Could not download file from URLs provided.
What does that mean?

Means you don't have a valid, active internet connection from the server. One way or another the requests to download the library files are being blocked. DNS, firewall, other plugins..

Okay, thanks! Would you know any plugins or settings that may interfere by any chance?

If another plugin does interfere, then let me know. Otherwise, none that I know of. Enjoy!

Juan did you manage to fix this? i have the same issue

Startup issues were fixed in #12 on authors and so if it's truly something else ¯_(⊙_ʖ⊙)_/¯

Unfortunately I wasn’t able to fix the issue. I believe that it’s because of the service that I use to host my server protecting their network.
I ran a server home-hosted and it worked well, so I though I could upload the files to the main server but that didn’t work and it tried to redownload them. Are there any files other than the plugin folder that need to be carried over or does it just check every time?

In #12 egg said something about the plugin doing a check Everytime it starts but should be able to run without. Maybe it's related?

Egg is it possible to disable that or is that necessary?

Looks like #10 #11 and #12 are loosely related due to the issues all coming from the same system.
AE downloads and inject dependencies it needs at startup. Additionally, it checks for dependency updates on startup. LuckPerms does this same thing, in fact.
I'll re-open and see if I can figure out an elegant way to handle dep version checking/updating on hosts that don't allow library downloads. In the meantime I'll see if I can talk to some hosts about how they handle this type of thing.

The plugin will go to external .xml files to check for new versions of certain libraries, but if they can't be updated it won't try to download them. Either the xml file fetching is being blocked, or for whatever reason the plugin is trying to download/re-download some libraries.
If copying the "external" directory from the plugin's main folder doesn't work, then your host has some VERY strict policies and you should talk to them about that. I understand blocking external jar downloads, but blocking fetching of XML files is a little ridiculous in my opinion.


7.2.0 & ANSI escape sequences (color) not working in GitHub Actions

Prerequisites

[X] Write a descriptive title.
[X] Make sure you are able to repro it on the latest released version
[X] Search the existing issues.
[X] Refer to the FAQ.
[X] Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce
GitHub Actions windows images recently upgraded from 7.1.5 to 7.2.0.  I've got a few repos with ps1 scripts that use ANSI color codes.  The code has worked since Feb-2020.
When Actions switched to 7.2.0, the color text was rendered monochrome.  Note that a custom action (js) does render color, along with a Ruby script.  Also, I noticed no changes locally when upgrading.  The script displaying color can be run locally.
Any ideas?  With Actions, the following line is monochrome:
Write-Host "`nThis is not color.."; Write-Host "`e[93mIs this color?`e[0m"; Write-Host "This is not color.."

Expected behavior
na

Actual behavior
na

Error details
na

Environment data
GitHub Actions windows images

Visuals
No response

/cc @SteveL-MSFT

This is likely because pwsh.exe is redirected in GitHub Action, and in 7.2, when the host is redirected, pwsh uses 'PlainText' as the output rendering style:
https://github.com/PowerShell/PowerShell/blob/fd4117e14552b6a8fa0bef6cc3c34bc67d4f8f0e/src/System.Management.Automation/engine/hostifaces/MshHostUserInterface.cs#L379-L382
However, there should be a way to force using ANSI even if the host is redirected, but that seems impossible today.

there should be a way to force using ANSI even if the host is redirected

Agreed.  Our Jenkins build logs contain ANSI sequences from various different build tools and within Jenkins web UI those display with the appropriate colors.  Whenever I need to view the log raw, I use a browser extension (RetroTxt) to render the ANSI seqs.

@daxian-dbw
Thanks.  So it could be considered an 'edge case' breaking change?  Regardless, I'm ok to close...

Same for Azure Pipelines....
with 7.1.5 ansi colors got displayed correctly, from 7.2.0 no colors in Azure DevOps Pipelines...

Please don't close this issue. It's affecting valid and important use cases.
We know so far that this issue impacts basically all major CI platforms and more likely to come:

GitHub Actions
Azure DevOps Pipelines
Jenkins

From my point of view this is not an 'edge case', unless you consider CI/CD pipelines as not important.
Please also consider that this issue is present in a LTS version.

I think the correct thing here is to NOT deliberately suppress ANSI if pwsh is redirected and instead rely on users to set the appropriate env vars that control this behavior if they don't want ANSI

env vars that control this behavior

And/or add a new pwsh command line switch -NoAnsi.

I can confirm that with PowerShell 7.2.1 ANSI Colors in Azure DevOps Pipelines are displayed correctly again.
(Without any change from our site since 7.1.5 when it worked last)
Thanks! :-)

I can confirm that with PowerShell 7.2.1 ANSI Colors in Azure DevOps Pipelines are displayed correctly again. (Without any change from our site since 7.1.5 when it worked last)
Thanks! :-)

It wasn't a solution-just going around in circles. 😺
All of this would have to be tested in an experimental feature, not in release...

I want to mention that with pwsh 7.2.1 something new happens related to ANSI color codes:
With pwsh 7.1.5 and 7.2.0 you could do a
$adUserString = Get-ADUser xxxx | Out-String
$adUserString -match "UserPrincipalName\s*:\s*([^\s]*)" | Out-Null
$emailAddress = $matches[1]

and you would get a fine EmailAddress what could be used with Send-MailMessage e.g.
with pwsh 7.2.1 this is not more working.
As $matches[1] will start with
`e[0

and cmdlets like Send-MailMessage will crash if you use that match as to-address.
With pwsh 7.2.1 this is necessary now:
$emailAddress = $matches[1] -replace '\x1b\[[0-9;]*m',''

@ClaudiuSchuster  Please open new issue.


get_volume_stats() issue

Version PyU4V Version <IP_ADDRESS>
I've tried all different combinations using this and I always get the same response back.  I've tried using volume_range_start and volume_range_end or a storage group.    The results are the same.
The response is always
PyU4V.utils.exception.ResourceNotFoundException: The requested resource was not found: Error POST Volume resource. The status code received is 404 and the message is None.
Is there a sample code using this or are there any requirements on the array and/or SMAS server side?  Thanks.

Hi,
What version of Unisphere are you using ?
https://pyu4v.readthedocs.io/en/latest/installation.html
Minimum Unisphere Version is 10.1
Phil

Closing issue as it's resolved


build args don't get populated

Hey!
Thanks so much for your work on this!  I noticed that given the following config:
version: '2'
services:
  org-api.app.local:
    build:
      context: .
      args:
        - VAULT_TOKEN
        - APP_VERSION

The build args don't get populated into the Service config.  This works in docker-compose, and it reads the values of those args from ENV variables per  https://docs.docker.com/compose/compose-file/#args:
Build arguments with only a key are resolved to their environment value on the machine Compose is running on.

I did notice that if I changed it to this:
version: '2'
services:
  org-api.app.local:
    build:
      context: .
      args:
        VAULT_TOKEN: bob
        APP_VERSION: dave

Then it would pass those values into the Service config, but it would not set the corresponding args in the actual Docker build.  I'm not sure if it makes a difference, but in the Dockerfile, all the args have default values set.
I did find this: https://github.com/docker/libcompose/pull/307 which seems like it would be relevant, but for environments rather than args.
Thanks!

Thanks! I didn't realize this was an option for build arguments.
If you're at all interested in taking a stab at this yourself, this should be a pretty easy fix by mimicking the similar functionality in environment variable lookups.

Ok.  I'll give it a go.  Thanks!


How to explicitly set rxSessions

It's not clear to me how to configure reactive feature into neo4j-ogm.
I'm using quarkus, and my related dependencies are:
    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-neo4j</artifactId>
    </dependency>
    <dependency>
      <groupId>org.neo4j</groupId>
      <artifactId>neo4j-ogm-core</artifactId>
      <version>3.2.27</version>
    </dependency>
    <dependency>
      <groupId>org.neo4j</groupId>
      <artifactId>neo4j-ogm-bolt-driver</artifactId>
      <version>3.2.27</version>
    </dependency>
    <dependency>
        <groupId>org.neo4j</groupId>
        <artifactId>neo4j-ogm-bolt-native-types</artifactId>
        <version>3.2.27</version>
    </dependency>

My SessionFactory:
@ApplicationScoped
public class SessionFactoryProducer {

    public static final String[] PACKAGES = {
            "me.jeusdi.fhir.beans.entity"
    };

    @Produces
    public SessionFactory produceSessionFactory(Driver driver) {
        return new SessionFactory(new BoltDriver(driver), PACKAGES);
    }

    void disposeSessionFactory(@Disposes SessionFactory sessionFactory) {
        sessionFactory.close();
    }

}

In order to open a session:
@Path("/hello")
@AllArgsConstructor
public class GreetingResource {

    private final SessionFactory sessionFactory;

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public Uni<String> hello() {
        this.sessionFactory.openSession();
        return Uni.createFrom().item("Hello RESTEasy");
    }

}

Nevertheless, I'm not sure it's using reactive session underlying.
Any ideas?

Hi @jeusdi Sorry to be straight forward here: OGM does not support reactive sessions and never will.
It requires a client side subgraph, and does not work on single records.
If you want to use the reactive API of the driver, you can't use Neo4j-OGM. Depending on your mapping needs, it may or may is not an issue.
If you happy with having a potentially blocking IO underneath your reactive flows, you can create flows of the things returned by OGM.
To repeat myself: It is not necessary to use Neo4j-OGM from within Quarkus to connect against Neo4j, query and map things.
The Neo4j-extension is enough.
the OGM mapping features are optional, but not reactive.


The app bar appear separate divider line when modal page on  iOS  platform

This my code below:
PageA：route code
Navigator.of(
      context,
      rootNavigator: true,
    ).push(
    MaterialPageRoute(
      fullscreenDialog: isFullscreenDialog,
      builder: (BuildContext context)=>PageB())
  );

This is pageB code blow:
class PageB extends StatelessWidget {

  PageB({
    Key key,
  }) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text('ColorNavPage'),
        backgroundColor:Colors.blue
      ),
      body: Container(color:Colors.blue),
    );
  }
}

This is  issue video and screenshot.Mybe you would look the video by frame.
https://user-images.githubusercontent.com/16165921/167577446-9c1d64cb-df5e-4c0f-a324-752693fc27b9.mp4

Expected the separate divider would not appear  when page presenting.

@gxl2024595
Can you provide flutter doctor -v and complete minimal reproducible code sample that we can directly use and run to verify this behavior ?


Update 02_cors.md

Updating docs page to reflect that CORS allowed-origins require the protocol on the front in order to function correctly.

This seems correct - reference https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#the_http_response_headers
Verified locally by sending a POSTing a form to myself - 'Origin' header sent by my browser includes the http(s):// prefix of the website I was on.


Fixed a bunch of broken scaladoc links

To improve developer experience while browsing the scaladoc on scalatest.org this PR aims to fix a lot of broken links.

This is all I planned to add for this PR.

@kelemensanyi This is helpful!  Thanks for helping, they looks good to me, I think what's still broken are those fixture/.html as they are no longer there, I'll recommend @bvenners to merge this first.
Cheers.