[Flaky Test] All gardener e2e tests

How to categorize this issue?

/area testing
/kind flake
Which test(s)/suite(s) are flaking:
Any gardener e2e test with a Shoot.
After merging of https://github.com/gardener/gardener/pull/10526, the e2e tests are flaking a lot. The reason is that the etcd-controller cache is populated bit late, so that the create event is not processed.
We (@unmarshall) saw that it's getting short-circuited at https://github.com/gardener/etcd-druid/blob/df3ff21d8cd9d3d785309223f543d72518dbeed2/internal/controller/etcd/reconciler.go#L96, because the etcd object is not found. And after that the object is not processed at all unless there is another event for the object.
CI link:
https://prow.gardener.cloud/view/gs/gardener-prow/pr-logs/pull/gardener_gardener/10734/pull-gardener-e2e-kind-ha-single-zone/1850794034683973632
https://prow.gardener.cloud/view/gs/gardener-prow/pr-logs/pull/gardener_gardener/10734/pull-gardener-e2e-kind-ha-multi-zone/1850794031261421568
https://prow.gardener.cloud/view/gs/gardener-prow/pr-logs/pull/gardener_gardener/10734/pull-gardener-e2e-kind-ipv6/1850794036806291456
Reason for failure:
We can see in different tests, there are etcds with no status, ie; the resource is never processed by the etcd-controller. Hence the shoot creation fails and thus the test.
https://gcsweb.prow.gardener.cloud/gcs/gardener-prow/pr-logs/pull/gardener_gardener/10734/pull-gardener-e2e-kind-ha-single-zone/1850794034683973632/artifacts/etcds.yaml
https://gcsweb.prow.gardener.cloud/gcs/gardener-prow/pr-logs/pull/gardener_gardener/10734/pull-gardener-e2e-kind-ha-multi-zone/1850794031261421568/artifacts/etcds.yaml
https://gcsweb.prow.gardener.cloud/gcs/gardener-prow/pr-logs/pull/gardener_gardener/10734/pull-gardener-e2e-kind-ipv6/1850794036806291456/artifacts/etcds.yaml
Anything else we need to know:
/assign @unmarshall

Created  https://github.com/gardener/etcd-druid/issues/898

fyi @LucaBernstein as current release responsible

A PR has been raised. Will hopefully get merged and subsequently released today. So IMO we can just wait a bit.

We can see in different tests, there are etcds with no status, ie; the resource is never processed by the etcd-controller. Hence the shoot creation fails and thus the test.

Also the description is a bit misleading. never processed is not quite right. It will eventually get processed but it might take some time (several minutes of no-action) which usually crosses the timeout for the tests. This is not seen always though and therefore making the tests flaky.

Afaik we use ReadyCheckz in gardener/gardener to avoid cache sync issues like that. https://github.com/gardener/gardener/blob/676d1bd9e95d80b9f4bc9c56807806031da5d1ce/cmd/gardenlet/app/app.go#L170-L172
This might make sense for etcd-druid too. I did not find a similar check there.
WDYT @unmarshall?

Isn't it this the same check?
https://github.com/gardener/etcd-druid/blob/77de3b6be0823539ed2717d34a899d8311959d78/internal/manager/manager.go#L111-L120

etcd-druid v0.23.2 has been released.

#10747 is merged.
/close


Adds an if condition not accounted for, namely (viewmodel != null)

The if  condition in question is based on view pages's DataContext (usually ViewModel) but what if it's a different kind of DataContext? You may be testing something different but a good idea to account for DataContext that is not for a ViewModel. I discovered this in the process of changing a view page to view-model and the page had a DataContext set in codebehind - took me a while to figure out the strange behavior as T10's navigation returns true on exhausting the if condition test.
Better return false same as the else if (viewmodel == null).  Is this related to #1380? May be the same from a different angle?

Sorry, I have to close this and re-open again. Untidy branch info.


Update npm package

Hey,
Could you also push latest version to npm? Current version there is 0.0.18 (0.0.20 on github).
Cheers,
Rafal

Hi @paranoida,
I just published the latest version. Thanks :smile:.

I just noticed the module isn't working due to a missing dependency. Hold on one second.

Fixed.

Works like a charm.
Thanks


Add upgrade param in cmd tool

Add upgrade option in cmd util. Mostly copied from create option. The next step will be updating machine object to perform update.

This change is


vscode.workspace.createFileSystemWatcher inconsistent yeoman trigger (debian)

This is a continuation of a issue#161224 that was closed due to inactivity ( I was on vacation ), where I described the detailed setup of my environment.
The updated vscode details as following:

The folder hierarchy that the yo generator creates on each iteration is exhibited below:

@bpasero  asked me for more logging data and here they are.
Watchers are not called:
[2022-10-24 14:57:38.211] [remoteagent] [info] Extension host agent started.
[2022-10-24 14:57:38.323] [remoteagent] [info] [<IP_ADDRESS>][b6e7f1ea][ManagementConnection] New connection established.
[2022-10-24 14:57:38.332] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] New connection established.
[2022-10-24 14:57:39.902] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> Launched Extension Host Process.
[2022-10-24 15:01:03.588] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_7
[2022-10-24 15:01:03.589] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_7/forms
[2022-10-24 15:01:03.589] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_7/sample-data
[2022-10-24 15:01:03.589] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_7/scripts
[2022-10-24 15:01:03.590] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_7/webcontent
[2022-10-24 15:01:03.590] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_7
[2022-10-24 15:01:03.590] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_7/forms
[2022-10-24 15:01:03.591] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_7/sample-data
[2022-10-24 15:01:03.591] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_7/scripts
[2022-10-24 15:01:03.591] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_7/webcontent
[2022-10-24 15:01:03.622] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_7
[2022-10-24 15:01:03.622] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_7/forms
[2022-10-24 15:01:03.622] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_7/sample-data
[2022-10-24 15:01:03.622] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_7/scripts
file creation event, path: /home/xxxxx/projects/mta/workflow-module_7/webcontent
[2022-10-24 15:01:04.575] [remoteagent] [trace] [File Watcher (parcel)] [CHANGED] /home/xxxxx/projects/mta/mta.yaml
[2022-10-24 15:01:04.575] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [CHANGED] /home/xxxxx/projects/mta/mta.yaml
[2022-10-24 15:01:04.583] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file change event, path: /home/xxxxx/projects/mta/mta.yaml

Logs of files generation from terminal - second time (watchers still not called):
[2022-10-24 15:05:56.574] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_8
[2022-10-24 15:05:56.574] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_8/forms
[2022-10-24 15:05:56.574] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_8/sample-data
[2022-10-24 15:05:56.574] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_8/scripts
[2022-10-24 15:05:56.574] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_8/webcontent
[2022-10-24 15:05:56.574] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_8
[2022-10-24 15:05:56.575] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_8/forms
[2022-10-24 15:05:56.575] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_8/sample-data
[2022-10-24 15:05:56.575] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_8/scripts
[2022-10-24 15:05:56.575] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_8/webcontent
[2022-10-24 15:05:56.606] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_8
[2022-10-24 15:05:56.606] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_8/forms
file creation event, path: /home/xxxxx/projects/mta/workflow-module_8/sample-data
[2022-10-24 15:05:56.607] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_8/scripts
[2022-10-24 15:05:56.607] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_8/webcontent
[2022-10-24 15:05:57.520] [remoteagent] [trace] [File Watcher (parcel)] [CHANGED] /home/xxxxx/projects/mta/mta.yaml
[2022-10-24 15:05:57.520] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [CHANGED] /home/xxxxx/projects/mta/mta.yaml
[2022-10-24 15:05:57.527] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file change event, path: /home/xxxxx/projects/mta/mta.yaml

After deleting some folders - as  mta/workflow-module_5 and mta/workflow-module_6 (then we see that the delete event for 'workflows' sub-folder also caught in the file watcher),
and repeating the flow to generate new workflow module (mta/workflow-module_11) - the watcher works and file is opened:
[2022-10-24 15:16:33.380] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_11
[2022-10-24 15:16:33.380] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_11/forms
[2022-10-24 15:16:33.380] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_11/sample-data
[2022-10-24 15:16:33.380] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_11/scripts
[2022-10-24 15:16:33.380] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_11/webcontent
[2022-10-24 15:16:33.380] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_11/workflows
[2022-10-24 15:16:33.380] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_11
[2022-10-24 15:16:33.381] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_11/forms
[2022-10-24 15:16:33.381] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_11/sample-data
[2022-10-24 15:16:33.381] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_11/scripts
[2022-10-24 15:16:33.381] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_11/webcontent
[2022-10-24 15:16:33.381] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_11/workflows
[2022-10-24 15:16:33.411] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_11
[2022-10-24 15:16:33.411] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_11/forms
[2022-10-24 15:16:33.411] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_11/sample-data
[2022-10-24 15:16:33.412] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_11/scripts
[2022-10-24 15:16:33.412] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_11/webcontent
[2022-10-24 15:16:33.412] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_11/workflows
[2022-10-24 15:16:34.364] [remoteagent] [trace] [File Watcher (parcel)] [CHANGED] /home/xxxxx/projects/mta/mta.yaml
[2022-10-24 15:16:34.365] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [CHANGED] /home/xxxxx/projects/mta/mta.yaml
[2022-10-24 15:16:34.370] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file change event, path: /home/xxxxx/projects/mta/mta.yaml
[2022-10-24 15:16:35.391] [remoteagent] [trace] [File Watcher (parcel)] [ADDED] /home/xxxxx/projects/mta/workflow-module_11/workflows/MyWorkflow.workflow
[2022-10-24 15:16:35.392] [remoteagent] [trace] [File Watcher (parcel)]  >> normalized [ADDED] /home/xxxxx/projects/mta/workflow-module_11/workflows/MyWorkflow.workflow
[2022-10-24 15:16:35.399] [remoteagent] [info] [<IP_ADDRESS>][50cffccb][ExtensionHostConnection] <876> file creation event, path: /home/xxxxx/projects/mta/workflow-module_11/workflows/MyWorkflow.workflow

This may get addressed when we further update parcel watcher to newer versions where new fixes have landed. Closing this issue here as we cannot control parcel watcher from within VS Code. Refs: https://github.com/parcel-bundler/watcher


Packaged validator for JSON schema's schema created with ajv-cli appears invalid

See https://github.com/jessedc/ajv-cli/issues/20

I've made some improvements but meta-schema is still not supported by ajv-pack yet.

Thanks for update. I appreciate it.


Define JSON Schema

...the data schema for the carculator (compare the carculator/data/default_parameters.json) is:
unique_id
  name
  year
  powertrain
  sizes
  (data)
  (metadata)

This is already reasonably abstract. For aircraft, we could simply change some of the field names:

🚗
🛩

powertrain
propulsion

sizes
pax or MTOW or ???

no correspondence
design-concept

The way I understand, the powertrain field already allows to "combine" powertrains (eg. hybrid-electric), by listing them as separate categories.
In aircraft, Hydrogen could be used by:

directly burning it in the turbine
convert it to electricity in a fuel cell

We could use the carculator powertrain logic to describe these cases as:
"propulsion" = "hydrogen-fuelcell"
"propulsion" = "hydrogen-gasturbine"
"propulsion" = "liquidfuel"
"propulsion" = "liquidfuel-hybrid-batteryelectric"

It might make sense to merge the design-concept into this descriptor also:
"propulsion-design" = "liquidfuel-hybrid-batteryelectric-blendedwingbody"

@romainsacchi, is this a good way to describe aircraft typologies?

3rd use case: hydrogen could also be used via a fuel cell (e.g., for cruising) AND combusted (e.g., for take-off), some sort of hybrid gig.
Regarding sizes, I was thinking more of "narrow body", "blended wing body", etc. (so, what you refer to as design), and have the seating capacity has an input parameter specific to each size. But, to be discussed.
For the propulsion terms, I'd suggest using acronyms as we do in carculator: TJ-k ? (turbojet vehicle using kerosene), TJV-h, FCV-h? I don't know, something like that.

I have been categorizing existing studies using the following. Additions to the discussion are italicized.

Airframe Configuration:  Tube and Wing,  Blended Wing Body,  Retrofit
Propulsion Type:  Hydrogen Combustion,  Hydrogen Fuel Cell,  Hydrogen Hybrid (H2 Combustion / H2 Fuel Cell),  Multi-Fuel Combustion (H2 Combustion / Kerosene Combustion)
Propulsion Configuration:  Conventional,  Distributed,  Boundary Layer Ingestion
Size:  Commuter,  Regional,  Small Narrow Body,  Large Narrow Body,  Small Wide Body,  Large Wide Body

Following discussions with @julienmctighe and @romainsacchi on 23.01.2024, we are suggesting a schema of the kind:
"uniqueid": {
    "parameter": "seats",
    "year": 2000,
    "fuselage": "TW",
    "energy_source": "H2",
    "energy_conversion": "FC",
    "transmission": "Elec",
    "propulsion": "Prop",
    "drag_reduction": "None",
    "sizes": [
        "commuter",
        "regional",
        "SNB",
        "LNB",
        "SWB",
        "LWB"
    ],
    "amount": 75,
    "loc": 75,
    "minimum": 50,
    "maximum": 100,
    "kind": "distribution",
    "uncertainty_type": 5,
    "source": "Cox et al. (2018)",
    "url":  "https://doi.org/10.1016/j.trd.2017.10.017"
    "comment": "for testing purposes only"
},

This can be finalized over the coming weeks. For now, we're working with this schema.


[CP] Stable AGP 8.0 minification support when using integration_test package

Support minification for apps that depend on AGP 8 and integration_test (#127628)

Pre-launch Checklist

[x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
[x] I read the [Tree Hygiene] wiki page, which explains my responsibilities.
[x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
[x] I signed the [CLA].
[x] I listed at least one issue that this PR fixes in the description above.
[ ] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making, or this PR is [test-exempt].
[x] All existing and new tests are passing.

@reidbaker this CP PR isn't mergeable in its current state. Can you rebase to only include the commit you are cherrypicking?

@camsim99 @gmackall  is this something you want to take over while Reid is OOO?

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact Hixie on the #hackers channel in Chat (don't just cc him here, he won't see it! He's on Discord!).
If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?
Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

This change has a test exemption on the original PR: https://github.com/flutter/flutter/pull/127628#issuecomment-1563648944


npm deprecated --global

Issue:
This may not be the correct repository for this issue, however, after installing the latest nodejs LTS (16.15.1) I'm getting this warning:
config global `--global`, `--local` are deprecated. Use `--location=global` instead
when using npm.
Somewhere the npm and npm.cmd files that are created include stuff like:
NPM_PREFIX=`"$NODE_EXE" "$NPM_CLI_JS" prefix -g
I don't know how these files are created or who is responsible so this repo may not the right place.
How To Reproduce:

Install NVM
nvm install 16.15.1
npm --version (8.11.0)

Expected Behavior:
For no deprecation warnings to appear.
Desktop:

OS: Windows 11, PowerShell terminal

This is an npm issue not an nvm issue. You didn't show your version of NPM but i'd bet you're using v8.11.
Here's the link to the fix implemented in v8.12

Yes you are correct, it's 8.11.0. I figured it probably wasn't an nvm issue. Is this just the npm version that node 18.15.1 bundles?


structure vuejs functions

Need a little more structure to get filters and pagination going, in addition to search

At least one for getting the data
matching
adding filters
removing filters
searching

The four last ones will call the first one to get the new data back

Got a little wiser, I think: https://gitter.im/vuejs/vue?at=57e6347987fa803c5ee7f804

Have the data as a function. That way, you can add inn categories and buckets later on to be used for filtering, and then updating the filters.
Have a component to get the JSON and set the data. This com
Have a method each for search, filterAdd, filterRemove, pagination, match etc. Each method calls the JSON get/set-component at the end.

Also, how to set data from method: https://vuejs.org/guide/reactivity.html#Initialize-Your-Data

Here's the best example so far. Changing data from two different methods. Doesn't seem like the data needs to be a function:
http://anyonecanlearntocode.com/blog_posts/why-vue-not-react-is-the-new-jquery
Only thing unanswered seems to be the get/set data from norch. Should (yes) and can it be a sub-method?

And the data should consist of two objects + input strings/objects:

query object (with query, buckets, categories, filters and pagination)
searchresult from norch

Input from user:

queryinput string (split into an array)
filterinput object (bucket- or category based)
pagination

The user input is added to the query that gets data from norch. That gives back a searchresult.

Do as much as possible in search method, before sending/setting data to the data model: query, categories, buckets and it's filter counterparts (to be used on buttons).
Maybe use the search to clean up the data structure in searchresult (or make it smaller, cleaner parts) to be used in the template?

How filters, type category (to be filtered on) should be listed in the data model:
[
    field: 'places',
    gte: 'uganda',
    lte: 'uganda'
  },
  {
    field: 'places',
    gte: 'zaire',
    lte: 'zaire'
  }
]

The only thing that needs to be done is to add one of these to the filter-part of the query. Then a new resultset will return and data model will be updated.

Help on how to map the filters from the searchresult:
https://github.com/fergiemcdowall/search-index/issues/315

Need to move the actual searching to method searcher. Methods search and filterOn will calculate query and feed that to the actual searcher that does the JSON querying to a norch instance, gets and sets the data and calculates filters to be filtered on.

Figured out how to call a method from another:
https://github.com/eklem/norch-vuejs-app/commit/a334dcaf6c4f39d70602e0b3e0cf4e6cae6a2bba
Start with splitting the search and searcher methods. Move most stuff to the searcher, except creating the query (will still be in search method). Maybe rename it to searchOn.

Working! just do:
this.methodname(input-var)


chore: test v22 upgrade with v20 base

Test v20 -> v22 upgrade since that will be what actually runs on mainnet.
https://github.com/zeta-chain/node/issues/3174 requires us to bump the upgrade height

Closing as we upgraded to v22


Notebook editor does not close when dirty after opened from Manage Dashboard

Connect to a MSSQL server in ADS -> Manage
Select 'SQL Server Bigdata Cluster' tab from Dashboard
Open notebook and edit it
Close notebook, we will get a "Confirm Save" dialog => Select "Don't Save"
Notebook still shows in editor instead of being closed.

Issue is fixed in https://github.com/Microsoft/azuredatastudio/pull/4568


Add color option to ol.style.Icon

This PR adds a color option to tint an ol.style.Icon.
See https://groups.google.com/forum/#!topic/ol3-dev/vPK4ygQHaqo

@ahocevar, I ended up multiplying pixel values instead of replacing white with the colour because the latter left undesired artifacts in the majority of cases (e.g. anything with antialiasing)

Thanks for this pull request @alexbrault. Great enhancement! I'll review it within the next few days.

This looks great @alexbrault. It looks like we do not have a CLA on file from you yet. Can you please submit one so we merge this?


The specified module could not be found. error shows up even though the DLL exists.

I am compiling a 64-bit program to be injected into a 32-bit program (yes I have the feature flag on).
When I attempt to use syringe.inject with a valid path, it fails with the following message:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: RemoteIo(Os { code: 126, kind: Uncategorized, message: "The specified module could not be found." })', src\main.rs:11:53
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I can confirm that the DLL I am inputting exists and should work. I've tried absolute and relative paths but it still will not load. How might I fix this?

So your injector is 64-bit and both the payload and target is 32-bit?

Can you try to load your dll with the same path in a small test program using LoadLibraryW and see if the same error occurs?

So your injector is 64-bit and both the payload and target is 32-bit?

Yes

Can you try to load your dll with the same path in a small test program using LoadLibraryW and see if the same error occurs?

Oh you mean in C++? I could try that

Did you try it?

Did you try it?

Not yet sorry. Wasn't feeling well after the response so I wasn't on my computer for a bit.


[auth] The user does not receive an email with the instructions to register his password

I'm submitting a ...

[x] bug report
[ ] feature request
[ ] support request

Do you want to request a feature or report a bug?
bug

What is the current behavior?
Despite configuring the environment variables, the email for registering the password is not sent to the user.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

AUTH_RESET_PWD_VIEW: "" #When using a front end with Auth, define this link to point to the password reset view.
AUTH_EMAIL_HOST: "" # SMTP server to be used. If set to NOEMAIL, this functionality is disabled.
AUTH_EMAIL_PORT: 25 # SMTP server port.
AUTH_EMAIL_TLS: "true" # Whether to enable TLS or not for SMTP server.
AUTH_EMAIL_USER: "" # SMTP user.
AUTH_EMAIL_PASSWD: "" # SMTP password.

Note: For internal testing, follow the guidelines at https://confluence.cpqd.com.br/pages/viewpage.action?spaceKey=TI&title=E-mail+via+SMTP to configure the environment variables

creating the new user

export JWT=$(curl -X POST localhost:8000/auth -H 'Content-Type:application/json' -d '{"username": "admin", "passwd" : "password"}' 2>/dev/null | jq '.jwt' -r)

curl -X POST localhost:8000/auth/user -H "Authorization: Bearer ${JWT}" -H 'Content-Type:application/json' -d '
{
  "username": "maria",
  "service": "teste",
  "email": "<e-mail from the @cpqd.com.br domain>",
  "name": "Maria",
  "profile": "admin"
}' 2>/dev/null | jq

auth_1                        | [13/09/21 - 12:47:09] |webRoutes| DEBUG: Creating new user...
auth_1                        | [13/09/21 - 12:47:09] |webRoutes| DEBUG: Reading request...
auth_1                        | [13/09/21 - 12:47:09] |webRoutes| DEBUG: ... request is:
auth_1                        | [13/09/21 - 12:47:09] |webRoutes| DEBUG: {'username': 'maria', 'service': 'teste', 'email'<EMAIL_ADDRESS>'name': 'Maria', 'profile': 'admin'}
auth_1                        | [13/09/21 - 12:47:09] |webRoutes| DEBUG: Inserting user into database...
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: Checking user data...
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: ... user data is OK.
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: Checking whether user already exist...
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: ... user doesn't exist.
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: Checking whether user e-mail is already being used...
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: ... user e-mail is not being used.
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: Creating user instance...
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: ... user instance was created.
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: User data is: maria created by admin
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: Configuring Kong...
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: ... Kong was successfully configured.
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: Adding new user to database session...
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: ... new user was added to database session.
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: Committing database changes...
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: ... database changes were committed.
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: Configuring user profile...
auth_1                        | [13/09/21 - 12:47:09] |RelationshipController| INFO: user maria added to group admin by admin
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: ... user profile was configured.
auth_1                        | [13/09/21 - 12:47:09] |CRUDController| DEBUG: Configuring user password...
auth_1                        | [13/09/21 - 12:47:10] |CRUDController| WARNING: Failed to retrieve SMTP socket. Is the SMTP port closed?
auth_1                        | [13/09/21 - 12:47:10] |CRUDController| DEBUG: ... user password was configured.
auth_1                        | [13/09/21 - 12:47:10] |CRUDController| DEBUG: Sending tenant creation message to other components...
auth_1                        | [13/09/21 - 12:47:10] |CRUDController| INFO: Will emit tenant lifecycle event teste - CREATE
auth_1                        | [13/09/21 - 12:47:10] |KafkaPublisher| DEBUG: Publishing event: kafka:9092, dojot.tenancy: {"type": "CREATE", "tenant": "teste"}
auth_1                        | [13/09/21 - 12:47:10] |messenger| INFO: Trying to publish something on kafka,                      current producer-topics: {'dojot.tenancy': {'dojot-management': 'dojot-management.dojot.tenancy', 'admin': 'admin.dojot.tenancy'}}
auth_1                        | [13/09/21 - 12:47:10] |messenger| INFO: Published message: {'type': 'CREATE', 'tenant': 'teste'} on topic dojot-management.dojot.tenancy
auth_1                        | [13/09/21 - 12:47:10] |KafkaPublisher| DEBUG: data published
auth_1                        | [13/09/21 - 12:47:10] |CRUDController| DEBUG: ... tenant creation message was sent.
auth_1                        | [13/09/21 - 12:47:10] |webRoutes| DEBUG: ... user data was inserted into the database.
auth_1                        | [13/09/21 - 12:47:10] |webRoutes| DEBUG: ... new user was created.

What is the expected behavior?
The new registered user should receive an email with the subject Account Activation (if created new user) or Password Reset (if forgot password).

What is the motivation / use case for changing the behavior?

Please tell us about your environment:

Version: v0.6.1, v0.7.0-rc.4
Environment: kubernetes, docker-compose
Operating system: Ubuntu 20.04

Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)

Sending internal emails via SMTP works. With the same settings, the email is not being sent by dojot.
# telnet smtp.cpqd.com.br 25
Trying <IP_ADDRESS>...
Connected to smtp.cpqd.com.br.
Escape character is '^]'.
220 smtp.cpqd.com.br ESMTP Postfix
MAIL FROM: <e-mail from the @cpqd.com.br domain>
250 Ok
RCPT TO: <e-mail from the @cpqd.com.br domain>
250 Ok
DATA
354 End data with <CR><LF>.<CR><LF>
SUBJECT: Teste de envio de email

Isso é um teste
.
250 Ok: queued as D041A9B181
quit
221 Bye
Connection closed by foreign host.

Steps to send email using gmail smtp server:

Go to Account Security
Activate two factors
Go to 'App Password'
Create a password for the app
Using password in variable 'AUTH_EMAIL_PASSWD'

Support - Login with app passwords

Test OK

Creating new user: Account Activation email is sent

auth_1                        | [09/12/21 - 20:14:18] |RelationshipController| INFO: user maria added to group user by admin
auth_1                        | [09/12/21 - 20:14:18] |CRUDController| DEBUG: ... user profile was configured.
auth_1                        | [09/12/21 - 20:14:18] |CRUDController| DEBUG: Configuring user password...
auth_1                        | [09/12/21 - 20:14:18] |emailUtils| INFO: Starting service of sending email
auth_1                        | [09/12/21 - 20:14:24] |CRUDController| DEBUG: ... user password was configured.
auth_1                        | [09/12/21 - 20:14:24] |CRUDController| DEBUG: Sending tenant creation message to other components...
auth_1                        | [09/12/21 - 20:14:24] |CRUDController| DEBUG: ... tenant creation message was sent.
auth_1                        | [09/12/21 - 20:14:24] |webRoutes| DEBUG: ... user data was inserted into the database.
auth_1                        | [09/12/21 - 20:14:24] |webRoutes| DEBUG: ... new user was created.

Forgot Password?: Password Reset email is sent

auth_1                        | [09/12/21 - 20:16:51] |emailUtils| INFO: Starting service of sending email


Right to left (RTL) improvements

Hello,
Everything is mostly fine when the page is set to RTL, just a few issues with layout
More information toggle

Filmstrip should be reversed (page 0/1 on the right hand side)

Arrows on the filmstrip should point the other way

The About & Content search panel text should be right-aligned
Page search button layout

Thanks again @lukew-cogapp This is an absolute need to address Thank you for bring it forward as an issue.


Bash a requirement? POSIX-friendliness?

Running FreeBSD 11-CURRENT.  Bash is not installed by default on these systems.
In many cases, you can use POSIX sh and have compatibility across all shells. I looked around for some issues on it but couldn't find anything addressing the topic.
Is bash meant to be a requirement in pyenv, or is it there out of habit?

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/28996794-bash-a-requirement-posix-friendliness?utm_campaign=plugin&utm_content=tracker%2F282009&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F282009&utm_medium=issues&utm_source=github).

Obviously bash is a requirement. POSIX sh isn't sufficient to run pyenv itself.

What about zsh?

pyenv's shell function (output of pyenv-init) should be able to work with every bash, zsh, ksh, fish and other POSIX sh. pyenv's libexec only works with bash, though.

I had a separate project where I tried to get script similar to what is in libexec to be as portable as possible.
I am trying to see if it's possible for pyenv to run without bash as a hard requirement (and just zsh for example).  On BSD, you can go surprisingly far without seeing bash requirement.
It may not be as relevant to many here, since bash is shipped with OS X and most linux distros. pyenv/rbenv is useful enough for me I'm fine installing bash for it.

bashism is also not welcomed in Debian/Ubuntu. It'd be nice if we can remove bash dependency from rbenv/pyenv completely. Though it'd be tough since it's depending on many of bash extensions like bash arrays....


didn`t work.

Hearthstone installation not found
~/git/hearthstone-linux/hearthstone ~/git/hearthstone-linux
Which region do you wish to install? [eu/us/kr/cn]: eu
Initialized in /home/tanghe/git/hearthstone-linux/hearthstone/.ngdp
Which locale do you wish to install? [deDE/enGB/enUS/esES/esMX/frFR/itIT/jaJP/koKR/plPL/ptBR/ruRU/thTH/zhCN/zhTW]: enGB
Region: eu
Online version: <IP_ADDRESS>711.198314
Installed version: Not installed
Update required.
Downloading Hearthstone via keg ...
Fetching http://eu.patch.battle.net:1119/hsb
Downloading: 4c4bd289b8e319e333285df3374814d3: 132 [05:13, 44.00s/]Traceback (most recent call last):d7168b09a70e001d5843e43dfde8773: Fetching archives: 10/59
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/urllib3/response.py", line 737, in _error_catcher639e7f: Waiting for data...
yield
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/urllib3/response.py", line 862, in _raw_read
data = self._fp_read(amt, read1=read1) if not fp_closed else b""
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/urllib3/response.py", line 845, in _fp_read
return self._fp.read(amt) if amt is not None else self._fp.read()
File "/usr/lib/python3.10/http/client.py", line 482, in read
s = self._safe_read(self.length)
File "/usr/lib/python3.10/http/client.py", line 633, in _safe_read
raise IncompleteRead(data, amt-len(data))
http.client.IncompleteRead: IncompleteRead(41943040 bytes read, 226492248 more expected)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/tanghe/git/hearthstone-linux/keg/bin/ngdp", line 1048, in
main()
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/tanghe/git/hearthstone-linux/keg/bin/ngdp", line 309, in fetch
ctx.obj.fetch_versions(versions, blobs, remote_cdn, metadata_only, tags)
File "/home/tanghe/git/hearthstone-linux/keg/bin/ngdp", line 263, in fetch_versions
item.fetch()
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/keg/core/fetcher.py", line 54, in fetch
temp_path = self.fetcher.local_cdn.write_temp_file(item.read())
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/urllib3/response.py", line 935, in read
data = self._raw_read(amt)
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/urllib3/response.py", line 861, in _raw_read
with self._error_catcher():
File "/usr/lib/python3.10/contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "/home/tanghe/git/hearthstone-linux/venv/lib/python3.10/site-packages/urllib3/response.py", line 764, in _error_catcher
raise ProtocolError(f"Connection broken: {e!r}", e) from e
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(41943040 bytes read, 226492248 more expected)', IncompleteRead(41943040 bytes read, 226492248 more expected))
%

Looks like a temporary connection error, same as in #30.
If the problem persists, you could try to remove the parameter --cdn "http://level3.blizzard.com/tpr/hs" inside the script and check if this helps :thinking:


Bug with t() and <T /> when set language option (translate in context language instead parameter language if parameter language is not cached yet)

Hello,
You'll find a complete reproduce bug example here :
https://stackblitz.com/edit/gpbse5j-tolgee-language-param-wycsuw?file=src%2FApp.tsx
When you setup language option in t() or <T /> and this language is not cached by tolgee yet, it display the translation in language from tolgee context.
My workaround is a hook calling tolgee.loadRecord in components where I need to display some translations from an entity language parameter. but it's quite annoying.
export function useTolgeeCacheLoad(
  language: string | undefined,
  namespace?: string,
) {
  const tolgee = useTolgee();

  useEffect(() => {
    if (!language) return;
    tolgee.loadRecord({ language, namespace }).catch(reportError);
  }, [tolgee, language, namespace]);
}

SDK should take care of language param and call loadRecord if needed (as explained in doc)

language: string - override current Tolgee language. This way you can switch to different language for separate translation. Load the language manually with tolgee.loadRecord.

https://tolgee.io/js-sdk/api/core_package/tolgee#t
Best regards,

Hey, I think this is not a bug. But I guess it's a bit confusing.
Tolgee doesn't load language automatically when you call t with the language property. It is stated in docs, that you have to load the language yourself. It will only override the current language set in tolgee and look at the cache.
Since the fallback language is set to en if the translation is not present it will fallback to english. So if the French is not loaded, it will display english.

Hello,
So I think the wording of the documentation is unclear. My understanding of doc was tolgee will loadRecord if needed. And I thought it's a handy approach.
As a user of tolgee api, I think I should not have to worry about the cache (and advanced features) of tolgee, when I use the basic api to translate. So my suggestion to you is to take care of this usage so auto loadRecord if needed.
If for you the API is good like this, your product your rules and no problem, but please at least improve the wording of documentation. maybe something like

You have to load the language translations manually with tolgee.loadRecord.

The thing with auto loading is that it's not ready right away, so you would still had to solve the loading state in your app. Otherwise for a brief moment tolgee would display invalid translations anyway (english in case of fallback or just key names). So this is kinda design decision to force you to solve this yourself.
I was considering an edge case, because usually you have whole your app in one language and only rarely you need simultaneously display multiple languages, so it was considered ok, to keep it a bit more "manual".
I'll try to improve the docs about this.

Thank you


zindex issue in mobile view [Concepts Page]

Describe the bug
A clear and concise description of what the bug is.
this is a mobile issue of the side bar is below the header of the
How to Reproduce
Steps to reproduce the issue. Attach all resources that can help us understand the issue:
use a mobile or mobile view and visit the concept page and use the side provided when clicking on the concept icon

Screenshots
Link to GitHub repository with project that has issues
Files that can help reproduce the issue, like your AsyncAPI file, either paste inside the issue in a code block or in a sharable gist

Expected behavior
A clear and concise description of what you expected to happen.
the zindex of the sidebar should be higher then that of the header

I would like to work on it!

@captain-Akshay there is already an open issue for this and someone is working on that so you may close this issue. 😄


debugIdea task added

This pull request adds task to start idea with installed plugin in debug mode.

Why do you need this?

I was debugging idea to find what code is triggered by some of the actions.
Also it was useful to understand what data is stored in PSI tree. Sometimes it is easier to evaluate expression then trying to find it in PSI viewer.

And what're you going to use for debug?

Idea is used to debug other instance of idea started by the plugin.

Won't merged then. IDEA is able debug gradle task without hardcoding debug port and introducing new task type.


Adding automation script to validate PSA rbac

Issue:
https://github.com/rancher/qa-tasks/issues/543
Problem
In this PR we are covering the RBAC cases for the new feature PSA.
Solution
PodSecurityAdmission is a new feature in k8s where we allow cluster owners/admins to configure a few labels to capture pod security. Other members such as Project owner/cluster member/Project members/Project read-only permissions should not be able to configure the labels at namespace level. This only fixes the namespace level permissions and not project level permissions.

Closing this PR and creating a new one as I accidentally merged release/v2.7 into my private branch.


Add elixir support

Elixir is a dynamic, functional, fault-torelant language that runs on BEAM, designed for concurrency.
https://elixir-lang.org/

Elixer has a notion of documentation in the code with @doc and @moduledoc blocks. You could debate about whether these are true comments. But for SLOC, I think it makes more sense to count them as comments than as source.

I'm sorry, it might be a misunderstanding, I'm talking about this here:
https://github.com/flosse/sloc/blob/2cb07c17715d1036cf0965411d5a06a8ed92627f/src/sloc.coffee#L265

...ah, I didn't see the new PR :see_no_evil:

Closing in favor of #148


Unable to extract "case" property of Javascript object

VSCode Version: 1.44.1
OS Version: Darwin x64 19.4.0

Steps to Reproduce:

Create a JS object with "case" as one of the property
const testing = { case: 10 };
Deconstruct to get the "case" property
const {case} = testing;

Does this issue occur when all extensions are disabled?: Yes

Using a keyword in a destructuring is a syntax error. That code will not run in node or browsers


age: age-keygen broken

It links against /lib64/ld-linux-x86-64.so.2, which doesn't exist on musl. To fix it, I got rid of the -buildmode=pie option, which seems to make it link statically.

@konimex
This is a stupid bug Alpine ran into years ago https://github.com/golang/go/issues/18243 and they dropped the fix for it because "bash.make now support the Alpine work around just fine". Which is... Not a solution to the problem?

And people wonder why C is still used...

Yeah, I'll just add that line to our go package and call it a day.


Allow relative/unstyled thumbs

It would be great if thumbs could be displayed in a non static way. Currently you have to choose one size before init, and it is in px. The width of the thumbs is then added as inline style to the elements - which makes it quite impossible to style thumbnails differently on different devices with media queries.
E.g. I would like to show thumbs on smartphones with a size of "20vw", which results in 5 thumbs displayed in the row at once, without thinking about the concrete pixel device width...

I also encountered this problem. I make the simple plugin which resizes the preview for different screens and devices. Maybe it will help you.
https://github.com/q-ber/fotorama-adaptive-thumbs.git


feat(babel): upgrade babel 6 ~> 7

Upgrade babel 6 ~> 7. This also adds the the bigInt plugin so it is now possible to mutate code with the BigN notation (for example 4n).
Fixes #1172

Could you add a test for the BigN notation? I don't think that works yet because we didn't alter the plugins for the parser.

Awesome remark @simondel, I was a bit fast in declaring that it should work 😅
Wat is the idea for the babel plugins? Are we going to make that configurable? I know there was some talk about that a while back, but it wasn't needed in the end?
Next up is support for babel 7 in the stryker-babel-transpiler. Let's do that in a different PR

There are no plans to make the plugin system link to your Babel config unfortunately :( (this is from Babel's perspective, not ours).
I think we should simply add the plugins when people run into an issue.

So this can be merged?

Looking good!


Spawn Player not working...

 (node:11243) UnhandledPromiseRejectionWarning: Error: LavaClient#spawnPlayer() Could not resolve PlayerOptions.guild.
let player = await this.client.lava?.spawnPlayer ( this.client.lava , {
                guild: message.guild ,
                voiceChannel: message.member?.voice.channel ,
                textChannel: message.channel ,
                deafen: true ,
                trackRepeat: false ,
                queueRepeat: false ,
                skipOnError: true
            } );

This is not valid issue. The LavaClient#spawnPlayer() accepts only one parameter. Just pass the PlayerOptions not the LavaClient. Thanks.


Remote execution? (Feature or doc request?)

Is your feature request related to a problem? Please describe.
CVXPY problems are often computationally expensive. Sometimes a person's / organization's compute power is not distributed evenly. It would make sense to be able to harness the organization's power to solve the hard problems.
Describe the solution you'd like
A way to run the CVXPY solver remotely and to distribute the result to the caller's Variable objects.
Describe alternatives you've considered
I have looked into the source code and thought of some ways in, but wondered if anyone else has done this, and perhaps this is a documentation / example / help request rather than a feature request.
Additional context
If assistance would be provided, I would be happy to implement the rest and share it with the community.

I thought a lot at one point about how to have the problem data distributed. We didn't end up implementing any of that. I can't think of any solution that would be different than remote execution.

@SteveDiamond this is what I was aiming to do. What do you think would be the correct way to elegantly implement this in cvxpy? Inherit from Solver class? A standalone class / module? What would be the correct information to pass to the other machine? A pickled "Problem" object? Is there any standardized "problem definition" file that can be generated? etc.

Chiming in here. @noamgat, I'm not sure if I understand your question, but at the moment we don't have a standardized "problem definition" file. Pickling a problem should work for transmitting model information though.

So the ideal way to do this would be to create a standalone function that:

Receives the problem object
Pickles it, transports it to the other machine, solve there, return the object and unpickle it.
Redistribute the solution's values to the original problem's variable.value fields?

OK, after experimenting, its impossible to pickle Problem objects after they have been solved, but pickling the variables is possible.
So the following code looks to be the correct structure for building such a solution:
from pickle import dumps, loads
import base64

from utils import solver_utils

def solve_problem(problem: Problem):
    pickled_problem = dumps(problem)

    # Transfer pickled_problem to the server...

    loaded_problem: Problem = loads(pickled_problem)
    opt_value = loaded_problem.solve(solver=solver_utils.get_solver_str())
    pickled_solved_variables = base64.b64encode(dumps(loaded_problem.variables()))

    # Transfer opt_value and pickled_solved_variables to the client...

    loaded_solved_variables = loads(base64.b64decode(pickled_solved_variables))
    for local_variable, loaded_variable in zip(problem.variables(), loaded_solved_variables):
        assert local_variable.name() == loaded_variable.name()
        local_variable.value = loaded_variable.value
    return opt_value```


doc: add install instructions

@salaboy added some initial installation instruction

@lburgazzoli awesome stuff Luca! merging now!


Updates uglier to 2.7.0

Updates uglier to 2.7.0 from 2.6.0.

@monfresh Ready for merge! Thx!

For simple updates like these, feel free to merge them without me reviewing them, as long as the tests pass.


ENH: More permissive S3 reading

When calling get_bucket, boto will by default try to establish that the S3 bucket exists by listing all of the keys that exist in it. This behavior is controlled by the "validate" keyword, which defaults to True. If your access key doesn't have permission to read everything in a bucket (even if you do have permission to read the file you're trying to access), this generates an uninformative exception.
This PR sets "validate=False". This means that boto will trust you that the bucket exists, and not try to check immediately. If the bucket actually doesn't exist, the get_contents_as_string call a couple of lines later will generate the exception "S3ResponseError: S3ResponseError: 404 Not Found".

I didn't know that validate behaved that way. @stephen-hoover do you know what version of boto you're using? The behavior may have changed in 2.25 http://boto.readthedocs.org/en/latest/ref/s3.html#boto.s3.connection.S3Connection.get_bucket
Either way this seems reasonable given the way we use it.

I didn't know that until yesterday myself. I'm using the most recent version of boto, 2.38.0.

Ok. I'll merge this later today assuming there aren't any objections.

if this is a newish feature in boto, then need some version checking

Oh, sorry, can you add a release not too. in doc/source/whatsnew/v0.17.0.rst, probably under bug-fixes. You can just refer to this PR for the issue number. Thanks.

release note as well

@jreback I don't think the keyword argument is new, just what boto does when validate is True.

I added a release note. Thanks for checking the old version of boto!

One of the test cases tried to read the file "s3://cant_get_it/tips.csv" and expected to get a permission denied error. With the change I made, this file is now accessible -- apparently the file itself is publicly readable, but the bucket contains non-public contents.
For now I modified the tests to expect that the file should now be readable -- it's a good check that my change works. You might prefer to change the permissions on that "tips.csv" file so that it's really not publicly accessible, and revert the test to expecting a "permission denied" error. If so, I'll change the test case back.

Did we set up that cant_get_it s3 bucket? Seems strange that it would exist and have that file be publicly available.
Anyway, can you add an additional test on a bucket / key that we know doesn't exist? I'd like to make sure that it still raises the S3ResponseError.

The remaining test in test_s3_fails tries to access a nonsense address and checks that it gets the expected "S3ResponseError: 404 Not Found" error, so I think that's already covered. I restored the test checking for a 403 error, this time hitting only the bucket. That generates a 403 error. For the purpose of this test, I don't think it matters that the test isn't trying to access a real CSV file, and I don't know of a non-permissioned file in S3 that I could point at instead.

Ahh, yes I missed that other test. Ok, I'll merge later tonight. Thanks

I am curious who owns those S3 buckets. There was already a test hitting a live bucket, so it seems reasonable to keep the check on "cant_get_it/tips.csv". But it does cost a (small) amount of money for external access to a file in S3, so it's a surprising thing to do in a big open-source project like this one.


Allow the EDS to be used with SLSC Switch

The EDS is designed to change any property of the card. The SLSC Switch is designed to control the switches. If you have a card that has "properties" and "switches" to be control (like the SLSC-12251), then you probably would need to have both functionalities. I think we should "expand" the SLSC-EDS to allow the card to support the switches AND have some properties available to change.
(*This was reported by Burt Snover from SET).

For this the SLSC reservation group comes into play. In order to support having the switch CD play nice with other SLSC CDs you need to have the reservation group match (an empty string apparently does not mean "don't reserve" but instead means "exclusive lock"). What we did for our projects was to use the module URL as the reservation group ie:


Accessing web server when running in QEMU

I was able to get the app to run in QEMU using provided instructions. Is there a way to access HTTP server inside a browser on the host machine?

I guess it is possible, but you have to consult the qemu documentation as there is nothing Rust or ESP-IDF specific in it.


no hmm output after BuildGroups.py

Trying to pull out orthologs after BuildGroups.py using the following command:
Subfolder exists: /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/ortho_align
Subfolder exists: /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/concat
Subfolder exists: /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/hmms
Parsing matrix to identify orthologs...
1471 orthologs found.
Indexing all_groups.hmm...

Error: File format problem in trying to open HMM file /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/all_groups.hmm.
File exists, but appears to be empty?

Extracting 1471 HMM files...0 already found.
	1400 remaining...
	1300 remaining...
	1200 remaining...
	1100 remaining...
	1000 remaining...
	900 remaining...
	800 remaining...
	700 remaining...
	600 remaining...
	500 remaining...
	400 remaining...
	300 remaining...
	200 remaining...
	100 remaining...
	0 remaining...
	Done!
Concatenating 1471 ortholog files...
Aligning 1471 ortholog files...

Error: File existence/permissions problem in trying to open HMM file /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/hmms/group_00026.hmm.
HMM file /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/hmms/group_00026.hmm not found (nor an .h3m bin

BuildGroups.py seemed to run fine
...
Clustering sequences...
Aligning groups...
Cleaning up /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/clustered
Building hmms...
Cleaning up /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/aligned
Emitting consensus sequences...
Writing multi-hmm file...
Writing multi-fasta consensus file...
Cleaning up /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/hmms
Cleaning up /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/consensus_seqs
Cleaning up /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/m8
Cleaning up /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/paranoid_output
Cleaning up /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/dmnd_tmp
Cleaning up /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_pyp/faa
________________________________________________________________________________
| ~/Dropbox/projects/2017_ortho_play/pyparanoid @ Seans-MacBook-Pro (seanleonard)

But the all_groups.faa and all_groups.hmm files are both empty. Any reason the .hmm wouldn't actually be written?

Oh! Yes my cd-hit installation was bad, I reinstalled manually.
It still wasn't working, so I tried turning off MP since I figured it was silently failing. In sequential mode I repeatedly got this error:
Sequential mode...
Traceback (most recent call last):
  File "/usr/local/bin/BuildGroups.py", line 551, in <module>
    main()
  File "/usr/local/bin/BuildGroups.py", line 531, in main
    cdhit_seqs()
  File "/usr/local/bin/BuildGroups.py", line 330, in cdhit_seqs
    CDHIT_RUN(f)
  File "/usr/local/bin/BuildGroups.py", line 340, in CDHIT_RUN
    proc = subprocess.Popen(cmds.split(),stdout=FNULL,stderr=FNULL)
  File<EMAIL_ADDRESS>line 390, in __init__
    errread, errwrite)
  File<EMAIL_ADDRESS>line 1025, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Eventually I got it working by passing the absolute path for each argument in BuildGroups.py. Relative paths kept throwing that error... not sure why. But its working.. so thanks!

Nevermind... still not working. cd-hit seems to run fine, but still no output in  all_groups.faa and all_groups.hmm
Any other ideas?

Can you double check that all of the programs (hmmbuild, hmmemit, muscle, mcxload, mcxdump, mcl, cd-hit) are accessible in your PATH and run them with no args to verify that they are all installed?
If they are, can you do a test run to generate a new database using ~5 genomes without the --clean and --use_MP flags and with the --verbose output? Then attach the whole output here - your previous error message looks like an issue writing to /dev/null but I want to be sure.
PS thanks for being a beta tester on this.

No problem! Thanks for being so responsive with my issues... I'm happy to beta test and excited to use the tool!  I checked, all of those programs are available in my path and seem to be working fine. Here's the output from a test run with 5 genomes and without --use_MP and --clean tags.
| => BuildGroups.py --verbose --cpus 8 snod_only/ snod_only/strainlist.txt sal_debug/
Formatting 5 fasta files...
Making diamond databases for 5 strains...
Running DIAMOND on all 5 strains...
	Done!
Getting gene lengths...
Parsing diamond results for 5 strains...
	Done!
Running InParanoid on 10 pairs of strains...
Sequential mode...
	0 remaining...
Parsing 10 output files.
	Done!
[mclIO] writing </Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_debug/mcl/data.mci>
.......................................
[mclIO] wrote native interchange 10133x10133 matrix with 36164 entries to stream </Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_debug/mcl/data.mci>
[mclIO] wrote 10133 tab entries to stream </Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_debug/mcl/data.tab>
[mcxload] tab has 10133 entries
[mclIO] reading </Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_debug/mcl/data.mci>
.......................................
[mclIO] read native interchange 10133x10133 matrix with 36164 entries
[mcl] pid 12320
 ite   chaos  time hom(avg,lo,hi) m-ie m-ex i-ex fmv
  1     3.79  0.01 1.00/0.60/1.75 1.02 1.02 1.02   0
  2     5.46  0.03 1.00/0.61/1.65 1.00 1.00 1.02   0
  3     7.15  0.03 0.99/0.23/1.48 1.00 1.00 1.02   0
  4     7.13  0.02 0.99/0.26/2.28 1.00 0.99 1.01   0
  5     2.92  0.01 0.99/0.50/1.29 1.00 0.99 1.01   0
  6     2.25  0.02 1.00/0.47/1.00 1.00 0.98 0.99   0
  7     1.84  0.02 1.00/0.39/1.00 1.01 0.99 0.98   0
  8     1.19  0.02 1.00/0.55/1.00 1.00 1.00 0.97   0
  9     0.86  0.03 1.00/0.61/1.00 1.00 1.00 0.97   0
 10     0.29  0.03 1.00/0.77/1.00 1.00 1.00 0.97   0
 11     0.25  0.03 1.00/0.76/1.00 1.00 1.00 0.97   0
 12     0.27  0.03 1.00/0.82/1.00 1.00 1.00 0.97   0
 13     0.35  0.01 1.00/0.77/1.00 1.00 1.00 0.97   0
 14     0.25  0.01 1.00/0.76/1.00 1.00 1.00 0.97   0
 15     0.13  0.03 1.00/0.87/1.00 1.00 1.00 0.97   0
 16     0.01  0.03 1.00/0.99/1.00 1.00 1.00 0.97   0
 17     0.02  0.01 1.00/1.00/1.00 1.00 1.00 0.97   0
 18     0.04  0.01 1.00/1.00/1.00 1.00 1.00 0.97   0
 19     0.08  0.01 1.00/0.98/1.00 1.00 1.00 0.97   0
 20     0.14  0.01 1.00/0.93/1.00 1.00 1.00 0.97   0
 21     0.22  0.01 1.00/0.83/1.00 1.00 1.00 0.97   0
 22     0.24  0.01 1.00/0.76/1.00 1.00 1.00 0.97   0
 23     0.12  0.01 1.00/0.88/1.00 1.00 1.00 0.97   0
 24     0.01  0.01 1.00/0.99/1.00 1.00 1.00 0.97   0
 25     0.00  0.02 1.00/1.00/1.00 1.00 1.00 0.97   0
[mcl] jury pruning marks: <100,100,100>, out of 100
[mcl] jury pruning synopsis: <100.0 or really really really good> (cf -scheme, -do log)
[mclIO] writing </Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_debug/mcl/mcl.out>
.......................................
[mclIO] wrote native interchange 10133x2431 matrix with 10133 entries to stream </Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_debug/mcl/mcl.out>
[mcl] 2431 clusters found
[mcl] output is in /Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_debug/mcl/mcl.out

Please cite:
    Stijn van Dongen, Graph Clustering by Flow Simulation.  PhD thesis,
    University of Utrecht, May 2000.
       (  http://www.library.uu.nl/digiarchief/dip/diss/1895620/full.pdf
       or  http://micans.org/mcl/lit/svdthesis.pdf.gz)
OR
    Stijn van Dongen, A cluster algorithm for graphs. Technical
    Report INS-R0010, National Research Institute for Mathematics
    and Computer Science in the Netherlands, Amsterdam, May 2000.
       (  http://www.cwi.nl/ftp/CWIreports/INS/INS-R0010.ps.Z
       or  http://micans.org/mcl/lit/INS-R0010.ps.Z)

[mclIO] reading </Users/seanleonard/Dropbox/projects/2017_ortho_play/pyparanoid/sal_debug/mcl/mcl.out>
.......................................
[mclIO] read native interchange 10133x2431 matrix with 10133 entries
Writing fasta files and parsing descriptions...
2432 groups equal to or larger than 0 sequences.
Clustering sequences...
Sequential mode...
Traceback (most recent call last):
  File "/usr/local/bin/BuildGroups.py", line 551, in <module>
    main()
  File "/usr/local/bin/BuildGroups.py", line 531, in main
    cdhit_seqs()
  File "/usr/local/bin/BuildGroups.py", line 330, in cdhit_seqs
    CDHIT_RUN(f)
  File "/usr/local/bin/BuildGroups.py", line 340, in CDHIT_RUN
    proc = subprocess.Popen(cmds.split(),stdout=FNULL,stderr=FNULL)
  File<EMAIL_ADDRESS>line 390, in __init__
    errread, errwrite)
  File<EMAIL_ADDRESS>line 1025, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

OK, it's the same message as before...So all of the external programs I pipe the stdout and stderr to the bitbucket (/dev/null) to avoid cluttering up the terminal where it's running. I use the os.devnull function which should is platform agnostic but something is going wrong...
Can you open the python interpreter and run:
import os
os.devnull
Then from your terminal window run the command:
echo fasdaadsvbkjl >> /dev/null

| ~/Dropbox/projects/2017_ortho_play/pyparanoid @ Seans-MacBook-Pro (seanleonard)
| => python2
Python 2.7.14 (default, Mar 22 2018, 14:43:05)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-9<IP_ADDRESS>)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import os
os.devnull
'/dev/null'
exit()

| ~/Dropbox/projects/2017_ortho_play/pyparanoid @ Seans-MacBook-Pro (seanleonard)
| => echo fasdaadsvbkjl >> /dev/null

try running the following as a script - i'm pretty perplexed about whats going on
import subprocess, os
cmds = "echo foo"
FNULL = open(os.devnull,'w')
proc = subprocess.Popen(cmds.split(),stdout=FNULL,stderr=FNULL)
proc.wait()
FNULL.close()
print "echo OK"
cmds = "cd-hit"
FNULL = open(os.devnull,'w')
proc = subprocess.Popen(cmds.split(),stdout=FNULL,stderr=FNULL)
proc.wait()
FNULL.close()
print "cd-hit OK"

| => python2 test.py
echo OK
Traceback (most recent call last):
  File "test.py", line 12, in <module>
    proc = subprocess.Popen(cmds.split(),stdout=FNULL,stderr=FNULL)
  File<EMAIL_ADDRESS>line 390, in __init__
    errread, errwrite)
  File<EMAIL_ADDRESS>line 1025, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

oh! is it misinterpreting the dash in "cd-hit" as an argument separator....?

ok! It was on my end. I could run cd-hit fine, something wrong with my path. I could run cd-hit fine, but "which cd-hit" returned nothing, see this:
| ~/Dropbox/projects/2017_ortho_play/pyparanoid/debug @ Seans-MacBook-Pro (seanleonard)
| => cd-hit
   	====== CD-HIT version 4.7 (built on Jul 18 2018) ======

Usage: cd-hit [Options]

Options

  -i	input filename in fasta format, required
  -o	output filename, required
  -c	sequence identity threshold, default 0.9
	this is the default cd-hit's "global sequence identity" calculated as:
	number of identical amino acids in alignment
	divided by the full length of the shorter sequence
  -G	use global sequence identity, default 1
	if set to 0, then use local sequence identity, calculated as :
	number of identical amino acids in alignment
	divided by the length of the alignment
	NOTE!!! don't use -G 0 unless you use alignment coverage controls
	see options -aL, -AL, -aS, -AS
  -b	band_width of alignment, default 20
  -M	memory limit (in MB) for the program, default 800; 0 for unlimitted;
  -T	number of threads, default 1; with 0, all CPUs will be used
  -n	word_length, default 5, see user's guide for choosing it
  -l	length of throw_away_sequences, default 10
  -t	tolerance for redundance, default 2
  -d	length of description in .clstr file, default 20
	if set to 0, it takes the fasta defline and stops at first space
  -s	length difference cutoff, default 0.0
	if set to 0.9, the shorter sequences need to be
	at least 90% length of the representative of the cluster
  -S	length difference cutoff in amino acid, default 999999
	if set to 60, the length difference between the shorter sequences
	and the representative of the cluster can not be bigger than 60
  -aL	alignment coverage for the longer sequence, default 0.0
	if set to 0.9, the alignment must covers 90% of the sequence
  -AL	alignment coverage control for the longer sequence, default 99999999
	if set to 60, and the length of the sequence is 400,
	then the alignment must be >= 340 (400-60) residues
  -aS	alignment coverage for the shorter sequence, default 0.0
	if set to 0.9, the alignment must covers 90% of the sequence
  -AS	alignment coverage control for the shorter sequence, default 99999999
	if set to 60, and the length of the sequence is 400,
	then the alignment must be >= 340 (400-60) residues
  -A	minimal alignment coverage control for the both sequences, default 0
	alignment must cover >= this value for both sequences
  -uL	maximum unmatched percentage for the longer sequence, default 1.0
	if set to 0.1, the unmatched region (excluding leading and tailing gaps)
	must not be more than 10% of the sequence
  -uS	maximum unmatched percentage for the shorter sequence, default 1.0
	if set to 0.1, the unmatched region (excluding leading and tailing gaps)
	must not be more than 10% of the sequence
  -U	maximum unmatched length, default 99999999
	if set to 10, the unmatched region (excluding leading and tailing gaps)
	must not be more than 10 bases
  -B	1 or 0, default 0, by default, sequences are stored in RAM
	if set to 1, sequence are stored on hard drive
	!! No longer supported !!
  -p	1 or 0, default 0
	if set to 1, print alignment overlap in .clstr file
  -g	1 or 0, default 0
	by cd-hit's default algorithm, a sequence is clustered to the first
	cluster that meet the threshold (fast cluster). If set to 1, the program
	will cluster it into the most similar cluster that meet the threshold
	(accurate but slow mode)
	but either 1 or 0 won't change the representatives of final clusters
  -sc	sort clusters by size (number of sequences), default 0, output clusters by decreasing length
	if set to 1, output clusters by decreasing size
  -sf	sort fasta/fastq by cluster size (number of sequences), default 0, no sorting
	if set to 1, output sequences by decreasing cluster size
  -bak	write backup cluster file (1 or 0, default 0)
  -h	print this help

  Questions, bugs, contact Limin Fu at<EMAIL_ADDRESS>or Weizhong Li at<EMAIL_ADDRESS>  For updated versions and information, please visit: http://cd-hit.org

  cd-hit web server is also available from http://cd-hit.org

  If you find cd-hit useful, please kindly cite:

  "CD-HIT: a fast program for clustering and comparing large sets of protein or nucleotide sequences", Weizhong Li & Adam Godzik. Bioinformatics, (2006) 22:1658-1659
  "CD-HIT: accelerated for clustering the next generation sequencing data", Limin Fu, Beifang Niu, Zhengwei Zhu, Sitao Wu & Weizhong Li. Bioinformatics, (2012) 28:3150-3152

________________________________________________________________________________
| ~/Dropbox/projects/2017_ortho_play/pyparanoid/debug @ Seans-MacBook-Pro (seanleonard)
| => which cd-hit
________________________________________________________________________________
| ~/Dropbox/projects/2017_ortho_play/pyparanoid/debug @ Seans-MacBook-Pro (seanleonard)

I checked my bash profile and like I fool I had used a relative path to the cd-hit directory... so I fixed my bash profile and your test script works fine, now.

Ok cool that makes more sense now - let me know how it goes when you rebuild the database. If you have the initial mcl clustering from your original database saved you can use --mode extract for BuildGroups.py to skip the DIAMOND searches and clustering.

Thanks! Appreciate you helping me through this. I'm going to close this issue to hide my shame in using a relative path ;)


Spacing is mismatched on comparing side bar button distance from edge.

Operating System:

[ ] Windows 10 64

Clear steps to reproduce the issue: The spacing in the bounding box of the sidebar button is not giving a good visual effect.

Relevant error messages and/or screenshots: No error

I have just overcome this problem. Sending the PR.


Use wfly collection

@guidograzioli a few change for 3trains to (hopefully) work with the new Wildfly and JCliff collections.


Proposed 1.3.0-b5

If merged this PR will:

close #2954
close #2951
close #2957
close #2941
close #2934
close #2949
close #2959
close #2922

Jenkins Build Summary
Built from this commit
Built at 20190614 - 05:43:19
Test Results

Build Type
Log
Result
Status

msvc.Debug
logfile
1178 cases, 0 failed, t: 566s
PASS :white_check_mark:

gcc.Release -Dassert=ON, MANUAL_TESTS=true
logfile
915 cases, 0 failed, t: 4m50s
PASS :white_check_mark:

docs, TARGET=docs
logfile
1 cases, 0 failed, t: 0m1s
PASS :white_check_mark:

gcc.Debug -Dcoverage=ON, TARGET=coverage_report, SKIP_TESTS=true
logfile
1178 cases, 0 failed, t: 16m34s
PASS :white_check_mark:

clang.Debug
logfile
1178 cases, 0 failed, t: 2m46s
PASS :white_check_mark:

msvc.Debug, NINJA_BUILD=true
logfile
1178 cases, 0 failed, t: 596s
PASS :white_check_mark:

gcc.Debug
logfile
1178 cases, 0 failed, t: 2m55s
PASS :white_check_mark:

clang.Debug -Dunity=OFF
logfile
1177 cases, 0 failed, t: 9m46s
PASS :white_check_mark:

clang.Release -Dassert=ON
logfile
1178 cases, 0 failed, t: 4m45s
PASS :white_check_mark:

gcc.Debug -Dunity=OFF
logfile
1177 cases, 0 failed, t: 10m47s
PASS :white_check_mark:

msvc.Debug -Dunity=OFF
logfile
1177 cases, 0 failed, t: 1100s
PASS :white_check_mark:

gcc.Release -Dassert=ON
logfile
1178 cases, 0 failed, t: 4m50s
PASS :white_check_mark:

gcc.Debug -Dstatic=OFF
logfile
1178 cases, 0 failed, t: 2m52s
PASS :white_check_mark:

msvc.Release
logfile
1178 cases, 0 failed, t: 397s
PASS :white_check_mark:

gcc.Debug -Dstatic=OFF -DBUILD_SHARED_LIBS=ON
logfile
1178 cases, 0 failed, t: 2m47s
PASS :white_check_mark:

gcc.Debug, NINJA_BUILD=true
logfile
1178 cases, 0 failed, t: 2m48s
PASS :white_check_mark:

clang.Debug -Dunity=OFF -Dsan=address, PARALLEL_TESTS=false, DEBUGGER=false
logfile
1177 cases, 0 failed, t: 10m19s
PASS :white_check_mark:

clang.Debug -Dunity=OFF -Dsan=undefined, PARALLEL_TESTS=false
logfile
1177 cases, 0 failed, t: 12m10s
PASS :white_check_mark:

Codecov Report

Merging #2971 into develop will increase coverage by 0.13%.
The diff coverage is 44.67%.

@@            Coverage Diff             @@
##           develop   #2971      +/-   ##
==========================================
+ Coverage    69.36%   69.5%   +0.13%
==========================================
  Files          710     709       -1
  Lines        56556   56433     -123
==========================================
- Hits         39231   39224       -7
+ Misses       17325   17209     -116

Impacted Files
Coverage Δ

src/ripple/protocol/STAmount.h
100% <ø> (ø)
:arrow_up:

src/ripple/protocol/STArray.h
87.8% <ø> (ø)
:arrow_up:

src/ripple/protocol/STTx.h
66.66% <ø> (ø)
:arrow_up:

src/ripple/overlay/impl/PeerImp.h
0% <ø> (ø)
:arrow_up:

src/ripple/protocol/STVector256.h
100% <ø> (ø)
:arrow_up:

src/ripple/protocol/STPathSet.h
98.37% <ø> (ø)
:arrow_up:

src/ripple/protocol/STInteger.h
100% <ø> (ø)
:arrow_up:

src/ripple/protocol/Serializer.h
85.84% <ø> (ø)
:arrow_up:

src/ripple/protocol/STLedgerEntry.h
64.7% <ø> (ø)
:arrow_up:

src/ripple/nodestore/NodeObject.h
100% <ø> (ø)
:arrow_up:

... and 55 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e83d367...efa9266. Read the comment docs.


Usage of minimap plugin with force layout makes slow rendering

When using the minimap plugin with an animated layout like force, the rendering becomes really slow.
Is there an option to control the rendering options of minimap ? It could help to only refresh every x seconds, or pause rendering while animation is running.

How many nodes in your graph? And what is the renderer?
We are improving the performance of minimap now. v.3.4.5 will be released today.
Use 'canvas' renderer for graph and 'delegate' or 'keyShape' type for minimap temporally.

About 200 nodes and 200 edges, using canvas renderer.
Can you elaborate on how to pass delegate or keyShape types for minimap? I am instantiating like this:
const minimap = new G6.Minimap({ container: graphDOM, size: [150, 100],  className: "my-minimap", });
then I pass plugins = [minimap] when creating the graph

const minimap = new G6.Minimap({
  container: graphDOM,
  size: [150, 100],
  className: "my-minimap",
  type: 'delegate'
});

Checkout the API here:
https://g6-v3-2.antv.vision/en/docs/api/Plugins/#configuration

Thanks. Performance of minimap with static layout is fine, but when using force animated layouts then it's very slow. I guess there isn't much to do about it, it's limitations of canvas. Can't wait for webgl to increase performance.

Fixed by 3.4.5. Minimap will be updated after the layout is done.

It is achieved by using debounce to reduce the frequency of the minimap's  rendering

This works perfectly now in 3.4.5, thanks for the update.


v4 failing tests on windows

Haven't dug into this, leaving here so I don't forget.  Running tests with coverage fails in a weird way on node v4.  Looks like a spawn-wrap bug maybe?
not ok 8 - Error: spawn UNKNOWN
  ---
  at:
    file: util.js
    line: 870
    column: 11
    function: exports._errnoException
  code: UNKNOWN
  errno: UNKNOWN
  syscall: spawn
  test: TAP
  message: 'Error: spawn UNKNOWN'
  stack: "exports._errnoException (util.js:870:11)\nChildProcess.spawn (internal/child_process.js:298:11)\nChildProcess.spawn (node_modules\\nyc\\node_modules\\spawn-wrap\\index.js:138:18)\nexports.spawn (child_process.js:362:9)\nTest.spawnTest [as spawn] (lib\\test.js:9:38042)\nTest._processQueue (lib\\test.js:9:50895)\nChildProcess.onclose (lib\\test.js:9:43112)\nmaybeClose (internal/child_process.js:818:16)\n"
  ...

Stack:
exports._errnoException (util.js:870:11)
ChildProcess.spawn (internal/child_process.js:298:11)
ChildProcess.spawn (node_modules\nyc\node_modules\spawn-wrap\index.js:138:18)
exports.spawn (child_process.js:362:9)
Test.spawnTest [as spawn] (lib\test.js:9:38042)
Test._processQueue (lib\test.js:9:50895)
ChildProcess.onclose (lib\test.js:9:43112)
maybeClose (internal/child_process.js:818:16)

I had similar error messaging (form-data) and using win-spawn helped. Also small things like this:
var fakePath = path.resolve('/why/u/no/exist');

I had similar error messaging (form-data) and using win-spawn helped. Also small things like this:
var fakePath = path.resolve('/why/u/no/exist');


Upload speed indication

It's good to see the upload speed in web interface, not only the current estimate, so the user will better understand if upload is stalled / etc.

Hello @mamay,
I saw there is a draft for this feature: https://github.com/owncloud/web/pull/8187
Do you know a developer who might work on that?

Don't know, will try to look at this and maybe put some ideas there.

Please close in favor of #8166 if applicable (this is where the web tickets go 🙂 )


RuntimeError: Numpy is not available

Found this error when running semantra demo file.
Semantra installation went smooth. Here is the environment
(semantra) C:\Users\andi\Downloads>pip list
Package      Version
------------ -------
argcomplete  3.4.0
click        8.1.7
colorama     0.4.6
numpy        1.26.0
packaging    24.1
pip          24.0
pipx         1.6.0
platformdirs 4.2.2
setuptools   65.5.0
userpath     1.9.2

(semantra) C:\Users\andi\Downloads>python -V
Python 3.11.9

Error found, although numpy is installed. Thanks in advance to suggest the fix.
Regards

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "c:\users\andi\.local\bin\semantra.exe\__main__.py", line 7, in <module>
    sys.exit(main())
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\semantra\semantra.py", line 606, in main
    model: BaseModel = model_config["get_model"]()
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\semantra\models.py", line 334, in <lambda>
    "get_model": lambda: TransformerModel(model_name=mpnet_model_name),
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\semantra\models.py", line 167, in __init__
    self.model = AutoModel.from_pretrained(model_name)
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\transformers\models\auto\auto_factory.py", line 563, in from_pretrained
    return model_class.from_pretrained(
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\transformers\modeling_utils.py", line 3556, in from_pretrained
    state_dict = load_state_dict(resolved_archive_file)
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\transformers\modeling_utils.py", line 515, in load_state_dict
    return safe_load_file(checkpoint_file)
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\safetensors\torch.py", line 313, in load_file
    result[k] = f.get_tensor(k)
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\torch\storage.py", line 321, in __getitem__
    return super().__getitem__(*args, **kwargs)
C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\torch\storage.py:321: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:84.)
  return super().__getitem__(*args, **kwargs)
hamlet.pdf:   0%|                                                                                | 0/1 [00:15<?, ?it/s]
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "c:\users\andi\.local\bin\semantra.exe\__main__.py", line 7, in <module>
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\semantra\semantra.py", line 619, in main
    documents[fn] = process(
                    ^^^^^^^^
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\semantra\semantra.py", line 304, in process
    flush_pool()
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\semantra\semantra.py", line 276, in flush_pool
    embeddings[
  File "C:\Users\andi\pipx\venvs\semantra\Lib\site-packages\torch\_tensor.py", line 1089, in __array__
    return self.numpy().astype(dtype, copy=False)
           ^^^^^^^^^^^^
RuntimeError: Numpy is not available```

me too

I suspect this is related to the recent numpy 2.0.0 release. Either semantra or a dependency needs to limit to numpy < 2.0.
If you manually install eg numpy==1.26.4 it resolves this error.

I suspect this is related to the recent numpy 2.0.0 release. Either semantra or a dependency needs to limit to numpy < 2.0.
If you manually install eg numpy==1.26.4 it resolves this error.

Thanks for the advise. I had the same thought that numpy 2.0 might be cause to the problem. And thus the numpy version was downgraded to 1.26 (as you might see in my post, that installed numpy package is already at 1.26).
Anyway, that error reproduced in Miniconda3 environment of other Windows 11 machine. However, no such error found (semantra runs perfectly at localhost:8080 when it is installed in Miniconda3 environment on PopOS (Ubuntu 22.04) which interestingly found no numpy package.
Package      Version
------------ -------
argcomplete  3.4.0
click        8.1.7
packaging    24.1
pip          24.0
pipx         1.6.0
platformdirs 4.2.2
setuptools   69.5.1
tomli        2.0.1
userpath     1.9.2
wheel        0.43.0
(semantra) andi@pop-os:~/Documents/Journal/genz$ python
Python 3.9.0 (default, Nov 15 2020, 14:28:56)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
>>> exit()

(semantra) andi@pop-os:~/Documents/Journal/genz$ semantra leslie2021.pdf --port 8081
leslie2021.pdf: 100%|█████████████████████████████████████████| 1/1 [00:00<00:00, 192.12it/s]
 * Serving Flask app 'semantra.semantra'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://<IP_ADDRESS>:8081
Press CTRL+C to quit```

Anyway I consider this is closed.


Add Restart Button

Add restart button at the end of each gameplay.

Hi, can you assign this issue to me please?

Done

Hie, I have updated the code please review it!

Hie, I have updated the code please review it!

Good Work!


Document Inoreader support limitations

Cf. #35
@der-lyse, please take a look at the new text. Does it look okay?

Coverage remained the same at 59.496% when pulling d91bb7bec4d83ab100d1f358b1a30483b08adae8 on doc/inoreader-problems into a42ff2986948be42c68071e450083315feddbaf9 on master.

Thanks!


Update ResourceBar Getter and Setter Functions

Applying commit 6288af5 with fixes to prevent NaN error.

After an offline discussion with @quisquous, I think the intention of this code is to preserve the existing functionality of the getter/setter functions (setAttribute maybe a little less, but definitely keep the getAttribute functionality, see: this comment). With that, I think it makes the most sense to keep the code as-is, and potentially address the setAttribute section separately if necessary.


Proposal: Move System/Diagnostics/Factory Defaults to /System/Firmware

I believe that resetting the firmware to factory defaults:

is not a diagnostic but a cure/setting/fact and
fits better to /System/Firmware, since it is a feature that has directly to do something with the firmware

Cheers

Agreed, I'll put this on my list.


Feature bonface emission model

Addition of the emission model

This is old, so I'm closing the PR


Setup the Indexer name for Testnet MIPs

simple explanations of how to set the Indexer name for MIPs participants

move to the diff branch


⚠ Fortnite has degraded performance

In 8c9d0fa, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1470 ms

Resolved: Fortnite performance has improved in e10b8eb.


Implement cpp-ethereum's test RPC API [WIP]

This API is used by Solidity's soltest (the testing architecture) and https://github.com/ethereum/ethereum-console
Also see: https://github.com/ethereum/interfaces/issues/4

@axic I'm curious, is cpp-ethereum still under active development / is it a client that is reasonably widely used?  All I hear about these days is geth and parity...

@tinybike it is still actively maintained, but I don't think it is widely used

This API is used by Solidity's soltest (the testing architecture) and https://github.com/ethereum/ethereum-console

@axic how do you rate this suite against something like karalabe/hive? Would it be worth the effort to move this sucker over to trufflesuite/ganache-core, or would it be better if we concentrated our efforts on hive? I'm somewhat inclined to think the latter.
Closing either way.

This has no relation to hive, it has a different purpose.

@axic sorry, I misunderstood the basis for this PR... Rereading, the real question I should've asked is how relevant this still is today? Do the solidity unit tests still rely on these RPC calls in cpp-ethereum?
@tcoulter said that you guys are testing against testrpc already today (since we broke the solidity tests last month)? How is this accomplished? Does this change make that task easier?


Add POST example

Summary
An example of sending JSON data would be very helpful in this example.

Hi @adegeo ,
I completely agree, and it also crossed my mind when I typed this up, but since POST is creating a resource, the user needs to get add credentials and then this will snowball into a pretty big PR with OTP password on top. In addition running a POST request will create something on their github repo's/profiles. I don't think that would be ideal, although a gist would be ok I guess.
I couldn't find any public APIs that uses POST. Any ideas? Otherwise I'm also OK with just closing this PR again. It really was just meant as a footnote for the next person.
Best,

That's sort of my best effort on this. I understand if it's not up to standard, in which case you can just close this PR again.
Thanks for your hard work on the docs! It really helps people like me who are new to the libs.

@Doerge @adegeo
I like the idea of extending this to cover REST clients in more detail. That would include POST, PUT, and DELETE.
However, I think that would make this article much longer, and would include many more concepts. I'll propose that this should be a separate tutorial that extends the existing tutorial. That would focus on REST. It could assume this tutorial as a pre-requisite so that it wouldn't need to explain the HTTPClient class, and async web requests.
Thoughts?

I (mis)used the tutorial more as a reference to figure out how to correctly use the async methods. It would have been fine for me with just one example of a POST/PUT appended to the current tutorial, but for someone new to REST there might be a different need.

closing as stale. We like the concept, but this has grown out of date.

closing as stale. We like the concept, but this has grown out of date.


Custom validation rule with additional parameters

Hello
The validation.By method offers no way to use additional parameters.
I would like to modify it to accept a varidic list of parameters after the value to validate, like so:
func checkAbc(value interface{}, args ...interface{}) error {
	s, _ := value.(string)
        caseSensitive, _ := args[0].(bool)
	if !caseSensitive && s != "abc" {
		return errors.New("must be abc")
	}
	if caseSensitive && !EqualFold(s, "abc") {
		return errors.New("must be abc")
	}
	return nil
}

err := validation.Validate("xyz", validation.By(checkAbc, true))

What do you think about it ?

Please forgive my stackoverflow-like usage of github.
My initial idea would be to not modify the 'validation.By' signature but to wrap your custom rule function.
func buildCheckABC(args ...interface{}) {
    return func checkAbc(value interface{}) error {
       // do something with 'args' and 'value'
    }
}

And then call it somewhat like
err := validation.Validate("xyz", validation.By( buildCheckABC(arg1, arg2)))

Just an idea, not tested at all.

I tried the same but it create cyclic import issue if I send the whole struct. (Possible way to send each field separately and handled the same in return func)
In my case, I need to validate a field by creating a HMAC using the other fields in the same struct and compare the same against the filed passed for validation.

I'm running into the same issue, has anyone figure out a solution for this?

Have you tried @burner-account's suggestion? It should work.

Oh that does work @qiangxue, thanks do you think its worth wile to add docs for this?

This is a typical usage of using closure to parameterize a function. But yes, it doesn't hurt to add an example to the docs.


ArgoCD deletes CustomResourceDefinitions that Cilium uses to implement networking and security

When deploying Cilium with ArgoCD, some users have reported that Cilium-generated custom resources are disappearing, which can cause a variety of issues.
To prevent this issue, declare resource exclusions in the ArgoCD ConfigMap by following these instructions.
Here is an example snippet:
   resource.exclusions: |
     - apiGroups:
         - cilium.io
       kinds:
         - CiliumIdentity
       clusters:
         - "*"

Note that the ArgoCD deployments may need restarting after configuring these settings in order for the settings to take effect.

Fredrik Björkman on Slack also reported that:

This problem is related to all workloads you deploy with ArgoCD in a cluster running cilium, not just cilium itself.

If you deploy cilium with ArgoCD, you also need these exclusions in the ArgoCD application definition to not get “out of sync” when hubble rotates it certificates.

ignoreDifferences:
      - group: ""
        kind: ConfigMap
        name: hubble-ca-cert
        jsonPointers:
        - /data/ca.crt
      - group: ""
        kind: Secret
        name: hubble-relay-client-certs
        jsonPointers:
        - /data/ca.crt
        - /data/tls.crt
        - /data/tls.key
      - group: ""
        kind: Secret
        name: hubble-server-certs
        jsonPointers:
        - /data/ca.crt
        - /data/tls.crt
        - /data/tls.key

thank you 🙏🏻 this just helped me fixed things on my cluster.

my kustomization.yaml for my whole stupid cluster looks approximately like this:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: argocd

resources:
- https://raw.githubusercontent.com/argoproj/argo-cd/master/manifests/install.yaml
- cilium-application.yaml
- appX.yaml

patches:
  - patch: |-
      - op: add
        path: /data
        value:
          resource.exclusions: |
            - apiGroups:
                - cilium.io
              kinds:
                - CiliumIdentity
              clusters:
                - "*"
    target:
      version: v1
      kind: ConfigMap
      name: argocd-cm

i'm very open to suggestions!
my considerations for this approach are:

i dont want to change how i'm managing argocd in my cluster. living on the 'bleeding' edge, perpetually applying https://raw.githubusercontent.com/argoproj/argo-cd/master/manifests/install.yaml has worked reasonably well for me.  the alternative (maybe) would be to switch to installing argocd with helm, but i'm not exactly sure that way would be cleaner/neater.
i dont think creating another argocd-cm resource/manifest is appropriate.  that resource would conflict with the resource already included in install.yaml.  my new argocd-cm manifest could be merged with the argocd-cm included in install.yaml, but i figured there had to be a way to "add a thing to an existing resource with kustomize".

please let me know if i did this in a really dumb way.  but this does appear to work.  all of my applications that were flapping synced/out-of-sync because of ciliumIdentities are happily synced now.

@travnewmatic Can accomplish the same thing using clusterResourceBlacklist in an AppProject? If so, you can declare it in one of the argocd-cm globalProjects.
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: default
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  description: application project
  clusterResourceWhitelist:
  - group: '*'
    kind: '*'
  clusterResourceBlacklist:
  - group: cilium.io
    kind: CiliumIdentity
  sourceRepos:
  - '*'
  destinations:
  - namespace: "*"
    server: "*"