pkgman-to-bundles does not propagate channels from channel heads

Bug Report
What did you do?
git clone https://github.com/k8s-operatorhub/community-operators
cd community-operators/operators
operator-sdk pkgman-to-bundle etcd etcd-bundles
cat etcd-bundles/bundle-0.9.0/bundle/metadata/annotations.yaml

What did you expect to see?
I expected the command to automatically traverse the replaces chain from each channel head to set the correct channels for all bundles. For example, I expected 0.9.0 to be in channels:

clusterwide-alpha due to being in replaces chain of 0.9.4-clusterwide
singlenamespace-alpha due to being in the replaces chain of 0.9.4

Also side note (this does not happen for this operator): I would NOT expect bundles to be added to the default channel as a default unless they're actually in the replaces chain for that channel. Put another way, I would expect bundles to only be added to channels based on traversing replaces values from channel heads.
What did you see instead? Under which circumstances?
In the operator-sdk pkgman-to-bundle etcd etcd-bundles output, I see:
INFO[0000] Supported channels cannot be identified from CSV etcdoperator.v0.9.0, using default channel singlenamespace-alpha

In the 0.9.0/metadata/annotations.yaml file, I see:
  operators.operatorframework.io.bundle.channels.v1: singlenamespace-alpha

Environment
Operator type:
N/A
Kubernetes cluster type:
N/A
$ operator-sdk version
operator-sdk version: "v1.11.0-4-g47d8d671", commit: "47d8d6714e75d858297f42088ce2ec32bd809e00", kubernetes version: "v1.21", go version: "go1.16.7", GOOS: "darwin", GOARCH: "arm64"

$ go version (if language is Go)
go version go1.16.7 darwin/arm64

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-12T14:18:45Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"darwin/arm64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.0", GitCommit:"cb303e613a121a29364f75cc67d3d580833a7479", GitTreeState:"clean", BuildDate:"2021-04-08T16:25:06Z", GoVersion:"go1.16.1", Compiler:"gc", Platform:"linux/amd64"}

@rashmigottipati would we be able to get this in v1.16.0 milestone? If not we can bump it to the next one.


feat: Use node 20

Also switch to yarn for package management.

@izhangzhihao PR to use node 20 and switch to yarn for package management.


NullPointerException after registration

I've downloaded the latest version and after registration always I have the same problem:
Execution exception
NullPointerException occured : Try to read admin on null object models.Account (controllers.Application.initSession, line 49)
In /app/controllers/Application.java (around line 49)
45: 	            if(account == null && tx.accounts.size() == 0) {
46: 	            	Bootstrap.index();
47: 	            }
48:
49: 	            if(account.admin != null && account.admin)
50: 	            	agencies = tx.agencies.values().toArray(new Agency[tx.agencies.size()]);
51: 	            else {
52: 	            	agencies = new Agency[] { tx.agencies.get(account.agencyId) };
53: 	            }
54:
55: 	            renderArgs.put("agencies", agencies);
Can someone help me?
Thanks

My first response (but I'm not a developer) would be that you have not created the initial startup account. Or do you mean you get this error when you first try to run the editor on <IP_ADDRESS>:9000?

I solved removing cookies from client browser and adding a slash "/" in address, ie http:/lampo.zapto.org:9000/
Sorry, I cannot replicate problem, hence I don't know which one (or both) solved.


[SPARK-25124][ML]VectorSizeHint setSize and getSize don't return values backport to 2.3

What changes were proposed in this pull request?
In feature.py, VectorSizeHint setSize and getSize don't return value. Add return.
(Please fill in changes proposed in this fix)
How was this patch tested?
Unit Test added

@jkbradley  backport to 2.3.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2542/
Test PASSed.

Test build #95230 has started for PR 22228 at commit 35092f0.

Test build #95230 has finished for PR 22228 at commit 35092f0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95230/
Test PASSed.

Awesome, thank you!
LGTM
Merging with branch-2.3

This is already merged, @huaxingao Could you please close this PR?


chore: update Hero section image ✨

Description
I have made changes images in home.ejs
Fixes #744
Screenshots

Checklist

[x] Tests have been added or updated to cover the changes
[x] Documentation has been updated to reflect the changes
[x] Code follows the established coding style guidelines
[x] All tests are passing

Hey @SurajPratap10  Please review the Pr and Merge it

@SurajPratap10 ,you have given level2 in issue,but in pr you have given level1 ,please value my contribution and give me level2 in this #742


added guide for building a search engine in meilisearch and javascript

What does this PR do?
nothing broken just adds a community guide for building a search engine with meilisearch
its awesome because it has a lot of reads, its hackernoon, and its quite a claim for a guide! haha
PR checklist
Please check if your PR fulfills the following requirements:

[x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
[x] Have you read the contributing guidelines?
[x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!

please let me know if you have any questions! thanks so much!

Hey Spencer, thanks so much for your contribution.
Actually it's a guide that comes from Meilisearch, so this won't qualify as a community guide.
Turns out it also needs to be updated, if you're willing to help, we would be grateful :)


Test cluster failover for one of openstack services

Steps:
1.SSH to one of cluster nodes "pkill nova-api" process
2.Start rally test which will boot instance change instance status delete it
3.Check that service running on other cluster nodes
4.Check that service recover on node where it was killed

 P6 (5) Test HA failover for MariaDB

@gosipyan Could you add more information to the commit message explaining what this change is about? I am not very familiar with QA setups and need some help understanding it :smile:. Thanks!

@nicolasbock Sorry I was really making this as my draft (I guess I need think way I am creating PR's feel I am doing something wrong here :/ ). Hope update will make any sense or I can try better.

No worries @gosipyan, I figured that I might learn something about QA :smile:.

@nicolasbock can you take look one more time
Here is latest run results
https://ci.suse.de/view/Cloud/view/QA/job/cloud-mkphyscloud-qa-rally-test/149/


can't install on by windows 11 by cargo because of LINK : fatal error LNK1181

Describe the bug
can't install on by windows 11 by cargo because of LINK : fatal error LNK1181
To Reproduce

cargo install sniffnet
LINK Error

Screenshots

Desktop (please complete the following information):

OS: windows 11 22H2
rustc: 1.65.0

Have you installed the dependencies as described in the README?

I had tried install dependencies,but get some error.Then i got this:

Can you try installing Sniffnet through the installer? You can find it in the latest release.

Closing the issue for inactivity and since issues of this kind has already been solved a number of times.
To see other issue like this, filter by windows issue tag.


Rename runtime validator into verifier

First result that comes up when searching for data validation vs data verification
Therefore, all of things which we have named validator/runtime_validator/permission_validator and similar should be renamed to verifier. Not permission_verifier because they don't only verify permissions they verify more things than just permissions

I have failed to find the term verifier used in any context, it always seems to be named validator whether it's validating or verifying.
Maybe instead of validator vs verifier we can differentiate static validator vs runtime validator?


Implement __toString on ViewErrorBag

Delegates to default bag, as __call does.

No clear reason as to why this should be done in description.


check if url to instance is correctly formatted

On the Splash screen, the url for the Mastodon instance can be set. At the moment, it doesn't check for the correct protocol (https://) nor if the server is around. Leaves you with a white follow up screen with nowhere to go.

fixed in V1.1.17


Refactor to have tests and error messages

A lot went on here. I made it more of a 'normal' erlang project and added a terrible test suite.
There were actually several problems with round-tripping which I think stem from some refactoring you did, but I've round tripped most of the major types now.
The included schema I borrowed from the cauterize repo, but it was outdated, so I modernized it.
You can run the test and generate a coverage report by doing make test.

Should the .swp file be in there?

Removed.


AMM pool 24 hour volume and fees

Related - Also adding an APY to the pool view

implemented in #275


Cut 2.0.0-beta.1

Blocked by all other items in the "blockers for the KIC 2.0 beta" milestone.
It is ok for the docs PR https://github.com/Kong/docs.konghq.com/pull/3081 to be not merged yet.

This is done now: https://github.com/Kong/kubernetes-ingress-controller/releases/tag/v2.0.0-beta.1


Update README.md

Added my github Profile
Related issue

What changes does this PR do?
Screenshots (If any)

Questions

Can you please review my this pull request #1774 (again sorry for the pull request title. Still trying to figuring out everything.)

Hey @nehanawar025
no worries, we all learn from our mistakes. The whole purpose of this repository to practice.
also could you please close this PR.

Duplicate PR.
__
  Tags: Inefficient


Hyperfile within Azure databricks

Hello,
I am creating an hyperfile inside Azure databricks. I want the hyperfile created to be saved in Azure Blob storage. How and where can i configure to save the .hyper file?

Hi @shomavg, the hyper file is just a normal file, so you should be able to copy it to Azure Blob storage. If you need help with this, I think you should rather ask the Azure databricks support.


Production harden test/dummy user security

Motivation and Context
Without IAP, testing local security requires some mechanisms to spoof user identity. We should ensure that the testing code and config can't be used in an attack, or be accidentally left in production config.
What has changed
The environment variable DUMMYUSERIDENTITY-ALLOWED has to be set equal to true to allow the dummy testing user. Naturally, it defaults to false.
How to test?
Change DUMMYUSERIDENTITY-ALLOWED to be false in docker dev and GCP and check that user access is totally forbidden for requests which don't go via IAP... e.g. try curl -i http://support-tool:80/api/whoami from the support tool in GCP.
Links
Trello: https://trello.com/c/7FMv2msJ
Screenshots (if appropriate):

 Production harden the "dummy user identity" in Support Tool (5)


Question: Is it possible to compile the mlf files to OCaml bytecode, or to OCaml's Flambda IR?

Possible as in supported right now: no. Possible as in possible in principle: yes, I think so. It requires hacking the Malfunction compiler a bit, but maybe it's not too hard.

Okay, thank you for the swift reply!


Fix list in user guide

https://github.com/iqm-finland/qiskit-on-iqm/pull/132 is merged, we can close this one.


GitHub action unit test

Hi Kepler,
I am try to add CI test for this repo to run go test with github action.

the bcc lib need a specific install
some test code need sudo premission
some test case seems not pass on github action
one of unused import removed
make script won't work for now with ginkgo on github action

Thanks and Regards
Sam

@rootfs can you help trigger the test for this?

------------------------------
• Failure [0.002 seconds]
Test Collector Unit
/home/runner/work/kepler/kepler/pkg/collector/collector_test.go:52
  Init and Run [It]
  /home/runner/work/kepler/kepler/pkg/collector/collector_test.go:53

  Expected
      <bool>: false
  to equal
      <bool>: true

  /home/runner/work/kepler/kepler/pkg/collector/collector_test.go:121

@rootfs , may I update this test case to make github action pass?

------------------------------
• Failure [0.002 seconds]
Test Collector Unit
/home/runner/work/kepler/kepler/pkg/collector/collector_test.go:52
  Init and Run [It]
  /home/runner/work/kepler/kepler/pkg/collector/collector_test.go:53

  Expected
      <bool>: false
  to equal
      <bool>: true

  /home/runner/work/kepler/kepler/pkg/collector/collector_test.go:121

@rootfs , may I update this test case to make github action pass?

Please allow me to respond to this.
The test case should be fixed by the following merged PR. It relates to dependency of bpf attachment.
https://github.com/sustainable-computing-io/kepler/pull/133/commits/a2dae7ee03f2c1534815d1334dca01204c21aee0#diff-496a02e131e8a6c835195c4de854d57e09304e82ebb5e3384f5de0c9e71b080c

The test case should be fixed by the following merged PR. It relates to system dependency.
https://github.com/sustainable-computing-io/kepler/commit/a2dae7ee03f2c1534815d1334dca01204c21aee0#diff-496a02e131e8a6c835195c4de854d57e09304e82ebb5e3384f5de0c9e71b080c

maybe we can rebase with latest code and run UT again? @SamYuan1990

The test case should be fixed by the following merged PR. It relates to system dependency.
a2dae7e#diff-496a02e131e8a6c835195c4de854d57e09304e82ebb5e3384f5de0c9e71b080c

maybe we can rebase with latest code and run UT again? @SamYuan1990

WIP

@rootfs , please rerun github action for this PR.
it passed my local github action as:
https://github.com/SamYuan1990/kepler/actions/runs/3001029481
with this PR race, verbose, cover, bench been added as test option.
@sunya-ch ,
I noticed that we have two build tag for bcc or not, hence the github action makes unit test for both go build tag.
@jichenjc , update to latest code, and squash.  :-)

thank you @SamYuan1990


OAuth not properly handling response

https://github.com/OfficeDev/Microsoft-Teams-Samples/tree/main/samples/bot-teams-authentication/python
This sample is not properly handling response received after OAuth.

@diegoriera - Thanks for raising your issue. We will check this at our end and will update you accordingly.

@diegoriera - We tried to repro this issue with the provided sample and observed that it works as expected.

You used SSO, to reproduce it use OAuth as described on this document
https://learn.microsoft.com/en-us/microsoftteams/platform/bots/how-to/authentication/add-authentication?tabs=dotnet%2Cpython-sample

After using OAuth we are also encountering same issue.

We will discuss this internally and will get back to you.

Yes, this is the exact behavior

I'm having the same issue. When I test the connection from Azure I get the token, but it doesn't work from my bot.

{     "errorCode": 1008,     "message": "<BotError>Agent returned unsuccessful status code NotImplemented",     "standardizedError": {         "errorCode": 1008,         "errorSubCode": 1,         "errorDescription": "<BotError>Agent returned unsuccessful status code NotImplemented"     } }

We are able to repro this issue. We have raised a bug for the same. We will inform you once we get any further update from engineering team. Thanks!

I encountered this issue too.
If using Nodejs sample then it works.


Navigation icons orient horizontally for vertical slides

The navigation icons are oriented horizontally even when the swipe direction changes to vertical.

This is by design. You can change icon orientation and position by changing related CSS rules if you need it


feat(sdk-node): add serviceInstanceIdDetector to NodeSDK

Note to reviewers: only last 2 commits are applicable. All prior ones are being reviewed in another PR, but added here because of the dependency.
Once this gets approved I will add proper documentation for it.

Follow up from https://github.com/open-telemetry/opentelemetry-js/pull/4608
Adds the resource detector ServiceInstanceIDDetector on the NodeSDK constructor.
It only gets added by default if the value serviceinstance is part of the list OTEL_NODE_RESOURCE_DETECTORS

Rebased branch. @pichlermarc let me know if this implementation is good, so I can add proper documentation on how to use the new detector and the OTEL_NODE_RESOURCE_DETECTORS


[AAH-2496] Add initial work for Remote details page

Add initial work for Remote details page

After it was approved, I just rebased this one. Merging.


Merging appstudio-shared API

See the JIRA ticket: https://issues.redhat.com/browse/GITOPSRVCE-236
For the reviewer:

Please let me know what needs to be removed/added into the config/rbac directory as I couldn't find how these are getting generated.

Changelog for promotion_types.go
Changes:

ApplicationPromotionRun struct rename --> PromotionRun (required a change in PromotionRunList as well as it's using it)
ApplicationPromotionRunList struct rename --> PromotionRunList
ApplicationPromotionRunSpec struct rename --> PromotionRunSpec (required a change to PromotionRun as well as it used this)
ApplicationPromotionRunStatus rename --> PromotionRunStatus (required a change to PromotionRun as well as it used this)

Also adds two member fields: PromotionStartTime and Conditions (including the PromotionRunCondition new type)

Rename 3 consts: ApplicationPromotionRunEnvironmentStatus_Success --> PromotionRunEnvironmentStatus_Success
ApplicationPromotionRunEnvironmentStatus_InProgress --> PromotionRunEnvironmentStatus_InProgress
ApplicationPromotionRunEnvironmentStatus_Failed --> PromotionRunEnvironmentStatus_Failed
Init function SchemeBuilder.Register(&ApplicationPromotionRun{}, &ApplicationPromotionRunList{}) --> SchemeBuilder.Register(&PromotionRun{}, &PromotionRunList{})   and it also required a make generate for the deepcopy to make it happy.

New stuff added:

PromotionRunCondition and PromotionRunReasonType struct
PromotionRunConditionType and PromotionRunConditionStatus string aliases
4 new consts: PromotionRunConditionStatusTrue, PromotionRunConditionStatusFalse, PromotionRunConditionStatusUnknown, PromotionRunReasonErrorOccurred

Changelog for snapshot_types.go
Changes:

ApplicationSnapshotSpec struct rename --> SnapshotSpec (required to introduce changes for SnapshotComponent as well)
ApplicationSnapshotComponent struct rename --> SnapshotComponent
SnapshotStatus struct rename --> `SnapshotStatus
ApplicationSnapshot struct rename --> Snapshot (required to change SnapshotSpec and SnapshotStatus as well)
ApplicationSnapshotList struct rename -- SnapshotList
Init chemeBuilder.Register(&ApplicationSnapshot{}, &ApplicationSnapshotList{}) --> 	SchemeBuilder.Register(&Snapshot{}, &SnapshotList{}) (required make generate to keep it happy)

Nothing new was added here.
FYI: the CI is not picking up this commit. I am testing it locally with make test ;)

Changelog for snapshotenvironmentbinding_types.go
Changes:

SnapshotEnvironmentBindingSpec struct rename --> SnapshotEnvironmentBindingSpec
ApplicationSnapshotEnvironmentBindingStatus struct rename --> SnapshotEnvironmentBindingStatus
ApplicationSnapshotEnvironmentBinding struct name --> SnapshotEnvironmentBinding (require change for SnapshotEnvironmentBindingSpec and SnapshotEnvironmentBindingStatus)
ApplicationSnapshotEnvironmentBindingList struct name --> SnapshotEnvironmentBindingList (require change for SnapshotEnvironmentBinding as well)
Init 	SchemeBuilder.Register(&ApplicationSnapshotEnvironmentBinding{}, &ApplicationSnapshotEnvironmentBindingList{})  --> 	SchemeBuilder.Register(&SnapshotEnvironmentBinding{}, &SnapshotEnvironmentBindingList{}) (required make generate to keep it happy for the deepcopy)

Ready for review folks
@jgwest @johnmcollier

I guess it's because //+kubebuilder:rbac: tags are missing in general, which makes me even more curious about how the config/rbac or even the config/samples were generated in the first place.

I cannot see how these can be generated with the existing files and Makefile. The kubebuilder tags seems to be missing and the reason for missing is because they are getting generated at the controllers, not on the api side. My guess is that the individual teams who created these APIs they originally bootstraped their respective projects using --controller flag of operator-sdk. This created the necessary //+kubebuilder:rbac tags but then I assume these were removed.
Long story short I've copied over the RBAC from appstudio-shared. To me precise here's what I've done:
Same: (no need for changes)

environment_editor_role.yaml
environment_viewer_role.yaml

Changes: (they were only names. No any other significant change was spotted). You can observe this at https://github.com/redhat-appstudio/application-api/pull/12/commits/1a48df17aeac12346d7eae6133c4e1de5dd62b1e

applicationpromotionrun_editor_role.yaml --> promotionrun_editor_role.yaml
applicationpromotionrun_viewer_role.yaml --> promotionrun_viewer_role.yaml
snapshot_editor_role.yaml --> applicationsnapshot_editor_role.yaml
applicationsnapshot_viewer_role.yaml --> snapshot_viewer_role.yaml
applicationsnapshotenvironmentbinding_editor_role.yaml --> snapshotenvironmentbinding_editor_role.yaml
applicationsnapshotenvironmentbinding_viewer_role.yaml --> snapshotenvironmentbinding_viewer_role.yaml

New files: These do not exist here but they were part of appstudio-shared back in the managed-gitops repository. I've relocated them here as well just to be on the safe side. You can observe these at: https://github.com/redhat-appstudio/application-api/pull/12/commits/8896ee07f54a944742ac9666ebd8a935e86acaf8

kustomization.yaml
leader_election_role.yaml
leader_election_role_binding.yaml
role_binding.yaml
service_account.yaml
auth_proxy_client_clusterrole.yaml
auth_proxy_role.yaml
auth_proxy_role_binding.yaml
auth_proxy_service.yaml


Support for sucker_punch?

Hi there!
Are you planning to add support for sucker_punch? Quite a lot of people use it for sending emails and performing non-critical jobs.
Thank you.

Hi fwl,
We hadn't planned support for sucker punch as of yet. It does seem like a good idea to support it.
I'll see if I can make time to add support. If you need it yesterday, maybe you can alias perform as done here in the sequel support: https://github.com/appsignal/appsignal/blob/master/lib/appsignal/integrations/sequel.rb

@matsimitsu thank you for the detailed response :-)

Hi, any updates on this feature request by any chance?

I don't think we'll be able to support it in the gem. SuckerPunch doesn't have any hooks we can use to wrap a job in AppSignal.
Option one:
I did play around with it today, and the following code works with minimal impact when using Rails and Ruby 2.0+.
# In an intializer: (config/initializers/job_monitor.rb)
module JobMonitor
  extend ActiveSupport::Concern
  included do
    proxy = Module.new do
      define_method(:perform) do |*args|
         Appsignal.monitor_transaction(
          'perform_job.sucker_punch',
          :class  => self.class.to_s,
          :method => 'perform'
        ) do
          super *args
        end
      end
    end
    self.prepend proxy
  end
end

#In your job include `JobMonitor`
class SingleJob
  include SuckerPunch::Job
  include JobMonitor
  workers 4

  def perform
    STORE << Time.now
  end
end

Option two:
Other options are creating a base class that defines perform and uses Appsignal.monitor_transaction to wrap a different method.

class MonitoredJob
  def perform
     Appsignal.monitor_transaction(
      'perform_job.sucker_punch',
      :class  => self.class.to_s,
      :method => 'perform'
    ) do
      perform_monitored
    end
  end
end

class SingleJob < MonitoredJob
  include SuckerPunch::Job
  workers 4

  def perform_monitored
    STORE << Time.now
  end
end

Option three
Or, just wrap it in Appsignal.monitor_transaction in the job itself:
class SingleJob
  include SuckerPunch::Job
  workers 4

  def perform
    Appsignal.monitor_transaction(
      'perform_job.sucker_punch',
      :class  => self.class.to_s,
      :method => 'perform'
    ) do
      STORE << Time.now
    end
  end
end

@matsimitsu amazing answer, thank you so much!


[chore] Add "Skip functional tests" and 'Skip Functional Test - EKS' label support

Closing this PR now due to an issue with PRs being created from fork repo branches.


update_field_value - labels field must be an array of strings

When running the update_field_value action with labels as the field in the action call, the value passed must be an array (list) of strings. I have tried a few ways of passing a value but given the input is a string this is sent as the wrong type and we get a 400 back from the jira api.
We should see if we can support value fields of types other than string in this action.

Labels cannot have spaces in jira, so we could simply do a split() if field is labels to convert to a list in the action.

Fixed in #59


grpcproxy: add cache related metrics

Fix https://github.com/coreos/etcd/issues/6917

lgtm, not sure if cached_total is the best label

@heyitsanthony changed it to cache_keys_total


terminal-stocks: init at 1.0.14

Description of changes
Added a stock price cli application called terminal-stocks.
Closes #211183.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/1731

Can't seem to find terminal-stocks package on the website or install it with: nix-env -iA nixpkgs.terminal-stocks

The package will show up once it's merged into the nixos-unstable branch.


ML

[ ] #2
[ ] #3
[ ] #4
[ ] #5
[ ] #6
[x] #7
[ ] #8
[x] #9
[x] #10
[ ] #11

[ ] Cross Entropy
[ ] Back Propagation
[ ] Convolution Neural Networks
[ ] Ensemble Methods: Bagging and Boosting
[ ] Non-linear Input Transformations and Kernels
[ ] The Bayesian Approach and Gaussian Processes
[ ] Generative Models and Learning from Unlabelled Data
[ ] Support Vector Regression
[ ] Random Forest
[ ] Back Propagation and Gradient Descent

[] PCA


Support rendering with skia

Partially implemented FormsPlotSkia user control. This control render data using Skia graphic library.
Only signal const implemented for now.
Project that used this control must specify x64 or x86 (untested) mode. Any CPU not supported.
FormsPlotSkia implemented in separate project, to avoid adding dependencies (SkiaSharp, OpenGL.NET) to main ScottPlot project.
Manualy adding libegl.dll and libglesv2.dll to output folder enable Angle backend for Skia (OpenGl by default). Simpliest way get this files from Chrome installation.
Skia library mach more faster in high resolution chart ( full screen ) or heavy drawing calls ( lots of lines like in PlottableConst Demo). Skia show the same performance as GDI+ on low resolution plots for me.
This PR is maked for tests and to show concept. It still a lot of work to fully support skia library, and need some change in main lib drawing calls to simplify Skia library injection.

This is an impressive improvement @StendProg! GDI+ is slow full-screen (especially with a 4K monitor), and using Skia is an excellent way to improve this limitation.
I created a new branch of ScottPlot for Skia:
https://github.com/swharden/ScottPlot/tree/Skia
Do you want to make a pull request there instead of master?
While Skia has many strengths, I think the master branch of ScottPlot will always be dependency-free. If the Skia branch gets developed fully we can make separate "ScottPlot.Skia" NuGet package for it.

I don't clearly understand the reason of separate branch. In this pr ScotPlot Project stay untouched by dependencies ( just maked some methods protected to allow override). All new dependencies only in other ScotPlotSkia project. You already can make ScottPlot nuget without dependencies. And ScottPlot.Skia nuget from skia project. May be i am wrong, not familar with nuget.
Or you want have copy ScottPlot project on skia branch that make directly  calls skia draws instead gdi+?
I thinked about backend solution. Where main ScottPlot lib call drawings to abstract graphic class not directly to gdi+. ScottPlot main lib implement that class using gdi+.So no dependencies.
Other Skia project in solution implement this abstract class using skia. And use existing code of ScottPlot main lib.

new dependencies are only in the ScotPlotSkia project

After initially inspecting this pull request I overlooked this important point. You created a very good design! I will delete the Skia branch and work toward merging this now.

On my system running the Skia demo is faster than GDI+ in fullscreen mode, but slower than GDI+ when using typical window sizes. I find this surprising and interesting.
After thinking about this topic for a long time I decided to move the Skia project to /dev where it has its own solution file and its own demos, completely separate from ScottPlot.
Two of the primary design goals for ScottPlot are that it (1) has no dependencies and (2) is radically simple and easy to use by anybody.

ScottPlotSkia requires dependencies, and even though ScottPlot doesn't have dependencies it does imply that the Skia project is a distinctly different from ScottPlot. I don't think Skia code belongs in the ScottPlot solution.

I have never used Skia before. It seems very good, but it also adds complexity. For example, copying DLLs from a Google Chrome installation. Further, I still can't get my Visual Studio to stop crashing every time I drag/drop the FormsPlotSkia user control onto my designer window (screenshot below). There may be a simple fix to this, but the fact that I have not been able to determine the fix so far means that projects which include ScottPlotSkia are not easy to use. I really like the concept of ScottPlotSkia, but I think it should be cleanly segregated into its own folder and solution.

I think continuing to develop this Skia project has great potential, but I don't think it's consistent enough with the core design goals of ScottPlot to include it the primary ScottPlotV3.sln file.

I moved your code to /dev and created its own demos and solution file there
I am happy to help you further develop your Skia project by modifying ScottPlot (such as making functions protected and separating Action from Impl methods wherever possible). If you make these pull requests, I will act quickly to merge them!

What do you think about this style of organization?
Thank you again for all of your valuable contributions to this project!

After thinking about this topic for a long time I decided to move the Skia project to /dev

Very good solution. I like it.

For example, copying DLLs from a Google Chrome installation

Native solution is to complile you own from Angle repo. I was disappointed too. There some nuget packages  like Avalonia.Angle.Natives, but i get platform not specified error during instalation.

I still can't get my Visual Studio to stop crashing every time I drag/drop the FormsPlotSkia user control onto my designer window (screenshot below).

VS editor try to create and render component in Any CPU mode.  Setting x64 for FormPlotSkia and Demo project fix this crashes. SkiaSharp dont't work with Any CPU. Its only wrapper on Skia native lib (x86 and x64).

I am happy to help you further develop your Skia project by modifying ScottPlot (such as making functions protected and separating Action from Impl methods wherever possible). If you make these pull requests, I will act quickly to merge them!

This some quick and durty hacks to make it work. Static methods on delegates is worst of that.
No need to make any of this changes if whole library will call draws on absract class like this.
//settings.gfxData.DrawLine(pen, xPx, 0, xPx, settings.dataSize.Height);
settings.gfxBackend.DrawLine(xPx, 0, xPx, settings.dataSize.Height, lineColor, linewidth, etc. );
...

//settings.gfxFigure.Clear(settings.figureBackgroundColor);
settings.gfxBackend.Clear(settings.figureBackgroundColor);
...

public class GDIBackend:IGraphicBackend
{
    DrawLine(float x, float y, float x1, float y1,Color lineColor,float  lineWidth, etc. )
    {
        var pen = new Pen(lineColor, lineWidth)
        settings.gfxData.DrawLine(settings.gfxData.DrawLine(pen, xPx, 0, xPx, settings.dataSize.Height);
    }

   Clear(Color color)
    {
        settings.gfxData.Clear(color);
    }

// ... implementation of all posible graphic calls from ScottPlot lib
// ...
}

And backend can be init in Plot() like this:
Plot plt = new Plot(new GDIBackend());

Plot(IGraphicBackend backend)
{
//.............
   settings.gfxBackend = backend;
}
public class Settings
{
//...
    IGraphicBackend gfxBackend;
}

So implementing other graphic libs would be just implementing IGraphicBackend.
There a lots of work to replace all GDI calls in abstract calls that provide enough information for different libs.

Fix designer crashes. For working with designer Any CPU mode must be set and rebuild solution. For debuging x64 mode must be set.

Skia backend changed from offscreen rendering with opengl.net to onscreen rendering to OpenTK.GlControl.
It works realy fast for now 1-10 ms to render plot at any resolution.
Negative side of  onscreen rendering, it's not posible render legend and benchmark without remaking them to onscreen.

Hi @StendProg, I'm still studying this topic. Thank you again for your insightful contributions! I would like to make ScottPlot easier to use for different rendering engines: GDK, Skia, and ImageSharp.
What is the primary reason you are using Skia? Is it because GDK is poorly supported on Linux or mobile? ... or is it because Skia is faster for rendering large images when combined with hardware acceleration? I'm interested in learning more about your use case.

Hi @swharden.
My primary reason is performance. There is only 10 fps on fullscreen plot using GDI+. I'm interested in windows platform only. At fist looked for some DirectX(Direct2D) library, but to my surprise did not find anything worthwhile. SharpDX is most powerfull lib for DirectX, but it's still too complex.
Skia looked very simple to use. It is developed by Google for Chrome. SkiaSharp is developed by Mono (Microsoft for now). So it looked very serious.
After some work on it, i think maybe simple DirectX(Direct2D) wrapper would be enouth. There is no so complex drawing technics used in ScottPlot. SharpDX closed at this year, it is very disappointing.
Anyway i will try to finish work on skia support. It goes slower bacause haven't much free time for now, and spending a lot of time on invistigations (zero expirience on opengl/skia).

Thanks for the details! This helps me understand why you chose Skia.

Hi @StendProg this pull has been extremely useful! Thanks again for working on it.
After more research, I concluded the complexity I disliked came from OpenGL (not SkiaSharp). It's OpenGL that requires your hardware and system to support it, requires you to compile for the right target, and when OpenGL fails it crashes Visual Studio with non-helpful error messages.
However, SkiaSharp is great! I have used it enough now to really like it. It is similar in speed to GDI+ (when not using OpenGL), but it has no dependencies which is consistent with the goals of this project. It also has great extra features like numerous image encoding options, and its objects (like colors and rectangles) are much more logical to use.
I'd like to design a new plotting library (QuickPlot) to use Skia everywhere (no System.Drawing at all). It will render on a mouse-interactive user control. A separate project (QuickPlotGL) can inherit the QuickPlot user control and replace the drawing area with an OpenGL control (like you do with this pull). This way we keep OpenGL and its complexity out of the primary package, but also fully support hardware-accelerated rendering while developing a plotting library for Skia.
What do you think?

Hi Scott.
I also liked Skia library, simple interface and powerfull implementation. OpenGl just one of the posible backend, it can be setup by few lines of code. It's possible to make fallback to software render if no OpenGL hardware available. And all this backend things good separated from main canvas abstraction, so you can choose design that you like, separate controls in separate projects.
One negative thing about SkiaSharp, it's just a wrapper on native skia library. PInvoke make performance costs.  This problem can be solved by replacing multiple calls with constracting SKPath and single call to draw it(like i maded in last commits). This solution add some complexity to drawing...
Designer crashes these are mostly problems of designer, many people blame it.
I will plan to make some investigation to fix it  after SkiaBackend will be finished.
OpenTK.GlControl have some interesting solution. They create dummy control if Designer mode detected, and it works stable in designer.

I'd like to design a new plotting library (QuickPlot) to use Skia everywhere (no System.Drawing at all). It will render on a mouse-interactive user control. A separate project (QuickPlotGL) can inherit the QuickPlot user control and replace the drawing area with an OpenGL control (like you do with this pull). This way we keep OpenGL and its complexity out of the primary package, but also fully support hardware-accelerated rendering while developing a plotting library for Skia.

Sorry haven't time to look carefull at this for now(very busy at main work), but very interested, maybe later. As i think it's this PR way. Make abstract call like DrawRectangles(color, widths, heigths) to backend and resolve it in backend with specific call to lib. It's very flexible with low performance cost.
If you chose skia as a primiry lib, you can make things in different ways. For example you can once draw plot, and multiple zoom and pan without redrawing, just change transformation matrix (don't really tryed, but think it's possible).

For what it's worth, I'm finding that System.Drawing (System.Drawing.Common NuGet package) is faster than I expected, especially on .NET Core 3 using WPF...
There's a compiled demo here if you're interested in running it yourself
https://github.com/swharden/QuickPlot/tree/master/demos/compiled

On my machine i get about 50 FPS with you demo. But it not use heavy draw calls.
WPF have DirectX acceleration for UI, but i am not sure you can get it from inmemory rendering to bitmap. Onscreen rendering in wpf control can achive high performance, and it can be done with help of this PR. But ScottPlot WPF user control initialy lagging for no reason.

Observations of WPF performance

WPF has DirectX acceleration for UI ... onscreen rendering in WPF control can achieve high performance

I've tried many different ways to draw lines on the screen using WPF. Full-screen performance is great for 1-50 lines. At 1000 lines it becomes much slower than GDI+, even at large resolutions. A simple-case example is: 2019-10-09-WPF-Draw-Lines. I know there are slightly more efficient ways to draw lines (such as LineSegment) but it doesn't seem to make much difference. I have noticed that most of the slowness comes from anti-aliasing of long lines.
I think this is because WPF is optimized to place complex objects (images, shapes, etc) and move them around on the screen. It does not seem optimized to display a large number of small lines, especially as they are rapidly created and destroyed.
Thanks for your work on this PR!
System.Drawing.Common works everywhere, but is slow. SkiaSharp is almost perfect! You are doing a wonderful job with this PR, and although I am exploring alternative rendering methods I highly value the work and time you have invested into this project.
Is your NuGet package publicly available? I'd like to try it on my system. Ironically, all of my troubles with SkiaSharp have been because it crashes Visual Studio on my computer when I try to interact with user controls that require it. Once compiled, the executable runs great and demonstrate fantastic performance!

Microsoft has an interactive charting user control for WPF
https://github.com/microsoft/InteractiveDataDisplay.WPF
Performance is pretty poor in my opinion. I tested it by running the LineGraphSample and resizing it by dragging the corner. It's ~5 Hz update rate. If that's the best Microsoft can do with WPF, maybe I shouldn't feel bad...

Is your NuGet package publicly available? I'd like to try it on my system. Ironically, all of my troubles with SkiaSharp have been because it crashes Visual Studio on my computer when I try to interact with user controls that require it. Once compiled, the executable runs great and demonstrate fantastic performance!

Started work on fixing crashes, but surprisely spend a hour on tryies to crash designer. As a result designer crashes only on first build of clean project (just downloaded sources), because nuget packages not restored. After restoring works stable. But i totaly did nothing to make it stable.
Can you check last version of this PR and try to crash it? if you get success write steps to reproduce.
There are 2 suggestions why this fixed itself:

You replaced Plot.Render with stub in designer mode #131.
I update VS to 16.3.1, maybe they fixed something.

all of my troubles with SkiaSharp have been because it crashes Visual Studio on my computer when I try to interact with user controls that require it

Can you check last version of this PR and try to crash it? if you get success write steps to reproduce.

I can compile the project and the output works great.
If I open Form1.cs in Visual Studio, I get a designer crash
If I create a new blank Form2.cs and try to drag/drop the FormsPlotSkia user control onto it, I get another designer crash

This problem is identical to the problem I had with QuickPlot. I have tested this on 2 different computers now with the same result. Note that the program compiles fine! It's just visual studio that crashes.

what am I doing wrong?

What! I updated Visual Studio (to 16.3.3) like you suggested and it works now. My Visual Studio was less than a month old! (16.3.1)
This probably means I should move QuickPlot back to Skia 🤕

This probably means I should move QuickPlot back to Skia 🤕

I rewrote QuickPlot to use Skia. It seems to work now, and I even got SkiaSharp to install and run properly on Azure Pipelines when the tests are run. I placed QuickPlot packages on NuGet.
Performance looks great! I am glad you encouraged me to continue working on this.
If you want to check it out, make a new Windows Forms application, use NuGet to install QuickPlot.WinForms, add the user control to a form, and place the following code to the startup sequence:
int pointCount = 1000;
double[] xs = QuickPlot.Generate.Consecutative(pointCount);
double[] ys = QuickPlot.Generate.Random(pointCount);
interactivePlot1.figure.plot.Scatter(xs, ys);

It should work even without specifying a build target! 🤞

I closed this PR because I don't think it will ever get merged.
There's a lot of valuable code here though! Perhaps this code would be better housed in a Skia branch?

Sad but true, its too complex. Its already in branch of my fork, so i don't think it need another.


Add support for custom paths to RPCProvider

What was wrong?
With the advent of INFURA, which uses URI paths to encode information about your account, it's useful to be able to specify a custom path on which to submit RPC requests.
How was it fixed?
I added an optional 'path' parameter to the constructor for RPCProvider.
Cute Animal Picture

👍 thanks for the contribution.  I'll get this merged and released once CI is finished with it.

Thanks for the well written Python module. And a surprisingly quick response to a PR. :)


to-do: document expect matchers and utilities

Currently, expect is missing examples for matchers and utilities (example). Do we link to Jest's examples? Do we write our own?

We write our own :)


App Name not modified

I modified default app name to my desired name but it is not updated.
Code is shown below.
var params = <String, dynamic>{
   'id': _currentUuid,
   'nameCaller': widget.user.name!,
   'handle': widget.user.completeNumber,
   'appName': 'Welcome to VWChat',
   'type': 1,
   'extra': {
     "receiverName": user.name!,
     "receiver": user.completeNumber!,
     "receiverImage": user.imageUrl ?? "user.png",
     "receiverIsoCode": user.isoCode,
     "receiverNumber": user.number,
     "senderName": widget.user.name!,
     "sender": widget.user.completeNumber!,
     "senderImage": widget.user.imageUrl!,
   },
   'ios': <String, dynamic>{'handleType': 'number'}
 };

What should I do?

@ljsharp for iOS > 14 Apple removed it (default is App name will be used)

@hiennguyen92 unfortunately, this issue still happening in android (full screen) on top it's always showing "CallKit" instead of the app name.

did you find any solution to replace it with our appname pls reply

did you find any solution to replace it with our appname pls reply

@sunil-singh-chaudhary you can hide it with isShowLogo: false,
or you may need to change this image in source code https://github.com/hiennguyen92/flutter_callkit_incoming/blob/master/android/src/main/res/drawable-xxxhdpi/ic_logo.png
I will try to add a properties for it in the next version.


Update qGAN algorithm with analytic Gradients

What is the expected enhancement?
The current qGAN implementation employs numerical gradients.
Employing the gradient framework would enable to run this QML algorithm with analytic gradients.

Hi @Zoufalc !
If it's not a problem, I would like to work on this - is there anything you feel I could take a look at that would help me get started?

Hi @SashwatAnagolum,
Great!
As a starting point, I would suggest the qGAN paper, as well as the gradient tutorial.

Awesome, done!
What I've got so far is that we want to replace the numeric differentiation in the optimizer used here with one that uses a function that returns analytically computed gradients - does that sound right?
Also, if you're Christa Zoufal - amazing work, especially on your publication on QNN expressiveness!

Thanks a lot :)
Yes exactly. What now needs to be done is the definition of an analytic gradient function that is passed into the optimizer.
In order to get this gradient you need to get the probability gradients.
More explicitly, if you have a look at Eq. 8 and 9 in the qGAN paper.
These equations can be rewritten (using a frequentist perception of probability) as
L_G(phi, theta) = -\sum_l p^l[log(D_phi(g^l))]
where p^l refers to the probability to measure the respective output from the quantum generator.
Now, we can formulate the respective gradient as
d_theta L_G = -\sum_l d_theta(p^l)[log(D_phi(g^l))]
The same applies to L_D.
To get d_theta(p^l) you can use the gradient framework.
I.e. Gradient().convert(StateFn(qc_generator), parameters_generator), where the l^th entry corresponds to d_theta(p^l).
So to enable this, the quantum generator has to use parameters.

Issue was taken care of by #1450 so closing this


Improve responsiveness on Mobile

Increase popup width on mobile devices.

This issue is closed with PR #32.


Usage example code not doing anything?

I didn't know where else to post this. I've tried using the example code on the Readme, and it doesn't do anything, but does seem to activate the stepper motor.
#include "FastAccelStepper.h"

#define dirPinStepper    2
#define enablePinStepper 4
#define stepPinStepper   3  // OC1A in case of AVR

FastAccelStepperEngine engine = FastAccelStepperEngine();
FastAccelStepper *stepper = NULL;

void setup() {
   engine.init();
   stepper = engine.stepperConnectToPin(stepPinStepper);
   if (stepper) {
      stepper->setDirectionPin(dirPinStepper);
      stepper->setEnablePin(enablePinStepper);
      stepper->setAutoEnable(true);

      stepper->setSpeed(1000);       // the parameter is us/step !!!
      stepper->setAcceleration(100);
   }
}

void loop() {
    stepper->moveTo(1000);
}

Does the FastAccelstepper library have some kind of stepper.run() function like accelstepper does?
I'm interested in using this library for a robotics project. It would be great to have a simple sweep code to begin with (stepper move clockwise.. stop. Stepper move anti-clockwise. Stop)

Do you use atmega328 aka AVR or an esp32 ?
In case of atmega328, then the step pin is restricted as per Readme:
avr: This must be connected for stepper A to Pin 9 and for Stepper B to Pin 10.
The FastAccelStepper library does not need a stepper.run(). For AVR all is done via interrupts and for esp32 interrupt and tasks.
In order to check the HW, I recommend to first use the StepperDemo. Need to adapt the HW configuration first.
Then via serial you can issue commands. For example you could:

M1 - select the stepper 1
N - enable the output driver, if enable pin is connected. After this, the motor axle should be fixed.
T - drive motor with direct port access. Motor should run 3200 steps in one and then the other direction.
Try the ramp function with e.g. A1000 V1000 R1000

Thanks for the heads up. Wouldn't it be better to be able to customize this, though? Say, if I have a PCB that is already wired a certain way for certain pins?

For atmega328, this is hardwired into the uC. The timer output compare pins are fixed like this. And the high speed comes from using those hardware timers.
If you need the freedom of pin assignment, then you should consider AccelStepper, which does not have this restriction.

Thanks, this makes perfect sense now. Works perfectly!


Will it be possible to parse a comment column from a SQL query and insert it after the value of the column?

technologia TINYINT(3) NOT NULL COMMENT 'comment text1',
kabel INT(11) NOT NULL COMMENT 'comment text2',
mufty INT(11) NOT NULL COMMENT 'comment text3'

If this is a feature request, please re-phrase the title and we'll label it as such. Thanks.


[Query] Signal structured error (NoAnswer) instead of just Error

Okay, adding NoAnswer to St/X seems to be quite tricky, there the default answer is nil and a lot of code depends on it (there's 38 subclasses of Query in "base image"). More thinking is needed.

the default answer is nil and a lot of code depends on it. More thinking is needed.

After a few more hours of contemplating, I think I am starting to understand this a bit better.  We are looking at another example of consequences of dynamic typing — a fundamental design choice in Smalltalk — on compositionality.
Consider the following example of the original question, due to Kleisli, seemingly unrelated to computer programming: the equation
ax = b

can be solved for x as b/a.  What if a=0?  We can special-case it (NB: this is happening in the control plane) and say "unless a=0"; but what if ax=b is part of a larger investigation, so a and b come from some other calculations, and x is used some later calculation?  The permutations of special-cases can quickly become unmanageable.  Another approach is to special-case in the data plane and adjoin NaN to the numbers (the possible values of x).  So we are looking at a functor lifting numbers to maybe-numbers; and they compose because trying to operate on NaN gives you another NaN, etc.  This is just Maybe with a special name ("NaN" instead of "nil").
In Haskell, many functions "F" come in two variants, "F" and "maybeF".  In Smalltalk terms, "divide" would signal a ZeroDivide but "maybeDivide" would answer NaN.  Or, Array>>at: would signal SubscriptOutOfBounds but Array>>maybeAt: would answer nil.  This composes in the data plane, and while in Haskell it has to be explicitly plumbed through the Kleisli arrow, in Smalltalk the array member could itself be nil (e.g. if it came from a failed computation).
So for our Query, this resolves nicely the problem of "what happens if the answer does exist and is actually nil?"  The question that remains, is naming consistency, but we have that problem across the board.  It is in general difficult to say by just looking at a selector, whether a failure will be signaled in the control or in the data plane.  In MA I have been trying to use maybeF-style names, especially when transliterating from LH, but it's not exactly parallel because in Smalltalk the Maybe is implicit (anything can be nil).

I'd like to know how where this is going to be used.

At this point in time, there are no uses of Query in the MA codebase; everything still uses the old #runReader:initialState:.  This is going to change in the future but I would like to do more foundational work in this area, I want to have the Reader concept (which is — most stunningly!!! — just the left adjoint of function application, #value:) decoupled from what to key it off (Class or Symbol).  So perhaps we can pospone the change in #defaultResumeValue, and limit this PR to just introducing the class NoAnswer.  My use of it is in https://github.com/shingarov/MachineArithmetic/pull/384/commits/3c206296a66c0263a94b5199d54f43f9bfd55b76 (I meant to merge that later after we have concensus whether we like the NoAnswer exception at all, which we seem like it).

Something like 539dffa for now?


Coral site doesn't link to the correct examples folder here

Description
While the site  now notes that their examples aren't compatible with the Dev Board Micro it points to the wrong url. Should be: https://github.com/google-coral/coralmicro/tree/main/examples over this.
Click to expand!
Issue Type
Others
Operating System
No response
Coral Device
No response
Other Devices
No response
Programming Language
No response
Relevant Log Output
No response

Hello @eggheaded the link got updated. Thanks for reporting.


Documentation (README.md) is outdated

// Start container

docker.startContainer(id, hostConfig);"

Isn't up to date. hostConfig is not an argument to startContainer anymore

I fixed this already back in July along PR #200. However, it seems to have been reverted by @nurkiewicz as part of commit 87fc813476dad47c21281bf60b653804ad433ab0 in August. At that time, the old API wasn't in master anymore to my understanding.

@Khazrak @timoreimann Thanks for noticing this. I've reverted https://github.com/spotify/docker-client/commit/87fc813476dad47c21281bf60b653804ad433ab0


Fix mapping double count

Problem
During mapping, the reported results for total attempted and total success were incorrect.  mappingResults was being evaluated twice, and so the accumulators that tracked these stats were double what they should be - and the mapping was unnecessarily inefficient / taking longer to run.  Also, total success was reported as equal to the total attempted even when not all records were successful.
Solution

Persist mappingResults so that it is only evaluated once.  Persisting this does add some memory pressure - I noticed for example that the Oklahoma mapping runs out of memory locally on the default memory setting - but we agreed that more memory pressure should be ok.

Get rid of the totalCount accumulator.  It was unnecessary because encodedMappingResults.count was already being evaluated and was equivalent to totalCount.

Get rid of the successCount accumulator.  It was incorrect, and accumulators can be unreliable, e.g. if nodes fail in the middle of computations.  Since @mdellabitta discovered that S3 has read-after-write consistency in our use case, we can use that instead to read in successResults and count them.  It is more reliable and adds very little time to the overall process.  Alternately, we could persist successResults so that we could write and count them without evaluating more than once, but that would add another dose of memory pressure - and it would probably overflow to disk so might not be much faster in a cluster environment than reading from S3.

Testing
I tested this locally with Oklahoma data.
BEFORE
                                Mapping Summary

Provider................................................................OKLAHOMA
Start date...................................................02/27/2019 12:44:49
Runtime.............................................................00:01:47.426

Attempted................................................................229,340
Successful...............................................................229,340
Failed.......................................................................213

                              Errors and Warnings

Messages
- Errors.....................................................................213
- Warnings...............................................................322,489

Records
- Errors.....................................................................213
- Warnings...............................................................111,943

                                Message Summary
Warnings
- Missing recommended field, language.....................................89,720
- Missing recommended field, publisher....................................51,307
- Missing recommended field, creator......................................47,890
- Missing recommended field, date.........................................43,869
- Missing recommended field, place........................................38,188
- Duplicate, rights and edmRights.........................................14,728
- Missing recommended field, type.........................................13,267
- Missing recommended field, format.......................................12,425
- Missing recommended field, description...................................9,951
- Missing recommended field, subject.......................................1,144
- Total..................................................................322,489
Errors
- Duplicate, originalId......................................................213
- Total......................................................................213

AFTER
                                Mapping Summary

Provider................................................................OKLAHOMA
Start date...................................................02/27/2019 14:56:21
Runtime.............................................................00:01:00.101

Attempted................................................................114,670
Successful...............................................................114,169
Failed.......................................................................213

                              Errors and Warnings

Messages
- Errors.....................................................................213
- Warnings...............................................................322,489

Records
- Errors.....................................................................213
- Warnings...............................................................111,943

                                Message Summary
Warnings
- Missing recommended field, language.....................................89,720
- Missing recommended field, publisher....................................51,307
- Missing recommended field, creator......................................47,890
- Missing recommended field, date.........................................43,869
- Missing recommended field, place........................................38,188
- Duplicate, rights and edmRights.........................................14,728
- Missing recommended field, type.........................................13,267
- Missing recommended field, format.......................................12,425
- Missing recommended field, description...................................9,951
- Missing recommended field, subject.......................................1,144
- Total..................................................................322,489
Errors
- Duplicate, originalId......................................................213
- Total......................................................................213

Q. Were the accumulators vastly incorrect once the persist call was put in? Is there a downside to keeping them?
They do provide feedback during the course of the run in a way that count doesn't, and are viewable on the job dashboard... We could theoretically keep them in, but not use them for the final stats.

@mdellabitta
After the persist call was put in, the totalCount accumulator was correct, but the successCount accumulator was incorrect (i.e. it was equal to totalCount when it should have been less than totalCount).
The only downside I can see to the accumulators is that they might be inaccurate for reasons beyond our control.  I don't think they add a lot of overhead, so I suppose we could keep them for dashboard-viewing purposes.
If we keep them, would you want to keep them both or just one so that we can monitor the mapping operation on the dashboard?  If we keep them both, we will need to do more troubleshooting to figure out why the successCount accumulator was incorrect.

This last commit, https://github.com/dpla/ingestion3/pull/279/commits/f85979d51f169ec64fe2cf148b410b2c020fbc51, fixes inaccurate counts in the mapping summary.
Before, the Attempted count did not equal sum of Successful and Failed.  The problem was that the Failed count was based on the unique record ids in all the error messages.  This count was inaccurate because some records have missing or duplicate ids.  Now, Failed is simply calculated by subtracting Successful from Attempted.
I also noticed that the Error counts in the Message Summary were inaccurate for Duplicate, originalId and Missing required field, originalId, and Missing required field, dplaUri.  This was because the error messages were being filtered to only take distinct messages.  In all three of these cases, identical errors can be created in different records.  Now, these counts are accurate.

Removed distinct from getErrors method, since @moltude agreed it was unnecessary.


No way to turn off mbed cache

The documentation at least is unclear OR the possibility to turn off the mbed cache is missing.
jankii01@ubuntu:~/mbed/mbed-cloud-client-example$ mbed config cache none
[mbed] none now set as default cache in program "mbed-cloud-client-example"

So, how do you turn it off?
One needs to turn it off, because currently the cache is failing randomly and you get a completely random SW  package which either compiles or not or fails to work.

According to the readme, you're supposed to be able to set the option to none and that should disable it: https://github.com/ARMmbed/mbed-cli#mbed-cli-configuration
Does it look like CLI isn't actually disabling the cache at this point? Can you give us the steps to reproduce the issue?

No, I think the mbed config --global cache none does the trick, but the UI isn't perhaps the most explicit one. I looked into the code as well and it seem 3 different values would do the trick;
https://github.com/ARMmbed/mbed-cli/blob/master/mbed/mbed.py#L886-L889
none, off and disabled. Perhaps we could just modify the README.MD a bit to clarify how to turn it off. Now it's a bit "assumed" implicitly there.

Apparently the cache issue was also fixed, but it just hasn't been released? We should really release the fix, if we've fixed the issue.

@JanneKiiskila the cache issues were fixed. Did you update the latest mbed CLI? If not, care to update more often?

Indeed, update done - thanks for the tip, I was clearly behind - my apologies for that (0.9.1). Is there a way to subscribe for release notes?
Anyways, PR done to add the addition to README.md; https://github.com/ARMmbed/mbed-cli/pull/483

@JanneKiiskila I don't think there's anything like that as of this moment. You could always watch the mbed CLI repo (though you'll get other emails as well).
@screamerbg Release notes would probably be useful to send across the team, similar to what we do for mbed OS releases.

PR merged, issue closed.


Add subway direction (last station)

Clicking on bottom part does not help.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. For example:
I'm always frustrated when [...]
Describe the ideal solution
A clear and concise description of what you want to see in Organic Maps.
Describe alternatives you have considered

How do you solve this issue now with Organic Maps or other apps?
Attach any examples, screenshots, or screen recordings from other apps that help us to better understand the idea.

Additional context
Add any other context or screenshots about the feature request here.

Related to #17.


Create codeql.ymlhttps://github.com/github-linguist/linguist/compare/f75c570..3391dcc

Darfu99

Darfu99:development

stake.com/?c=BJVPQFu4
On Sun, 25 Aug 2024 at 08:00, Darfu99 @.***> wrote:

Darfu99
You can view, comment on, or merge this pull request online at:
https://github.com/pi-apps/pi-explorer/pull/142
Commit Summary

5f9e9ca
https://github.com/pi-apps/pi-explorer/pull/142/commits/5f9e9cadf80d1013da041b7a49196e26e8caef54
Create codeql.yml

File Changes
(1 file https://github.com/pi-apps/pi-explorer/pull/142/files)

A .github/workflows/codeql.yml
https://github.com/pi-apps/pi-explorer/pull/142/files#diff-12783128521e452af0cfac94b99b8d250413c516ec71fe6d97dbea666ff7ba27
(92)

Patch Links:

https://github.com/pi-apps/pi-explorer/pull/142.patch
https://github.com/pi-apps/pi-explorer/pull/142.diff

—
Reply to this email directly, view it on GitHub
https://github.com/pi-apps/pi-explorer/pull/142, or unsubscribe
https://github.com/notifications/unsubscribe-auth/BD3OMLFXIOPC4VD6HEY6C3LZTFXIVAVCNFSM6AAAAABNCHTNDKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ4DKMBXGQ3DMNY
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

My kyc applied review in progress verification long time

https://TONFree.win/?r=99933
On Tue, 27 Aug 2024 at 16:38, Bhavin995 @.***> wrote:

@.**** approved this pull request.
—
Reply to this email directly, view it on GitHub
https://github.com/pi-apps/pi-explorer/pull/142#pullrequestreview-2263594241,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BD3OMLAH2SFWWHDQNKSZGVDZTSFNFAVCNFSM6AAAAABNCHTNDKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDENRTGU4TIMRUGE
.
You are receiving this because you commented.Message ID:
@.***>

@.***
On Wed, 28 Aug 2024, 1:40 am florim1213, @.***> wrote:

https://TONFree.win/?r=99933
On Tue, 27 Aug 2024 at 16:38, Bhavin995 @.***> wrote:

@.**** approved this pull request.
—
Reply to this email directly, view it on GitHub
<
https://github.com/pi-apps/pi-explorer/pull/142#pullrequestreview-2263594241>,

or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/BD3OMLAH2SFWWHDQNKSZGVDZTSFNFAVCNFSM6AAAAABNCHTNDKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDENRTGU4TIMRUGE>

.
You are receiving this because you commented.Message ID:
@.***>

—
Reply to this email directly, view it on GitHub
https://github.com/pi-apps/pi-explorer/pull/142#issuecomment-2313489704,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BCEFRIEGSWZUUNXHLUVKKXLZTTP5NAVCNFSM6AAAAABNCHTNDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJTGQ4DSNZQGQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

سلام دوستان

https://TONFree.win/?r=99933
On Fri, 30 Aug 2024 at 13:14, Ferrarrimmd @.***> wrote:

سلام دوستان
—
Reply to this email directly, view it on GitHub
https://github.com/pi-apps/pi-explorer/pull/142#issuecomment-2320889259,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BD3OMLGW5WYHAKJ464VMDXLZUBHZLAVCNFSM6AAAAABNCHTNDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRQHA4DSMRVHE
.
You are receiving this because you commented.Message ID:
@.***>

https://TONFree.win/?r=99933
On Sat, 31 Aug 2024 at 02:03, Alexstriker @.***> wrote:

@.**** approved this pull request.
—
Reply to this email directly, view it on GitHub
https://github.com/pi-apps/pi-explorer/pull/142#pullrequestreview-2273678850,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BD3OMLBZXLH6NCPPJUUXGTTZUEB4TAVCNFSM6AAAAABNCHTNDKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDENZTGY3TQOBVGA
.
You are receiving this because you commented.Message ID:
@.***>

I don't know

https://TONFree.win/?r=99933
On Fri, 6 Sep 2024 at 00:58, Tituskoech @.***> wrote:

I don't know
—
Reply to this email directly, view it on GitHub
https://github.com/pi-apps/pi-explorer/pull/142#issuecomment-2332819528,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BD3OMLBTLRDFV3FWOQDVRETZVDO2FAVCNFSM6AAAAABNCHTNDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZSHAYTSNJSHA
.
You are receiving this because you commented.Message ID:
@.***>

https://TONFree.win/?r=99933
On Tue, 17 Sep 2024 at 19:36, Cyphergaminola @.***>
wrote:

@.**** approved this pull request.
—
Reply to this email directly, view it on GitHub
https://github.com/pi-apps/pi-explorer/pull/142#pullrequestreview-2310414035,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BD3OMLC4ZNACK6WXU3WKXB3ZXBSCPAVCNFSM6AAAAABNCHTNDKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDGMJQGQYTIMBTGU
.
You are receiving this because you commented.Message ID:
@.***>

https://TONFree.win/?r=99933
On Wed, 18 Sep 2024 at 11:37, Godstimendako @.***>
wrote:

@.**** approved this pull request.
—
Reply to this email directly, view it on GitHub
https://github.com/pi-apps/pi-explorer/pull/142#pullrequestreview-2312151928,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BD3OMLABH7PVCBGPX4WQLMDZXFCVJAVCNFSM6AAAAABNCHTNDKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDGMJSGE2TCOJSHA
.
You are receiving this because you commented.Message ID:
@.***>

All finished

Good

👍👌🙋

good

加速你的处理

On

Yes

gh pr checkout 142

Ok
On Tue, Dec 10, 2024, 11:11 AM ojichenel @.***> wrote:

gh pr checkout 142
—
Reply to this email directly, view it on GitHub
https://github.com/pi-apps/pi-explorer/pull/142#issuecomment-2532174010,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYIGK43FJXCBF7FLBEYQYQD2E4HDDAVCNFSM6AAAAABNCHTNDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMZSGE3TIMBRGA
.
You are receiving this because you commented.Message ID:
@.***>

Được gửi từ iPhone của tôiVào ngày 10 thg 12, 2024, lúc 18:17, Darfu99 @.***> đã viết:
On
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

Hi
On Tue, Dec 10, 2024, 11:10 AM ojichenel @.***> wrote:

Yes
—
Reply to this email directly, view it on GitHub
https://github.com/pi-apps/pi-explorer/pull/142#issuecomment-2532172452,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYIGK47GK3NWBS6OGVEDQXT2E4HATAVCNFSM6AAAAABNCHTNDKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMZSGE3TENBVGI
.
You are receiving this because you commented.Message ID:
@.***>


Additions and changes to rejected invoices email

Need to send a rejected invoices email for each acq unit: one to SUL<EMAIL_ADDRESS>and one to Law (lawlib-acq-payments@list.stanford.edu).
The rejected invoices includes more criteria than just a po line with an amount split in its fund distribution. The additional exclusions are in #745 . Probably need to keep track of why an invoice is excluded in order to group them together in the rejected invoices email.

Partially addressed in #796

Closed by #802


What is the correct format of the config file?

I've tried putting the following in a file wheel2deb.yml:
---
maintainer_name: 'Testuser'
maintainer_email<EMAIL_ADDRESS>
...

.. and run wheel2deb like so: wheel2deb convert -c ./wheel2deb.yml
The result in output/ is not a .deb but a directory with debian/, entrypoints/, missing.txt and src/ in it. If I don't add the -c-flag I do get a .deb.

You are missing a key with a regex to tell wheel2deb which wheels it should apply your config to:
Here is an example:
.:
  maintainer_name: 'Testuser'
  maintainer_email<EMAIL_ADDRESS>
PyTurboJPEG:
  revision: 2
  depends:
    - libturbojpeg0

sounddevice:
  revision: 2
  depends:
    - libportaudio2

The "." in ".:" is a regex that will match all wheels, i.e. wheel2deb will apply the specified maintainer_name to all wheels.

You can be more specific: "PyTurboJPEG" will only match wheel names starting with "PyTurboJPEG", e.g. PyTurboJPEG-1.7.0-py3-none-any.whl


ARIA-397 Added sample install/command files for different OS

Closing this PR and opening a new one that addresses Tom N's comments


Add unit tests for configuration lua module

What this PR does / why we need it:
This PR adds unit tests for the configuration lua module. We need this because we currently do not have unit tests on this module.

@hnrytrn Addressed your comments 👍

Looks pretty decent, will do one more once over. Can you PR this upstream instead of to the fork @fmejia97 ?

@ElvinEfendi Lua tests are now been executed under a headless nginx, therefore we now have open resty API's available for use on our tests without the need to stub / mock them. The tests are executed on the timer phase, therefore unfortunately not all open resty's API's are available (just those available under this phase). In order for the tests to work we still need to mock the ngx global variable, otherwise we get API disabled in the current context. The only update I did to the test was to remove setting the package path at the beginning of the file, since the package path is set when running the resty cli.

Please delete all should prefixes in it calls. you can just say it("does that")

@ElvinEfendi Updated the PR with your comments

this's done in upstream


How to get the pretrained-model?

Thanks for your excellent work!
I would like to inquire how to get the model checkpoint file for pre-training via argo dataset that you mentioned in the README?

In the meantime, happy Mid-Autumn Festival ^_^

Thank you for your interest in our project. Unfortunately, due to certain restrictions, we are currently unable to open-source the pre-trained checkpoints. However, with the provided data processing scripts and model code, you should be able to train the model on your own.


Update internal use of old macros

We introduced a consistent set of NGEN_WITH_FOO macros, and started using them. We should go through and replace uses of the old, inconsistent ones.

$ git grep -o -h -E '[A-Z_]*_ACTIVE' | sort | uniq
BMI_ACTIVE
BMI_C_LIB_ACTIVE
BMI_FORTRAN_ACTIVE
LSTM_TORCH_LIB_ACTIVE
MPI_ACTIVE
NETCDF_ACTIVE
NGEN_ACTIVE
NGEN_BMI_CPP_LIB_TESTS_ACTIVE
NGEN_BMI_C_LIB_ACTIVE
NGEN_BMI_C_LIB_TESTS_ACTIVE
NGEN_BMI_FORTRAN_ACTIVE
NGEN_BMI_FORTRAN_LIB_TESTS_ACTIVE
NGEN_BMI_PY_TESTS_ACTIVE
NGEN_C_LIB_ACTIVE
NGEN_FORCING_ACTIVE
NGEN_MPI_ACTIVE
NGEN_MPI_TESTS_ACTIVE
NGEN_OUTPUT_ACTIVE
NGEN_ROUTING_ACTIVE
ROUTING_PYBIND_TESTS_ACTIVE
UDUNITS_ACTIVE

Plus some others, e.g.
ACTIVATE_PYTHON
#ifdef WRITE_PID_FILE_FOR_GDB_SERVER
#ifdef __APPLE__
#ifdef __GNUC__
#ifdef __cplusplus
#ifndef MPI_HF_SUB_CLI_FLAG
#ifndef NCEP_OPERATIONS

I'm going to create a separate issue for ACTIVATE_PYTHON, and call the present PR done. There's a lot more instances of that one, and I'm about done for the day.


Add dataSource.address and .network host exports

Adds support for basic data source introspection. See https://github.com/graphprotocol/graph-ts/pull/90.
Resolves https://github.com/graphprotocol/support/issues/3. Resolves https://github.com/graphprotocol/support/issues/37.

@leoyvens I still approve. 😁

@Jannis ha thanks.


🦠 User Guidance token issue

The token expired in User Guide , updated but some further issues ensued which need ironing out...
The same docs keep being alerted for review, despite the fact that they have been reviewed recently:

https://user-guide.operations-engineering.service.justice.gov.uk/documentation/services/dockerhub.html
https://user-guide.operations-engineering.service.justice.gov.uk/documentation/services/maintenance-pages.html

https://mojdt.slack.com/archives/C033QBE511V/p1722418079238389
I think ☝ issues are to do with the deploy failing due to something else.

The failing deploy is likely due to mismatched upload artefacts and download artefacts action versions. Fix - bump all.


Allows display of Solr function query results per document

Changelog

Modify fieldSpecSvc to accept fields of role "function"
fieldSpecSvc returns these to the fieldList of a syntax amiable to Solr function queries, ie function:foo -> foo:$foo
normalDocsSvc consumes function responses in Solr docs and normalizes them into the "subs" array
Associated tests

Acceptance Criteria

Specify some complex query in Splainer/Quepid like

q=text:law&
someFunctionQuery=tf('text','law')

Tell Splainer/Quepid to display the result of someFunctionQuery in the field list
catch_line text function:someFunctionQuery

Confirm you see "someFunctionQuery" as a field in the displayed documents with a numerical value

Allowable variations
We allow some variation in this syntax to account for common shortcuts or likely errors. The following should all be equivalent to the above syntax.
f:someFunctionQuery
func:someFunctionQuery
f:$someFunctionQuery
func:$someFunctionQuery

I allow the $ because Solr developers will often do this out of habit.
What won't work
We don't currently support directly specifying a function query. For example, this in your field list won't work
func:tf('text','law')

Elasticsearch is also not supported.
This syntax should silently fail in these cases (nothing is displayed).

:+1:


Fixed a side effect on double-to-long conversions in TileSystem

Impacted classes:

ProjectionTest: added @Maradox's unit test method test_conversionFromPixelsToPixels; removed delta parameter in tests when possible
TileSystem: fixed a side-effect in method ClipToLong ; deprecated now unused method Clip

the change looks fine, however i'm not familiar with how ClipToLong is used and what possible side affects this may cause

@spyhunter99 ClipToLong is only used to round mercator geo point coordinates (from 0 to 2^zoom, as double) into pixel (from 0 to 2^zoom - 1, as long). Nothing related to tiles.
Basically, we used to say longValue = Math.floor(doubleValue - .5).
Now it's longValue = Math.floor(doubleValue).
The impact on UI is that Markers will be moved one pixel to the right in half of the cases, or something like that. And the rounding computation is now more accurate.


log arbitrary headers

Feature request
We should be able to log arbitrary headers to STDOUT for debugging / etc. We should be setting the names of the headers in the lorax startup args or an env variable or something.
Motivation
Helps debugging and observability.
Your contribution
I can do this PR :)

I think this can be done entirely in the router part of the code


Add channel links to profile pictures in media controls

yes

i can't figure out git for the life of me, ignore this


Instead of copying and pasting contracts move to a dependency

The simplest solution is to merge it into hmt-escrow and archive this repo.

Done, merged to monorepo


How we going to create issues

Proposal
We can have a general proposal for 3 months work.
We can split work to individual technical tasks as follows:

ping @harshithdwivedi @wtrocki for idea

Yeah that works for me

Looks good 👍


let virtiofsd and qemu run in parallel

Currently, qemu will be launched after virtiofsd start if virtiofs enabled,
which is not necessary. Both of virtiofsd and qemu only depend on unix
socket file. so we can run virtiofsd and qemu in parallel after socket
file settles down.
tests show it will speed up startup time of kata around dozens to one
hundred milliseconds.
Fixes: #2483
Signed-off-by: Jianyong Wu<EMAIL_ADDRESS>@jodh-intel @grahamwhaley @bergwolf @Pennyzct

/test

nitpick. Your commit has the title subsystem: let virtiofsd and qemu run in parallel. - you are meant to replace subsystem with the name of the subsystem the patch affects, in this case, probably something like vc: qemu: let virtiofsd and qemu run in parallel. would work.

@grahamwhaley  oh! I have misunderstood the subsystem. thanks.

/test

I'm surprised that this change makes a measurable difference since startup was already parallelized.  cmd.Start() does not wait after fork+exec (https://golang.org/pkg/os/exec/#Cmd.Start) and therefore QEMU will already be launched while virtiofsd is starting up.
Also, does moving the code into goroutines guarantee parallelism?  I thought Go has an M:N thread model and requires explicit configuration to make goroutines always run in separate threads.
I don't object to this patch but I don't understand why it's faster and whether it will be faster on all systems.  I would suggest investigating the performance difference using tracing (e.g. perf record -a -e sched:* -e syscalls:*) to understand why there is a difference.  Then we'll know whether the improvement is accidental or really something that can be relied on.

Hi @stefanha , yeah, in most cases, without this patch, virtiofsd has nearly run in parallel with qemu. But it is not the case of the first kata runs in the system or you clean cache(including inode, page...) before run kata. Loading virtiofsd binary will consume dozens to more than a hundred of milliseconds in different systems. This patch will work well in that case. If there is cache in the system, this patch can also speed up a few milliseconds in my tests in different system.

Hi @stefanha , yeah, in most cases, without this patch, virtiofsd has nearly run in parallel with qemu. But it is not the case of the first kata runs in the system or you clean cache(including inode, page...) before run kata. Loading virtiofsd binary will consume dozens to more than a hundred of milliseconds in different systems. This patch will work well in that case. If there is cache in the system, this patch can also speed up a few milliseconds in my tests in different system.

This performance improvement seems like an accident - a side-effect of how your kernel, Go implementation, etc work today - and not something that is guaranteed work in the future or on anyone else's machine.
The kata-runtime process already forked and was able to continue running before this patch.  The exec syscall, which is where the cold page cache slows things down, is not invoked from the parent kata-runtime process, so it's unclear to me why this patch should make a difference.
Also, I mentioned that goroutines are not mapped to threads 1:1, so this patch is not guaranteed to add parallelism.  It depends on the Go implementation and configuration.  If threading is necessary then it should be done explicitly.
Can you explain why the patch improves performance?

I agree with you @stefanha. Before this patch virtiofsd and qemu have started in different processes and run in parallelism. This patch will not  improve it again.
If I find nothing special after further tests, I will close this PR.

I agree with you @stefanha. Before this patch virtiofsd and qemu have started in different processes and run in parallelism. This patch will not improve it again.
If I find nothing special after further tests, I will close this PR.

@jongwu I hope I did not discourage you.  The startup time improvement you measured was real and would be nice to have!  Now it's a question of figuring out why and how to make it work every time.

I agree with you @stefanha. Before this patch virtiofsd and qemu have started in different processes and run in parallelism. This patch will not improve it again.
If I find nothing special after further tests, I will close this PR.

@jongwu I hope I did not discourage you. The startup time improvement you measured was real and would be nice to have! Now it's a question of figuring out why and how to make it work every time.

Very kind of you @stefanha , I will do further research before next step. Thanks for your kindness and help!

Hi @stefanha, after lots of elaborate test, maybe I can explain how this patch work.
Originally,  virtiofsd startup func use vfork() to fork a child process and then exec, which may block the parent process, so that may lead to a fake parallelism with qemu.
This patch may cancel this issue and make virtiofsd and qemu run in parallelism virtually.
This patch will work more efficient when start of kata in the first time, means with no cache available and with low speed io device.
WDYT?

Hi @stefanha, after lots of elaborate test, maybe I can explain how this patch work.
Originally, virtiofsd startup func use vfork() to fork a child process and then exec, which may block the parent process, so that may lead to a fake parallelism with qemu.
This patch may cancel this issue and make virtiofsd and qemu run in parallelism virtually.
This patch will work more efficient when start of kata in the first time, means with no cache available and with low speed io device.
WDYT?

@stefanha , just ignore this rush conclusion.  I have no decided conclusion now.

@stefanha ,  the delay exist in preparing config for qemu before launchCustomqemu func, I can't explain very clear why this delay exist, maybe because of gc or schedule but no relationship with the intention of this PR.
optimization for that delay relates to govmm/qemu so I will close this PR.


Avoid using + signs in S3 urls for Bundles manifest

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/lgtm
/approve


[Backport 17.07] service connectivity issue

First commit moves the peerAdd/peerDelete operation behind a channel
Second commit deals with a possible deadlock on the pMap Lock. Because of the lazy instantiation logic of the overlay driver, the initSandbox has in all cases to fetch peers from the DB. Moreover this operation has to be atomic on the whole table to avoid a race with subsequent peerAdd/peerDelete that can asynchronously come from the networkDB.
Backport of: #1893

no conflict during cherry-pick


You or one of the Babel plugins you are using are using Flow declarations as bindings

When using //recursive type, babel throw this warning:
You or one of the Babel plugins you are using are using Flow declarations as bindings.
    Support for this will be removed in version 6.8. To find out the caller, grep for this
    message and change it to a `console.trace()`.

Hi @singggum3b, I'm not able to reproduce. Also, currently the babel version used in tests is 6.9.0. Could you please provide more infos in order to reproduce this issue?

Hi @gcanti , thanks for your fast response.
Here's the info :
"babel-core": "^6.10.4",
    "babel-eslint": "^6.1.2",
    "babel-loader": "^6.2.4",
    "babel-plugin-tcomb": "^0.3.7",
    "babel-polyfill": "^6.9.1",
    "babel-preset-es2015": "^6.9.0",
    "babel-preset-react": "^6.11.1",
    "babel-preset-stage-0": "^6.5.0",
    "eslint": "^3.1.1",
    "eslint-config-airbnb": "^9.0.1",
    "eslint-plugin-flow-vars": "^0.4.0",
    "eslint-plugin-flowtype": "^2.3.1",
    "eslint-plugin-import": "^1.11.1",
    "eslint-plugin-jsx-a11y": "^2.0.1",
    "eslint-plugin-react": "^5.2.2",
    "flow": "^0.2.3",
    "flow-bin": "^0.29.0",
    "html-webpack-plugin": "^2.22.0",
    "json-loader": "^0.5.4",
    "react": "^15.2.1",
    "react-dom": "^15.2.1",
    "react-router": "^2.6.0",
    "source-map-loader": "^0.1.5",
    "tcomb": "^3.2.8",
    "webpack": "^1.13.1",
    "webpack-dev-server": "^1.14.1"

bugged code:
// recursive
export type JSONComponentType = {
	id: string,
	name: string,
	path?: string,
	section: string,
	exactPath?: boolean,
	type: "component",
	props: {
		[name: string]: any
	},
	componentsList?: Array<JSONComponentType>,
}

Ah, got it! The problem araises when you define an exported recursive type
// recursive
export type A = {
  a?: A
};

p.s. As a (hopefully temporary) workaround this should work
// recursive
type _A = {
  a?: A
};

export type A = _A;

Thanks a lot !

Released a fix in https://github.com/gcanti/babel-plugin-tcomb/releases/tag/v0.3.8


Supporting AudioDirectives with Multi-Worker Deployments

While building a skill that allows multiple users to stream various audio podcasts and live streams, I found the current means of tracking audio stream information by using LocalStack caused issues once deployed in production configured with Gunicorn and multiple workers.
We found we couldn't guarantee which worker would service the request from Amazon, thus we couldn't guarantee the worker knew enough about the user's current audio stream to properly handle the suspend/resume AudioDirectives. Amazon seems to only provide the stream token in requests, but not the stream url itself. We also needed to track this information across many users simultaneously.
The proposed solution is to leverage the existing cache framework inside Werkzeug. It supports a simple in-memory form for single-worker use cases (SimpleCache). More importantly, it also supports remote cache layers such as Redis, allowing for a safe, scalable way to share audio stream data between Flask-Ask workers. (We're currently running Redis being used by 2 workers in production.)
This PR addresses the above by doing the following:

Switches to uuid.uuid4() for generating tokens instead of the current random.randint() allowing for lower risk support of many simultaneous users
Leverages werkzeug.contrib.cache to provide a caching layer for the stream information, using a key value of the context.System.user.userId unique value provided in each Amazon request.

Additional changes made in support of the above enhancements include:

Added additional tests runnable by python setup.py test or python -m unittest to validate some of the new behavior. This includes the beginnings of keeping an eye on the samples by at least starting them and sending a LaunchRequest to make sure code changes at least don't break their ability to start up.
Adding pip-friendly requirements.txt and requirements-dev.txt files.
Setting the path keyword parameter in Ask.init_app() to default to templates.yaml just as the Ask.__init__() constructor. (This change from a recent merge into master broke code using the .init_app() method.)

There's a lot going on in this PR, so please let me know what questions you have!

Wow! Thank you so much! Lots of work into this one. It's going to take me a while to review it, but I'll try to get it merged soon. Thank you!

nice work! this is awesome


question for plane detection and mesher outputs for backend

Hi, I have two questions about the Mesher module that I want to confirm.
1, I did not see you call the plane detection related functions in the Mesher module although you put some related functions in the code. Is that correct?
2, I did not see you pass any outputs (from the Mesher) back to the backend (pose optimization & loop closure) for further optimization. Is that correct?
Thank you so much for your time! I really appreciate it.

Both of these are correct at this time. For joint pose-mesh optimization, we use Kimera-PGMO (located here.


feature/data-events

Oracle Data Events

sending data to api
sending data events to api

@jdutchak Looked through the updates.  To confirm, the LLM chats are sent in the same way as other worker output?


Seems very out of date. Dcs to not appear to match workflow at all

As per title

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0f07f556-0bd5-c782-f34f-ba9c22244763
Version Independent ID: a4803b09-98a7-11b5-4d3f-c710168f3b0b
Content: Add a non-gallery application - Microsoft identity platform
Content Source: articles/active-directory/manage-apps/add-non-gallery-app.md
Service: active-directory
Sub-service: app-mgmt
GitHub Login: @kenwith
Microsoft Alias: kenwith

@swinster
Thanks for your feedback! I have assigned this issue to the author who will investigate and update as appropriate.

@swinster thank you for your feedback. I know it takes time to open an issue and I appreciate it. I have refactored this content into the quickstart article on adding an app https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/add-application-portal and setup a redirection. This should be live shortly. I recently joined the team and am updating all of the content in this area. If you spot any other areas that should be prioritized before I get through all the content please let me know. Thanks again!

#please-close


feature request - tf.stack

Given a tensor x[i,j,k] in a graph, I often want to construct tensors like X[i,j,k,p,q] = x[i,j,k].
You can get this done using tf.pack, but it can get kind of messy if you are adding a bunch of dimensions.
It would be nice if there was a function tf.stack(x,list) which added dimensions to a tensor as above. The name is motivated by the following situation. If x is a matrix, then tf.stack(x,[n]) would create a 3-tensor by stacking the matrix up n times. I imagine that tf.stack would fit nicely under tf.tile in the documentation
It is possible that there is already a better way to do this than using tf.pack, but if so, it is not in the docs.

Try tf.tile ?

If I understand correctly, tf.tile can't change the dimension of a tensor. What is the best way to cast a tensor of dimension [a,b,c] to a tensor of dimension [a,b,c,1,1]?

You can use tf.expand_dims, then followed by tf.tile, perhaps!

@vrv that works very well.

Glad I could help, and no worries -- you might also ask these types of questions on StackOverflow -- the community is also quite helpful for these kind of modeling / programming questions.


Manually set the location / zone

Would it be possible to manually define the location / zone where the player is?
For example, some players use geforce now and would not have access to .log or .txt files.
I know it may sound stupid, but any button to scroll through the area and then selecting the one you are in would be very cool for this case.
Anyway, the tool is sensational. Thanks for the work.

I need too for GFN
great tool!

I need too for GFN
great tool!

If the tool comes up using GFN, then you don't need to change the exe.
If there is a mouse, you can you select the zone you want from the drop down menu. I used to have hotkeys for cycling the zones but no one was using them. I can add them back if they would be beneficial, but I need to know what would be a good hotkey for you.

If the tool comes up using GFN, then you don't need to change the exe.
If there is a mouse, you can you select the zone you want from the drop down menu. I used to have hotkeys for cycling the zones but no one was using them. I can add them back if they would be beneficial, but I need to know what would be a good hotkey for you.

ctrl + Page Up and ctrl + Page down seem like good hotkeys, right? what do you think? ty man !

ctrl + Page Up and ctrl + Page down seem like good hotkeys, right? what do you think? ty man !

Ok, I added a stealth hotkey for cycling to the next zone. I don't have room in the settings for it now, but I'll expand it in the next release a few weeks from now.
For the time being please download the master branch and test it out: https://github.com/JusKillmeQik/PoE-Leveling-Guide/archive/master.zip
Only Control+PageDown works for now, but you can cycle through the whole list that way and it will loop back around. Try it out and if you find you need a hotkey for going backwards, or a hotkey for changing Acts, please let me know.

Ok, I added a stealth hotkey for cycling to the next zone. I don't have room in the settings for it now, but I'll expand it in the next release a few weeks from now.
For the time being please download the master branch and test it out: https://github.com/JusKillmeQik/PoE-Leveling-Guide/archive/master.zip
Only Control+PageDown works for now, but you can cycle through the whole list that way and it will loop back around. Try it out and if you find you need a hotkey for going backwards, or a hotkey for changing Acts, please let me know.

Great, arriving from the work done the tests and I give you some feedback.
Another point, I made some changes so that the program could be used in GFN, would it be of any use to you? I believe it is a silly code and is probably already planned for the next update.

Great, arriving from the work done the tests and I give you some feedback.
Another point, I made some changes so that the program could be used in GFN, would it be of any use to you? I believe it is a silly code and is probably already planned for the next update.

Are you familiar with Git enough to fork my repo and create a merge request with your changes? I don't have GFN so I don't have a way of testing anything. If you show me what you did, I can make sure to add it to the right places. I may even be able to get it working automatically so I don't need the extra hotkey.

Are you familiar with Git enough to fork my repo and create a merge request with your changes? I don't have GFN so I don't have a way of testing anything. If you show me what you did, I can make sure to add it to the right places. I may even be able to get it working automatically so I don't need the extra hotkey.

I'm not exactly familiar with git, but it doesn't look like a seven-headed bug. I believe I can fork and then "create a new branch for this commit and initiate a pull request".
About automating zone change, I'm really curious about this solution. Bearing in mind that the logs are unfortunately on GFN servers. It would be really cool.

I'm not exactly familiar with git, but it doesn't look like a seven-headed bug. I believe I can fork and then "create a new branch for this commit and initiate a pull request".
About automating zone change, I'm really curious about this solution. Bearing in mind that the logs are unfortunately on GFN servers. It would be really cool.

If you'd like to just list the changes you made to get the tool to work with GFN I can also make them myself. Let me know.