citadel client having trouble to create certificate with remote istiod

Bug description
Seeing this in the ingress gw pod in the remote cluster log when configured to use main cluster's istiod.  @irisdingbj @myidpt I should have the change from you https://github.com/istio/istio/pull/22989
generateSecret called.
2020-04-29T02:06:38.232657Z	error	citadelclient	Failed to create certificate: rpc error: code = Unauthenticated desc = request authenticate failure
2020-04-29T02:06:38.232693Z	error	cache	resource:default request:e62d77ea-45b1-43b9-90db-38e15b65865d CSR hit non-retryable error (HTTP code: 0). Error: rpc error: code = Unauthenticated desc = request authenticate failure
2020-04-29T02:06:38.232726Z	error	cache	resource:default failed to generate secret for proxy: rpc error: code = Unauthenticated desc = request authenticate failure
2020-04-29T02:06:38.232739Z	error	sds	resource:default Close connection. Failed to get secret for proxy "router~<IP_ADDRESS>~istio-ingressgateway-74d5f6f58b-8rdxs.istio-system~istio-system.svc.cluster.local" from secret cache: rpc error: code = Unauthenticated desc = request authenticate failure
2020-04-29T02:06:38.232830Z	info	sds	resource:default connection is terminated: rpc error: code = Canceled desc = context canceled
2020-04-29T02:06:38.233091Z	warning	envoy config	[bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:92] StreamSecrets gRPC config stream closed: 16, request authenticate failure

Expected behavior
Able to get the certificate from remote istiod.
Steps to reproduce the bug
Version (include the output of istioctl version --remote and kubectl version and helm version if you used Helm)
master (close to 1.6 beta0 code level)
How was Istio installed?
https://github.com/istio/istio/issues/22934
Environment where bug was observed (cloud vendor, OS, etc)

@myidpt seems we are missing some code when citadel agent sends the create cert request to remote istiod to authenticate the create cert request... would appreciate your thoughts and expertise on this!

@linsun can you share the steps to recreate the problem? also what is the log looks in the istiod in the main cluster?

@irisdingbj I'm having the following in istiod log
2020-04-29T02:06:17.469020Z	warn	failed to validate the JWT against cluster "remote0": tokenreviews.authentication.k8s.io is forbidden: User "system:serviceaccount:istio-system:istio-reader-service-account" cannot create resource "tokenreviews" in API group "authentication.k8s.io" at the cluster scope
2020-04-29T02:06:17.469177Z	warn	serverca	Authentication failed: Authenticator ClientCertAuthenticator at index 0 got error: no verified chain is found. Authenticator KubeJWTAuthenticator at index 1 got error: failed to validate the JWT: no remote cluster found. Authenticator ClientCertAuthenticator at index 2 got error: no verified chain is found.
2020-04-29T02:06:17.469207Z	warn	serverca	request authentication failure
2020-04-29T02:06:20.966696Z	warn	failed to validate the JWT against cluster "remote0": tokenreviews.authentication.k8s.io is forbidden: User "system:serviceaccount:istio-system:istio-reader-service-account" cannot create resource "tokenreviews" in API group "authentication.k8s.io" at the cluster scope
2020-04-29T02:06:20.966780Z	warn	serverca	Authentication failed: Authenticator ClientCertAuthenticator at index 0 got error: no verified chain is found. Authenticator KubeJWTAuthenticator at index 1 got error: failed to validate the JWT: no remote cluster found. Authenticator ClientCertAuthenticator at index 2 got error: no verified chain is found.
2020-04-29T02:06:20.966791Z	warn	serverca	request authentication failure

I do have the secrets in the main cluster by create secret cmd:
$ k get secrets -n istio-system
NAME                                               TYPE                                  DATA   AGE
cacerts                                            Opaque                                4      27h
default-token-g4k26                                kubernetes.io/service-account-token   3      27h
istio-ingressgateway-service-account-token-6vf65   kubernetes.io/service-account-token   3      27h
istio-reader-service-account-token-8rp7b           kubernetes.io/service-account-token   3      27h
istio-remote-secret-remote0                        Opaque                                1      22h

steps:
main cluster, deploy with instruction: https://istio.io/docs/setup/install/multicluster/shared/#main-cluster
remote cluster, deploy with pilot.enabled=false
Need to manual create cm istio-ca-root-cert (unless has Iris's change)
run create secret cmd on main cluster to create token for accessing remote cluster
Check logs for istio ingress gw.

@linsun we guard our cluster roles with flag now. you need to add -set values.global.centralIstiod=true  besides the instructions in  https://istio.io/docs/setup/install/multicluster/shared/#main-cluster in both main and remote cluster. Let me know if you still have issues after this change.

After setting values.global.centralIstiod=true on both main cluster and remote cluster, this issue is gone, at least not in logs.  Thank you @irisdingbj !!

i'm getting similar logs like here, i try to install the multi-primary on different networks
$ cat <<EOF > cluster1.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  values:
    global:
      meshID: mesh1
      multiCluster:
        clusterName: cluster1
      network: network1
      **centralIstiod: true**
EOF

is my understanding correct that i should add centralIstiod at the cluster1.yaml and cluster2.yaml?

I meet the same question. How can it solve in Istio 1.10+?


New created volume doesn't exist, if original volume is deleted in ceph

Reproduced steps:

create VM
create data volume from disk offering
attach data volume to VM.
detach data volume
create data volume template from data volume
delete data volume created in step 2
create data volume from data volume template, which is created in step 5
delete data volume tempalte
attach the new data volume to VM will fail:

Unable to attach data volume to a vm, details: failed to attach data volume[uuid:5e3017066bfd495db5dcd98f7f08790b, installPath:ceph://bak-t-850848dad53a44a0b9aeb4130fd77fbd/f56d498b31504806b6a94c017ee50331] to vm[uuid:b2911dda274c4739a7bd8c50a054a56b, name:virtualRouter.l3.00843d787eaa47cbae9cb88bd022dd20] on kvm host[uuid:edc0ace252b74c52a4265b3d51b43c91, ip:<IP_ADDRESS>], because unable to attach the volume[5e3017066bfd495db5dcd98f7f08790b] to vm[uuid: b2911dda274c4739a7bd8c50a054a56b], internal error: unable to execute QEMU command '__com.redhat_drive_add': could not open disk image rbd:bak-t-850848dad53a44a0b9aeb4130fd77fbd/f56d498b31504806b6a94c017ee50331:id=zstack:key=AQAUEOBViOzkDxAA1NnSV6YFA6vdtB3uRFtgJQ==:auth_supported=cephx;none:mon_host=bji-bm-node2:6789: Could not open 'rbd:bak-t-850848dad53a44a0b9aeb4130fd77fbd/f56d498b31504806b6a94c017ee50331:id=zstack:key=AQAUEOBViOzkDxAA1NnSV6YFA6vdtB3uRFtgJQ==:auth_supported=cephx;none:mon_host=bji-bm-node2:6789': No such file or directory.]

fixed in a0ce2c56b1cd7c5b2776b35de36f820bfe5a3ac9


Get File Object SHA

Something like this:
{
  repository(owner: "appatalks", name: "test_repo") {
    object(expression: "main:my-json-file.json") {
      ... on Blob {
        oid
      }
    }
  }
}

Example Response:
{
  "data": {
    "repository": {
      "object": {
        "oid": "7a4****c4f"
      }
    }
  }
}

Needed for example if an actor wants to update an object

Added
https://github.com/appatalks/the-powerQL/blob/main/json/15_g_get_obj_sha.json


wikidot.com

However, there are also some great cable television offerings produced by conventional cable television businesses, so before you purchase to be sure to carefully check exactly what your own funnel possibilities. wikidot.com

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: efe1b860-8d24-1742-4f20-c3e75cacbba3
Version Independent ID: 2ac226c0-df97-f729-fe09-3d51d61b9083
Content: Azure Backup FAQ
Content Source: articles/backup/backup-azure-backup-faq.md
Service: backup
GitHub Login: @markgalioto
Microsoft Alias: markgal;arunak;trinadhk;sogup;

However, there are also some great cable television offerings produced by conventional cable television businesses, so before you purchase to be sure to carefully check exactly what your own funnel possibilities. http://adildnal1.bravesites.com/ docs.microsoft.com


Add python 3.11 to CI

Codecov Report

Merging #490 (826c380) into main (8738ff4) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #490   +/-   ##
=======================================
  Coverage   91.89%   91.89%
=======================================
  Files          18       18
  Lines        2023     2023
=======================================
  Hits         1859     1859
  Misses        164      164

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Cannot find module './build/Release/analytics.node'

Hi guys, I'm trying to run Zenbot on Ubuntu server 16.04 with node v7.10.1, npm 4.2.0 and python 2.7. After npm install and npm link, if I try to launch any zenbot commands I get this:
module.js:472
    throw err;
    ^

Error: Cannot find module './build/Release/analytics.node'
    at Function.Module._resolveFilename (module.js:470:15)
    at Function.Module._load (module.js:418:25)
    at Module.require (module.js:498:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/home/leota/zenbot/node_modules/forex.analytics/index.js:1:79)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)

Has anyone experienced similar issues?

Same issue here, Debian with Node v8.2.1, npm 5.3.

Working! Problem was that I run npm install without having "build-essential" installed on my machine, so I installed the package and run npm install again, but without completely wiping previously installed node_modules. So If you're starting with a fresh new install (I'm testing on DigitalOcean droplet Ubuntu 16.04) this has worked for me:
    #install node
    curl -sL https://deb.nodesource.com/setup_8.x | bash -
    apt-get install nodejs

    #install build-essential
    apt-get install build-essential

    #install python 2
    apt-get install python

    # clone repo and install
    git clone https://github.com/carlos8f/zenbot.git
    npm install
    #optional
    npm link

Don't forget to also install mongodb :https://www.digitalocean.com/community/tutorials/how-to-install-mongodb-on-ubuntu-16-04

Same issue on aws ubuntu 16.04 and aws amazon ami linux.

Same issue here on Ubuntu server 16.04 with node v6.11.2, npm 3.10.10 and python 2.7.12:
/bitcoin/zenbot# zenbot
module.js:471
    throw err;
    ^

Error: Cannot find module './build/Release/analytics.node'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/bitcoin/zenbot/node_modules/forex.analytics/index.js:1:79)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)

Guys, please upgrade to nodejs 8, that should solve the build problem.
Please make sure to rm -rf node_modules && npm install to reinstall everything.
Please report back!

I had the same problem and now it works after upgrade do node-js 8 and clear modules folder
thanks

A small log from the Discord channel from a user with the same error:
[18:53] derivagral: i'm not sure what you mean? this is from a checked out master on an ubuntu16 digitalocean box
[18:54] derivagral: i upgraded npm from 5.3 to 5.4 and am rerunning npm install atm, perhaps it'll just go away
[18:58] derivagral: looks like it works now, ty for the ping
[19:10] khinklenj: How did you upgrade to 5.4
[19:12] derivagral: v5.3 bugged me to run npm i -g npm, after which i removed node_modules & reran npm install

Using node 8.9.1 > still cant get it ti work :(
C:\zenbot>npm install
C:\zenbot>npm install
npm ERR! path C:\zenbot\node_modules\forex.analytics
npm ERR! code EISGIT
npm ERR! git C:\zenbot\node_modules\forex.analytics: Appears to be a git
repo or submodule.
npm ERR! git     C:\zenbot\node_modules\forex.analytics
npm ERR! git Refusing to remove it. Update manually,
npm ERR! git or move it out of the way first.
thanks for any insight.

@HeyIamJames
Remove the node_modules directory and try it.


SPI_demo not simulating.

1). Hi, your work on NEORV32 is amazing. I am trying to simulate the examples. Im getting hello world and hello cpp output via UART, but when i run _DEMO_SPI, the simulation is freezed and getting max runtime error.
2). so every example, sw/simple/testbench's output is displayed in the console VIA UART. So, actually where it is defined.
3). In the example files, can we call them as "c test cases" ?
4). So, we have a uart receiver in sim/simple. So, how to send value from the testbench, so that the uart receiver can receive?
THIS IS NOT FOR VERILOG VERSION, IM ASKING FOR VHDL VERSION. THANK YOU.

This is just a copy of https://github.com/stnolting/neorv32/issues/1080, right?

Yes sir. It can be closed. Sorry for posting twice.

No worries!


margin bottom isn't applied to the content of the admin page even though the class my-md-5 is used

can I help here with this issue? where this issue is (in the code?).


(refactor): Convert dataPlatforms to GMA aspect models and associated resource to GMA resource.

Convert dataPlatforms to GMA aspect models and associated resource to GMA resource.
BREAKING CHANGE: /datasets/dataPlatforms API is now changed to become GMA resource. The schema of the resource also changes.
What breaks: Get method of /dataset/dataPlatforms will return com.linkedin.dataPlatforms.DataPlatform schema. Earlier the API returns schema com.linkedin.dataplatform.DataPlatformInfo.
How to fix for existing readers: dataPlatformInfo is a field in then new schema returned by the API. This field should be the same as previous response of the API
Checklist

[x] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[x] Tests for the changes have been added/updated (if applicable)
[x] Docs related to the changes have been added/updated (if applicable)

You may want to add migration plan for this API? Do we plan to backfill these existing platforms and remove this json file? https://github.com/linkedin/datahub/blob/125ae288f1601bef2c42f44db6d8eb58b380a304/gms/impl/src/main/resources/DataPlatformInfo.json

You may want to add migration plan for this API? Do we plan to backfill these existing platforms and remove this json file? https://github.com/linkedin/datahub/blob/125ae288f1601bef2c42f44db6d8eb58b380a304/gms/impl/src/main/resources/DataPlatformInfo.json

You may want to add migration plan for this API? Do we plan to backfill these existing platforms and remove this json file? https://github.com/linkedin/datahub/blob/125ae288f1601bef2c42f44db6d8eb58b380a304/gms/impl/src/main/resources/DataPlatformInfo.json

Until there is a S&D use case for data platforms, I will like to keep using the json file. Easy to maintain and readable as there won't be too many platforms.

You may want to add migration plan for this API? Do we plan to backfill these existing platforms and remove this json file? https://github.com/linkedin/datahub/blob/125ae288f1601bef2c42f44db6d8eb58b380a304/gms/impl/src/main/resources/DataPlatformInfo.json

Until there is a S&D use case for data platforms, I will like to keep using the json file. Easy to maintain and readable as there won't be too many platforms.

You may want to add migration plan for this API? Do we plan to backfill these existing platforms and remove this json file? https://github.com/linkedin/datahub/blob/125ae288f1601bef2c42f44db6d8eb58b380a304/gms/impl/src/main/resources/DataPlatformInfo.json

Until there is a S&D use case for data platforms, I will like to keep using the json file. Easy to maintain and readable as there won't be too many platforms.

Thanks for the reply, Naga. I assume this PR is for this feature? https://github.com/linkedin/datahub/issues/2018
Could you please describe the ingestion plan (write) for dataPlatforms? It sounds like we will still use immutableDAO to write with the content from JSON file?

You may want to add migration plan for this API? Do we plan to backfill these existing platforms and remove this json file? https://github.com/linkedin/datahub/blob/125ae288f1601bef2c42f44db6d8eb58b380a304/gms/impl/src/main/resources/DataPlatformInfo.json

Until there is a S&D use case for data platforms, I will like to keep using the json file. Easy to maintain and readable as there won't be too many platforms.

Thanks for the reply, Naga. I assume this PR is for this feature? https://github.com/linkedin/datahub/issues/2018
Could you please describe the ingestion plan (write) for dataPlatforms? It sounds like we will still use immutableDAO to write with the content from JSON file?

You may want to add migration plan for this API? Do we plan to backfill these existing platforms and remove this json file? https://github.com/linkedin/datahub/blob/125ae288f1601bef2c42f44db6d8eb58b380a304/gms/impl/src/main/resources/DataPlatformInfo.json

Until there is a S&D use case for data platforms, I will like to keep using the json file. Easy to maintain and readable as there won't be too many platforms.

Thanks for the reply, Naga. I assume this PR is for this feature? #2018
Could you please describe the ingestion plan (write) for dataPlatforms? It sounds like we will still use immutableDAO to write with the content from JSON file?

This PR is actually step 1 for #1956.
That said, after this we can extend from this state of using ImmutableDAO to using EbeanDAO (#2018)


Upgrade: drop the potential manual patch upon rancher shell-image

Upgrade: drop the potential manual patch upon rancher shell-image to v0.1.26 on Harvester v1.3.2
IMPORTANT: Please do not create a Pull Request without creating an issue first.
Problem:

Harvester v1.4.0 will use Rancher v2.9 version, and the later uses a new shell image defaultShellVersion: rancher/shell:v0.2.1
https://github.com/rancher/rancher/blob/3f7c25803c00e5311d89d1144b3c6684b288ed51/pkg/settings/setting.go#L128C78-L128C97
https://github.com/rancher/rancher/blob/a9946d267616beb1be27c3142a4ebaf6186ba3e7/build.yaml#L4
From issue https://github.com/harvester/harvester/issues/6352, the shell-image was patched to v0.1.26 on Harvester v1.3.2 upgrade path, that may affect the further upgrade to v1.4.0.
Solution:

Upgrade: drop the potential manual patch upon rancher shell-image
Related Issue:
https://github.com/harvester/harvester/issues/6352
v1.3.2 installer PR https://github.com/harvester/harvester-installer/pull/813
v1.3.2 upgrade PR https://github.com/harvester/harvester/pull/6347
v1.3.2 issue https://github.com/harvester/harvester/issues/6283
Test plan:

v132->v140 upgrade, Rancher should upgrade smoothly, and the value of below CRD object should be empty after upgrade
setting.management.cattle.io/shell-image patched
NAME          VALUE
shell-image

@mergify backport v1.4

code is rebased with master-head

I'm unsure if there is any possibility that a user sets this to rancher/shell:v0.1.26 on purpose. If that's the case, it could be an unexpected behavior for them. Since the PR is targeted to the master branch, maybe we can add a version check for this, or just move the snippet to say package/upgrade/migrations/upgrade_manifests/v1.3.2/pre-hook.sh to limit the scope.

At the moment, we seemed not seeing the scenario for a user to set it. The only occurance is PR https://github.com/harvester/harvester/pull/6347.
After v1.4.0 is released (which can deal with potential v1.3.2, v1.3.3...), we can drop this code from master-head.


Gsdt - Fixup to Chisel connections and direction - PR 200 revisited.

In the following, optional checks are currently enabled by a compileOptions instance built from a String -> Value map with the following (Boolean) keys:
"connectFieldsMustMatch" - source and sink aggregates must have identical fields,
"regTypeMustBeUnbound" - Reg type argument must be unbound,
"autoIOWrap" - automatically wrap Module io's in an IO(),
"portDeterminesDirection" - if both LHS and RHS of a <> connection are bound with the same direction (I.e., both Input or both Output), if one of them is an IO port, use its bound direction to determine the connection orientation,
"tryConnectionsSwapped" - swap the connection orientation for the := operator if LHS is Input and RHS is Output,
"assumeLHSIsOutput" - if neither side is bound as an Input or Output for the <> operator, assume the LHS is Output,
"assumeNoDirectionIsInput" - if the source direction is unbound, assume it is Input.
By default, the compileOptions are set such that existing code compiles. It is anticipated these may be changed on a per-file basis and will be set by an appropriate import.
This pr should not be merged prior to ucb-bar/rocket-chip#220.
Below are the relevant comments from #200:
First set of changes:
Optionally require io be wrapped in IO, add Binding to track how/if Data is bound
Adds internal.Binding, which is a marker on Elements (subtype/leaves of Data), that tracks whether a Data has been placed onto the hardware graph as a register, port, literal, wire, operation result, etc.
This requires the io of a module actually be explicitly placed on the graph AS it is being created. (Since the module will want to connect to its own IO, module close is too late to do the binding.)
The binding check will also enforce that a bundle constructor does not actually create logic that is interpreted as part of the 'core bundle'.
Subsequent CLs will expand upon the Binding system to carry more data and execute more checks.
Now, frontend emissions of undefined declarations should be classed as bugs that should be caught by this system.
Second set of changes:
First, a listing of API changes:
For working with Chisel Data,
-- You can no longer supply a direction to UInt|SInt|Bool
-- You can no longer use .asInput|.asOutput|.flip. They have been marked as deprecated.
++ Instead, wrap in Input(), Output(), Flipped() call. These functions can only be called on unbound data (avoiding extraneous copies)
++ Reg and Wire now requires that outType/t variable be unbound (to avoid potential user errors).
On connection semantics and directionality,
++ Internal nodes like Reg and Wire that have no defined notion of input or output direction now lose any direction information when going from Unbound to
Bound. It may be enforced later that the types be explicitly undirectioned.
++ <> is now back to being a bidirectional connect mean to operate between
Module IOs only
++ <> requires Bundles on both sides to have exactly the same fields
++ := is now back to being a monodirectional connect that assumes all
subelements of the RHS are sources for corresponding subelement sinks on the
LHS.
++ := (optionally) requires the source Bundle to have at least the fields of the sink
Bundle.
In ChiselUtils,
++ DecoupledIO has enq, noenq, deq, nodeq, similar to old EnqIO, DeqIO methods
++ ValidIO type renamed to Valid, no longer forces Output
Second, some notes on implementation changes.
Direction is no longer stored on the Data but on the Binding attached to the data. The concept of Binding has been enriched to also introduce a 'Binder' function that a caller users to describe how they want a Data's nodes to be bound dependent on the specifics of the UnboundBinding. For example, PortBinder will create a Binding with the same direction information of the UnboundBinding.
The MonoConnect and BiConnect operators will only emit FIRRTL connections for Elements, never Aggregates. This is to avoid the issues with FIRRTL connection semantics.
Vec types now contain a sample_element that tracks all the binding changes to the actual elements. This sample_element is used when emitting the type of the Vec to FIRRTL and when creating a dynamically indexed port. This is needed to handle the case of an empty Vec having no elements but still requiring those descriptions. For consistency, this sample_element is used even when the Vec is not empty.
cloneTypeWidth behavior is somewhat weird. However, this is private and only used to create operators and should not be an immediate issue. This will eventually be removed (since cloneTypeWidth previously and currently will not create appropriate Muxes of Aggregates).
The use of DynamicVariable in Builder has been more carefully hidden from the rest of the internals by making dynamicContext private and adding accessors to the other global state.

Is there an example of how to use these options on a per file basis somewhere?

There currently isn't. There should be. I'll put one together this afternoon.

On Aug 24, 2016, at 12:22 PM, Colin Schmidt<EMAIL_ADDRESS>wrote:
Is there an example of how to use these options on a per file basis somewhere?

@sdtwigg what width-propagation assumptions are out of sync?  Chisel doesn't support most mixed UInt-SInt arithmetic, so those recent issues are irrelevant.

I don't know of all of them but, the one that I had in mind was:
Chisel3 assumes the result of the % operator is 'this.width'
FIRRTL (according to spec) assumes the result in the minimum of the two widths.
(Amusingly, the verilog spec seems to say the result is the maximum of the two widths, which leads to Verilator warnings.)
This is more of a case of, given that the presence of one mismatch means it is possible for there to be other unknown mismatches lurking around and thus it would be comforting if it was architected such that Chisel used the code in firrtl/PrimOps.scala to determine widths rather than re-implementing it for itself.

As a continuation of the above conversation, consider the following example which compiles and executes without error. T_2016 was chosen since it is highly likely this manifests with unnamed signals, like if you accidentally declare a wire instead of Port inside your submodule IO.
package sandbox

import chisel3._
import chisel3.util._
import chisel3.iotesters.ChiselFlatSpec
import chisel3.testers.BasicTester

class SandboxTest extends BasicTester {
  val (cycles, end) = Counter(Bool(true), 16)
  val submodule = Module(new SubModule)
  submodule.io.in := cycles

  // Create an internal wire, set it to cycles.
  val T_2016 = Wire(UInt())
  T_2016 := cycles

  // Separately, mistakingly try to set a submodules internal wire
  submodule.T_2016 := 1.U

  // Observe that you ended up setting your own internal wire instead!
  printf("%d: T_2016=%d sm.io.out=%d \n", cycles, T_2016, submodule.io.out)

  when(end) { stop() }
}

class SubModule extends Module {
  val io = new Bundle {
    val in  = UInt(width=4).asInput
    val out = UInt(width=4).asOutput
  }

  val T_2016 = Wire(UInt())
  T_2016 := io.in
  io.out := T_2016
}

class SandboxTester extends ChiselFlatSpec {
  "Sandbox" should s"work." in {
    assertTesterPasses(new SandboxTest)
  }
}

What should the default CompileOptions be?

Currently blocking things:

Design doc so we an more fully understand the code and merge with confidence
Give a default CompileOption, either Strict or "you forgot a default" Default, specified using the materialize pattern (which has lower precedence in implicit resolution)
Remove DataMirror, which should be re-submitted later as a more complete PR

Future items (tracking bugs to be opened):

Standardize the One True Way to construct literals (UInt / etc)
Remove default CompileOption from chisel3 core, instead inheriting them from user-level implicit resolution. This is a longer term fix needs to play nicely with source info macros and name propagation macros

Issue that arose when discussion with someone on the particulars of how this PR differed from the original PR:
If you are trying to keep the aggregate-level directions (unlike the original PR which propagated directions to leaf elements and used only those directions for emission) then you need to amend chiselCloneType to also do a deep copy the aggregate-level directions. It looks like the current copy is only on the root Data.
Also, it would be more in-style to use firrtlDirection and firrtlDirect_= as setter and getter rather than set* and get*

@sdtwigg that may well be the cause of our remaining tilelink2 problems...

Good points @sdtwigg. Initial firrtl failures were fixed by maintaining the original top-level direction separately from the propagated applied direction, and not much thought was given to what this meant in the context of cloning.
It would be appreciated if someone fixes this before @ucbjrl attempts to do so.

@ucbjrl unfortunately, it's crunch time over here, so I can't lend a hand for a few days... but doesn't chisel3/master already do something akin to this?


esp32s2: Add missing ESP32S2 SPIRAM config

Summary
Add missing ESP32S2 SPIRAM config
Impact
Users will be able to select the SPIRAM support
Testing
CI

Let's ignore the temp ci broken and merge this simple patch directly

Thank you @xiaoxiang781216 ! Any idea why our CI is failing too often recently?

tool change upgrade issue, here is the fix: https://github.com/apache/incubator-nuttx/pull/7294


Unclear 'namespace' for this plug in

Hello,
We have a large industrial C# repo (~5MLoC, 125 .sln files, 600 .csproj files).
We would like to speed up the build of this repo significantly, by making it fully incremental.
We are investigating whether Gradle would be a good solution for that.
We don't have prior knowledge of Gradle, so I may be asking the obvious.
We have a sample repo with a few solutions, each with a separate build.gradle file, containing the dotnet section.
What we would like to do is to centralize the dotnet section in a conventions file in the buildSrc directory.
Similar to this Gradle user guide.
So we created:
buildSrc\build.gradle
plugins {
    id 'groovy-gradle-plugin'
}

repositories {
    gradlePluginPortal() // so that external plugins can be resolved in dependencies section
}

dependencies {
    implementation 'com.github.Itiviti.dotnet:com.itiviti.dotnet:1.9.5'
}

and we created a file buildSrc\src\main\groovy\myproject.dotnet-conventions.gradle.
When we try to use this with gradle build, it says:
> Could not resolve all files for configuration ':buildSrc:compileClasspath'.
   > Could not find com.github.Itiviti.dotnet:com.itiviti.dotnet:1.9.5.
     Searched in the following locations:
       - https://plugins.gradle.org/m2/com/github/Itiviti/dotnet/com.itiviti.dotnet/1.9.5/com.itiviti.dotnet-1.9.5.pom
     If the artifact you are trying to retrieve can be found in the repository but without metadata in 'Maven POM' format, you need to adjust the 'metadataSources { ... }' of the repository declaration.
     Required by:
         project :buildSrc

My assumption is that the prefix 'com.github.Itiviti.dotnet:com' is not correct, as I have just made something up.
Now my question: what prefix do I have to use here?
If I am totally off, please let me know too.

honestly I didn't test if incremental works correctly
for using this plugin, you will need to have one gradle project per .sln and making them as gradle subprojects, then hopefully the plugin setup the Input source correctly and skip the task execution
but first you should go for simple, that builds a few sln and try
for your current issue, remove
    implementation 'com.github.Itiviti.dotnet:com.itiviti.dotnet:1.9.5'
}```
and put the following inside the plugins block, as mentioned in the README

id 'com.itiviti.dotnet'```

Hey. I am currently trying to get gradle caching to work so that if you revert something you had already compiled it doesn't recompile again but unfortunately it doesn't seem to work since it always compiles the project.
I can confirm that the task is not executed if the sources/destinations haven't changed but I can't get caching between runs to work. Currently in the DotnetBuild task I am just declaring sources and the destinations directory nothing more. Is this supported, and if yes, am I missing something?


Adding Expert Prototyping to FastMoE

Hi, thanks for your provding end-to-end training framework in Pytorch for MoE models. We have recently implemented MoE in tensorflow and found out that categorizing experts to different groups can bring improvements in model quality. More details can be referred to our paper https://arxiv.org/abs/2105.15082. I wonder if it is possible to add this feature as FastMoE really facilitates research in sparse expert models.
Generally, this strategy categorizes experts to different groups, each of which has its own gating function for routing. It is compatible with the conventional routing method like Switch or top-2 routing as you can set the group number to 1. We find that increasing the value of k in top-k can increase model performance and k top-1 can achieve similar effect. Also, it is possible to try out more complex strategies, say k top-k' or so.
We have a code snippet in the appendix, which may be helpful.

Here is another recent work about MoE.
https://arxiv.org/abs/2106.03760
The idea is to activate all experts at the beginning of training, but quickly converge to sparse activation. I wonder whether such mechanism can help train better pre-trained models when our expert pool is not that large.
Let me know how do you think about it?


🍒 Cherry pick PR #13707 to staging 🍒

🍒 Cherry pick https://github.com/Expensify/App/pull/13707 to staging 🍒

🚀 Cherry-picked to staging by @OSBotify in version: 1.2.41-3 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@Expensify/applauseleads please QA this PR and check it off on the deploy checklist if it passes.

🚀 Deployed to production by @yuwenmemon in version: 1.2.41-4 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Cherry-picked to staging by https://github.com/AndrewGable in version: 1.3.28-2 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@Expensify/applauseleads please QA this PR and check it off on the deploy checklist if it passes.

🚀 Deployed to production by https://github.com/AndrewGable in version: 1.3.28-5 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Consider using Coding Style and .editorconfig

Hello!
Please, consider using Coding Style and .editorconfig. Because it’s unusual to see, for example, the following code
https://github.com/microsoft/qsharp-compiler/blob/9bdc21409409b8e74cde1e0a24c49fc80bb602f3/src/QsCompiler/Compiler/CompilationLoader.cs#L408-L409

Thanks for the suggestion! We've discussed standardizing the coding style recently. An .editorconfig file sounds like a good idea. It looks like a lot of editors have support for it, including Visual Studio and Visual Studio Code.
We're also interested in tools to automatically check code style, for example as part of the automated builds for PRs. It looks like dotnet-format can do this, but we're open to other suggestions.

In C# extension in VS Code, editorconfig support is an opt in flag for OmniSharp (https://www.strathweb.com/2019/07/editorconfig-support-in-omnisharp-and-c-extension-vs-code/), we hide most of these newer features behind an opt-in flag on order to keep the base featureset is as lightweight as possible. but yes it definitely makes sense to me too!
For enforcing code style at build time you can also have a look at Microsoft.CodeAnalysis.CSharp.CodeStyle prerelease package (only exists on Roslyn CI feed). It ships support for whitespace (IDE0055) violations (same as those covered by dotnet-format) in a regular analyzer package and reads the settings from editorconfig file.

We actually have an editorconfig for the samples repo for precisely this reason. We've looked into using dotnet-format with precommit before to help with code style, but to limited success.

I'm starting to work on an .editorconfig file in the marshallsa/editorconfig branch. Some mixed results so far.
First, it seems like the only way to reformat @rstm-sf's foreach example is to set csharp_preserve_single_line_blocks = false. However, this also reformats auto-properties that are defined on one line:
public int Foo { get; set; }

becomes
public int Foo
{
    get; set;
}

I think this is unnecessary, but I can't find a way to apply the reformatting to foreach without also applying it to auto-properties. So it's not ideal, but maybe worth it?
Second, it seems like Visual Studio does not support reformatting F# files at all. So unfortunately we might have to manually enforce an F# coding style. At least it could still be a good idea to write out some of the rules explicitly so we can refer back to them (probably based on the F# style guide with a few modifications), unless someone knows another way to auto-format F#.

It seems to me that the following option looks better csharp_preserve_single_line_blocks = false :)
It seems that I was mistaken in that I looked at the settings of editorconfig from far: for such a case it is difficult to choose an option. Maybe because it is not popular? Before that, I saw two equivalent options:
for ()
    smth

and
for ()
{
    smth
}

But this was the first time I met such an option and it seems to me that it is overloaded: trying to fit everything on one line only complicates further support

Second, it seems like Visual Studio does not support reformatting F# files at all. So unfortunately we might have to manually enforce an F# coding style. At least it could still be a good idea to write out some of the rules explicitly so we can refer back to them (probably based on the F# style guide with a few modifications), unless someone knows another way to auto-format F#.

There is a Fantomas project. It also has an issue supporting editorconfig  fsprojects/fantomas#650

We reformatted all of the C# code using StyleCop, so the code sample mentioned in the original post has been fixed. :)
The F# code still has no formatting rules currently, though, so I'll leave this issue open until we do. Thanks @rstm-sf for pointing out Fantomas!


Can't access predictions

System information

OS Platform and Distribution: Windows 10
TensorFlow installed from: binary
TensorFlow version: Tensorflow 1.4.0
Python version: Python 3.6
CUDA/cuDNN version: CUDA 8 / cuDNN 6
GPU model and memory: GTX M950

Describe the problem
When trying to print the predictions from DNNClassifier class i only get "<generator object Estimator.predict at 0x000001AFE1E24EB8>". I used the exact code written in the Tensorflow tutorials.
Source code / logs
train_input_fn = tf.estimator.inputs.numpy_input_fn(x={"x": np.array(X_train)}, y=y_train,num_epochs=None,shuffle=False)
test_input_fn = tf.estimator.inputs.numpy_input_fn(x={"x": np.array(X_test)}, y=None,shuffle=False)
classifier.train(input_fn=train_input_fn, steps=100)
preds = classifier.predict(input_fn=test_input_fn)
print(preds)

preds is indeed a generator object that cannot be printed on its own. The Estimator Tutorial demonstrates this:
predictions = list(classifier.predict(input_fn=predict_input_fn))
predicted_classes = [p["classes"] for p in predictions]

print(
    "New Samples, Class Predictions:    {}\n"
    .format(predicted_classes))

If you have more issues like this, please post a question on Stack Overflow, which is more suited to individual issues that are not bugs or feature requests. Thanks!

I have tried as the tutorial. But it emitted error
WARNING:tensorflow:Input graph does not contain a QueueRunner. That means predict yields forever. This is probably a mistake. error: None values not supported.
so, what's the problem..

rude

Can we pass batched features from dataset iteration into GBT estimator for prediction instead input_fn?


update() with returning("*") is returning 1

Environment
Knex version:
"knex": "^0.21.0",
Database + version:
"oracledb": "^4.2.0",
OS:
Oracle Linux
NodeJS:
v12.16.3
Issue is about oracledb support, tag @ atiertant.
Bug
update() with returning("*") is returning 1

When use object bellow with "*":

user = await connection('users').where('user_id', req.params.user_id).update(req.body).returning("")
or
user = await connection('users').where('user_id', req.params.user_id).update(req.body).returning([""])
or
user = await connection('users').where('user_id', req.params.user_id).update(req.body, ["*"])
Results:
{
"user": 1
}

When the columns are specificied works well:

user = await connection('users').where('user_id', req.params.user_id).update(req.body,['user_id','user_name']);
or
user = await connection('users').where('user_id', req.params.user_id).update(req.body).returning(['user_id','user_name'])
results:
{
"user": [
{
"user_id": "1",
"user_name": "Wellysson Godinho"
}
]
}

When use "*" shoud be returning all collumns

I would like know why work well with insert function and not work with update function.

Here is runnable reproduction: https://runkit.com/embed/658sj80v3lox
Looks like oracle has special treatment for returning('*')

Not working accordingly yet. I need that returning in JSON format.

See the result now (here I put all colluns):

user = await connection('users').where('user_id', req.params.user_id).update(req.body).returning(['*']).toSQL().sql;
Result:
{
"user": "update "users" set "user_avatar" = ?, "user_country" = ?, "user_country_cca2" = ?, "user_country_calling_code" = ?, "user_region" = ?, "user_region_cca2" = ?, "user_phone" = ?, "user_bio" = ?, "user_status" = ?, "user_verification_code" = ? where "user_id" = ? returning "ROWID" into ?"
}

When use the same example with insert function, the result the field work well:

const user = await connection('users').insert(req.body).returning('*');
Result:
{
"user": [
{
"user_id": "1",
"user_name": "Wellysson Godinho"
....
}
]
}
I would like know why work well with insert function and not work with update function.

Thanks for helps, but not working accordingly yet.
I need that returning in JSON format.
See the result now using special treatment for oracle:
user = await connection('users').where('user_id', req.params.user_id).update(req.body).returning(['*']).toSQL().sql;
Result:
{
"user": "update "users" set "user_avatar" = ?, "user_country" = ?, "user_country_cca2" = ?, "user_country_calling_code" = ?, "user_region" = ?, "user_region_cca2" = ?, "user_phone" = ?, "user_bio" = ?, "user_status" = ?, "user_verification_code" = ? where "user_id" = ? returning "ROWID" into ?"
}
When use returning ('') with insert function, the result work well:
const user = await connection('users').insert(req.body).returning('');
Result:
{
"user": [
{
"user_id": "1",
"user_name": "Wellysson Godinho"
....
}
]
}
It should was obtain the same result of the insert function exhib for update function with returning('*').

Yes... It is not working. That reproduction I added is just for other to easily see what is the problem if they want to fix it.
I have no idea why it seems to work for insert...

Released in 2.5.0


Interactive examples for aeon

Hi aeon team!
My team maintains a few open-source projects, and we've seen that having interactive examples greatly helps newcomers! We're maintaining a Binder cluster that provides ephemeral JupyterLab sessions (example). It essentially spins up a JupyterLab session with pre-made examples.
We're happy to sponsor aeon with some computing power. If you're interested, please get in touch with me! (my contact info is in my profile)

hi, that sounds interesting, we will take a look and get back to you

Sounds great! In the meantime, I enabled aeon in our binder cluster:
https://binder.ploomber.io/v2/gh/aeon-toolkit/aeon/main?filepath=examples
This cluster is managed by my team so it doesn't have the capacity issues as the public one.
Happy to open a PR to switch the binder link on the README!

Hi, I have been away for a bit so did not see this.
Would this just be as simple as swapping out the links in the README and conf.py etc. to your link? Happy to use a less congested server for our examples.

Hi @MatthewMiddlehurst. Yes, the change implies swapping the links. Unfortunately, we have already onboarded a few open-source projects and have no capacity. We might open new slots soon; I’ll contact you if that happens!


Add/remove member from Doing/Completed CampaignRun Mobile Commons Groups

Refs https://github.com/DoSomething/gambit/pull/672#issuecomment-255138491
When a new Campaign is cached, create 2 new Mobile Commons Groups if they don't exist already with names

campaign_[id]_run_[id]_doing -- stores Users who haven't completed the Campaign yet
campaign_[id]_run_[id]_completed -- stores Users who have completed the Campaign

We'll use these Mobile Commons Groups to segment our Mobile Commons broadcast messaging.
When a Gambit Signup document is created, add the User to the signedup Mobile Commons Group.
When a Gambit Signup is updated (which could happen from a POST #642) and the User has reported back, remove them from the signedup MC Group and add them to the completed Group.

We talked through how to implement today, and Quicksilver will handle adding/removing members to the relevant Doing/Completed groups. If Gambit handled this, Quicksilver would need to POST Reportback activity anyway (in order to move any Doing's into Completed if they reported back via web).
I was hoping we'd be able to use the Group machine name for adding/removing, but we need to pass the numeric Mobile Commons Group ID when adding or removing.
One idea here is that when we add a new Campaign to Gambit's list of active campaigns (CAMPAIGNBOT_CAMPAIGNS) -- Gambit creates the groups if they don't already exist and could expose the ID's on a new GET Gambit campaigns endpoint exposing our campaign documents.
Another idea could be for Quicksilver to GET groups and parse the list looking for the correct group by its name (campaign_id_run_id_doing) results... but I think a GET Gambit campaigns/:id to find the correct Mobile Commons Groups sounds cleaner.
cc @sergii-tkachenko

This branch starts on creating the Mobile Commons Groups per DS Campaign upon app launch.  I could see a use case for also exposing the Campaign keywords and msg overrides defined in the Gambit document in a Gambit campaigns API as well.
Another idea that I'm not crazy about is just manually creating the Groups in Mobile Commons and storing their ID's in the Phoenix Third-party Opt-in form (replacing the Mobile commons OIP field with 2 fields to store Doing/Completed groups...but it'd be a lot nicer to automate the DS Campaign / Group ID associations to avoid potential epic subscription fails when entering the wrong Group ID.
cc @deadlybutter

Chatted about this issue with @sergii-tkachenko today -Quicksilver will retrieve the Group ID's it needs from new Campaigns endpoint #677.  #674 added auto-creating of the groups, so closing this to continue working on exposing group id's in #677.

This issue could use some cleanup:

[ ] Delete mobilecommons_group and mobile_commons_group properties from DB, refactor as mobilecommons_group_doing and mobilecommons_group_completed properties per #681

[ ] Only log the mobilecommons.createGroup message if we're creating/storing new groups. If the campaign document doesn't have a group stored and we get the Mobile Commons error of invalid name.. worst case is we could GET https://secure.mcommons.com/api/groups and search all results for the name...

[ ] Would make sense to add a pull staging DB script (https://github.com/DoSomething/gambit/blob/2.1.0/pull_config.sh) to have Thor/Prod apps responsible for the Group generation (vs localhost)

[ ] Delete test Mobile Commons Groups

@aaronschachter in most of our API responses we group similar fields together, why are we creating lots of single independent fields instead?
Also I would probably throw in a veto for fetching all groups. we have over 10 thousand if I remember right, and none of them are actually deletable. I can't see the mobile Commons docs on my phone but when I get in I'll try & think through some ideas with you.
Also I don't know if I like the idea of mixing Thor/local groups, but we might not have much of a choice


Add login endpoint

This PR introduces a SQL Injection and a Path Traversal in new login, logout and attachments endpoints, which Mayhem for API detects, preventing the PR from being merged.

Codecov Report

Merging #2 (6d79230) into main (a420fd2) will increase coverage by 4.81%.
The diff coverage is 88.23%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main       #2      +/-   ##
==========================================
+ Coverage   89.47%   94.28%   +4.81%
==========================================
  Files           1        1
  Lines          19       35      +16
==========================================
+ Hits           17       33      +16
  Misses          2        2

Flag
Coverage Δ

vulnerability-tests
94.28% <88.23%> (+4.81%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

src/main.py
94.28% <88.23%> (+4.81%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Update pc-saas-fulfillment-api-v2.md

changed author to v-qiwe since prior author is no longer at Microsoft

@BrianLevenson : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@qianw211 - approve and/or sign off when you're okay with this PR. PRMerger may give you an error comment, but you can tag me and I'll merge it. thanks.

#sign-off

@qianw211: I'm sorry - only the author of this article, @v-qiwe, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.

@Jak-MS please go ahead and merge.  Thanks!


How to embed cmd input box in another program?

I want to use cmd inputbox in my own program, and I've tried many ways but none of them successed ( Include Create The conhost window as a child window, SetParent Function, and so on )
The might be many people want to use this powerful inputbox as well, And I wonder what should I do to make that.
Thanks !!!

For early 2019, you'll want to look at the pseudoconsole API -- a good starting point is here on MSDN, Creating a Pseudoconsole Session.
We're planning to make the TerminalControl and DirectWrite renderer/terminal buffer/VT parser components of Windows Terminal reusable and hostable for other applications. There's some work there going on in #1274 and #1263.

For early 2019, you'll want to look at the pseudoconsole API -- a good starting point is here on MSDN, Creating a Pseudoconsole Session.
We're planning to make the TerminalControl and DirectWrite renderer/terminal buffer/VT parser components of Windows Terminal reusable and hostable for other applications. There's some work there going on in #1274 and #1263.

Thanks!
Is there any difference between creating a psedoconsole session and creating a new cmd exe process with hStdInput/hStdOutput set in STARTUPINFO ?

@yh-git Yes there is. When a commandline application is started with redirected input and output, it can't make use of the large majority of the Console API. For example, a user wouldn't be able to use the arrow keys to navigate the input line if cmd.exe was just created redirected. The Pseudoconosle API, however, does attach the commandline application to a full console, so the application can use the full console API. The pseudoconsole will then do the hard work of translating those API calls into a stream of characters, with VT sequences embedded in the output. You'll need to be able to handle a few types of VT sequences to be able to properly display pseudoconsole output.

I'm gonna close this one as it's been answered. Thanks!


Feature/unit tests

Add more unit tests.

Coverage increased (+1.9%) to 9.765% when pulling d86cccf5d3c51ab2c8da980cfc5b2a34c93feaa6 on feature/unit_tests into ba6529bec994a035856a2d8af4fa542f7820ea72 on master.

Coverage increased (+1.9%) to 9.765% when pulling c5e24b324e221745237b1b9e2de9d09b94cf4c8e on feature/unit_tests into ba6529bec994a035856a2d8af4fa542f7820ea72 on master.

👍

👍

Coverage increased (+1.9%) to 9.581% when pulling 296dbff87360906584e3d3f6b2a2b56ec677ccd4 on feature/unit_tests into 4f36e447c3a45ed1951a52836b4658f5429c8cff on master.


import linux_distribution from distro

As it turns out, python3.8+ no longer ships with the linux_distribution function in the platform module (see https://bugs.python.org/issue28167). The docs recommend instead to use the homonym function from the distro package. This pull request

installs the distro package
imports linux_distribution from distro
upgrades the package version to 1.0.1

I submitted #8 as an alternative solution to the same problem.

i am creating new module with syncronous asyncronous and cli named PyIpify


Question about the variance matrix

Hello.
Thank you for sharing your great code!
I have a question about the variance matrix V.
When I first read the paper, I thought that V was calculated per image pair.
However, in the code, it turns out that V was calculated per layer.
Also, V was not calculated during the training stage, but in the validation stage.
Is there a reason why you design the calculation of V like in the code?
Thank you.

Hi, Thank you for your interest in our paper and code!
As stated in the paper, it is correct to calculate the variance of the covariance between the photometric transformed image and the original image. Those two images are concatenated as a batch dimension (dim=0) and the variance is obtained via torch.var().
https://github.com/shachoi/RobustNet/blob/56cb41baa286082dfa7f534ea56750bc091e1842/train.py#L505
https://github.com/shachoi/RobustNet/blob/56cb41baa286082dfa7f534ea56750bc091e1842/network/deepv3.py#L480-L481
https://github.com/shachoi/RobustNet/blob/56cb41baa286082dfa7f534ea56750bc091e1842/network/deepv3.py#L543
The following codes with "validate_for_cov_stat" for calculating the variance matrix are included in the training phase.

https://github.com/shachoi/RobustNet/blob/56cb41baa286082dfa7f534ea56750bc091e1842/train.py#L239-L269
If you have further questions, please feel free to ask.
Thank you!


Dependency issue

Does it really require graphql 0.4.x?  And not work with 0.5.x?

It should work. Just updated package dependencies with https://github.com/kadirahq/graphql-errors/pull/4

Deployed v2.1.0 with the fix


[Discussion] Train a model for narration

Hi,
your package is fantastic, but I would like to use it in use-case that has to do with narration.
As so, in order for it to work better I would like to train a model between two big chunks of music and narration and eventually separate my target file.
As so I have three questions:

Can you provide instructions of how to train a model that are for custom-datasets?
How much time of audio samples is it needed, to have a good model?
Is there a tool that creates a synthetic dataset from two pure chunks of sound?

https://github.com/deezer/spleeter/wiki/2.-Getting-started#train-model
But opus already has algorithms to discern between speech & music ,  https://people.xiph.org/~xiphmont/demo/opus/demo3.shtml . Or you can use other metrics such as M/S comparison. But I think the default spleeter models do fine on separating music/narration as long as the music has no voice.


CDO classes

Some adjustments are needed for CDO classes, in order to coherently extend the ontology. Specifically:

The Grain Boundary class is not needed on this level, I propose just one class PlaneDefect, the Grain Boundary class can be extended later on.
Include another class Bulk Defect
Also, LineDefect could be included here as superclass of dislocation? Considering you want to include a disclination class, if not, I guess having LineDefect as equivalent to Dislocation works.

Picture for reference:

Let me know what you think!

Hi @aazocar, thanks for the review.
That makes sense now and I'll try to incorporate within the next few days.

@aazocar would you fix it by yourself making a pull request? or let me fix it?

Hi @az-ihsan, thanks for checking. I think you can fix it as you see fit. Since you have a better idea how the changes should be made :)


Updated README.md

Changes Proposed
I tried to improve the grammar of README.md file

Check List

[x] The title of my pull request is a short description of the changes
[ ] This PR relates to some issue:
[x] I have documented the changes made (if applicable)
[ ] I have covered the changes with unit tests

Thanks!


chore(deps): update dependency kubernetes/kubernetes to v1.30.0

This PR contains the following updates:

Package
Update
Change

kubernetes/kubernetes
minor
1.29.4 -> 1.30.0

Release Notes

kubernetes/kubernetes (kubernetes/kubernetes)
v1.30.0: Kubernetes v1.30.0
Compare Source
See<EMAIL_ADDRESS>Additional binary downloads are linked in the CHANGELOG.
See the CHANGELOG for more details.

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

Edited/Blocked Notification
Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.
You can manually request rebase by checking the rebase/retry box above.
⚠ Warning: custom changes will be lost.


Partial endpoint remediation

Add the ability to partially remediate endpoints. Before, an endpoint remediated on one finding would be remediated for every single finding the endpoint was attached to. Now, each endpoint now has its own status and activity logging so that each endpoint on a given finding can be manipulated, but not carried over to other vulnerabilities. An endpoint status is created when an endpoint is attached to a finding in all avenues (API, UI, Import, re-import).
Motivated by #1975

[x] Your code is flake8 compliant.
[x] Your code is python 3.5 compliant (specific python >=3.6 syntax is currently not accepted).
[] If this is a new feature and not a bug fix, you've included the proper documentation in the ReadTheDocs documentation folder. https://github.com/DefectDojo/Documentation/tree/master/docs or provide feature documentation in the PR.
[x] Model changes must include the necessary migrations in the dojo/db_migrations folder.
[x] Add applicable tests to the unit tests.
[x] Add the proper label to categorize your PR.

Nice work, good to see some improvements towards better endpoint management. I would like to also make the dedupe a little better around them if anybody ever has a need for that.
Some things popping up in my head while looking at the code:

should we stick to 'mitigated' instead of 'remediated', same as for findings? Or maybe change mitigated status for findings to remediated if that sounds better? :-)
did you consider using a m2m relationship with a "through" table? I think that's how Django ORM envisions these m2m relationships with extra properties
should we allow authorized users in the UI to change the status of endpoints now that we have our quickwin authorized user permission pr merged? (2615)
i see a bit repetitive html snippets around display a table of endpoints + statuses. are those suited to be put into a "snippet" html template that is included?
did you consider any impact on dedupe? it looks like there is none?

I will update the marked boxes:

[x] Mitigation probably makes a bit more sense from a definition perspective. Easy fix there.
[ ] Have not heard of the the "through" attribute. Looks to accomplish what I did but in a much cleaner and simpler way. May be a lot of work converting due to templatetag/html.
[x] Yes, these should be permissioned the same way bulk findings are. Will update.
[x] It is a bit repetitive and not very maintenance friendly. A snippet would be ideal. Great suggestion!
[ ] No impact as it was intentionally avoided. Considered the impact of dedupe, adjusting to the "new way of doing things", and converting from old system to the new. Surely there are angles I was not considering, so I decided to wait for the mod meeting for discussion.

Another thing I thought of:

When you run re_import or normal import with close_old_findings=True, DD will close (mitigated) any findings that are no longer present in the scan report being imported. Should something similar happen for endpoints now?


Package request: FFMPEG - RIST support

Project description
The NixOS package FFMPEG currently has SRT, but not RIST.  It would be really helpful if RIST was included.
https://www.rist.tv/articles-and-deep-dives/2024/3/7/2024-rist-vs-srt-comparison
Rist library is here
https://code.videolan.org/rist/librist
It's great to see SRT is already included!
https://github.com/NixOS/nixpkgs/blob/74411387f2c763a5c490025ae0595ff4d47e4b03/pkgs/development/libraries/ffmpeg/generic.nix#L111

Metadata

homepage URL: https://code.videolan.org/rist/librist
source URL: https://code.videolan.org/rist/librist
license: BSD2 ( https://code.videolan.org/rist/librist/-/blob/master/COPYING )
platforms: unix, linux, darwin,

Add a :+1: reaction to issues you find important.

Added pull request
https://github.com/NixOS/nixpkgs/pull/353473


unable to add plugin

I am trying to add this plugin via ionic CLI using ionic cordova plugin add https://github.com/DavidBriglio/cordova-plugin-local-notifications but unable to add and getting this error

any idea ? how to fix this

Try using https://github.com/DavidBriglio/cordova-plugin-local-notifications.git instead and see if this works.

Let me try

Are you using the newest version of git and ionic?

Not newest but ionic cli verion is 3.19.1 and cordova 8.0

Okay sure will do that and let u know..thanks for your time man


Cannot use 'in' operator to search for 'default' in undefined     at material-ui.production.min.js:6

[x] This is a v1.x issue (v0.x is no longer maintained).
[ ] I have searched the issues of this repository and believe that this is not a duplicate.

i found below code in https://unpkg.com/<EMAIL_ADDRESS>"use strict";var W="default"in u?u.default:u;

What's the browser? Do you have a reproduction example?

Your issue has been closed because it does not conform to our issue requirements.
Please provide a full reproduction test case. This would help a lot 👷 .
A live example would be perfect. This codesandbox.io template may be a good starting point. Thank you!

If you use the development version you'll see where exactly it fails.
You may have included mui before react/react-dom.


Add more infos to rawREST event

Can we pass more information in the rawREST event?
The information are already available and are passed as debug message a few lines later.
I would like to have the following information in the event to be able to create better stats on a dashboard:

request.route
res.status

https://github.com/Snazzah/slash-create/blob/acfa883e96458222d355a2c2099b0d706a9e8f64/src/rest/sequentialBucket.ts#L157-L166

You could still get the status code from response.status (which should've been typed anyways), as for request route, I could just give you the internal request and you can have that and more from it.

You're right, I completely overlooked the response object.. 😅 and the internal request would be nice to have

Added in v6.1.2.


Remove promise jsdoc

Removes @return {!Promise} for functions already marked as async.

Should we keep the types for future closure type checking?

nope, it is implied by async.


Display empty result instead of not displaying anything

Fixes #33

Renames setMatchQueryOutput to append for OutputSinks for consistency
Added a newline between the output and the query time
Doesn't display MATCHED on the results of each tuple in the result of a OneTimeMatchQuery

Coverage increased (+0.005%) to 60.376% when pulling aee529bcbb9e79397e3262ef854fbd6f14df93c7 on nare469:display-empty into d9656c7eb448436c2cfdaf7640d38dcd8197c0ce on graphflow:master.

Coverage increased (+0.005%) to 60.941% when pulling 405c4bcb27f8bd71e83717fc7652f30d7b45ef1c on nare469:display-empty into 80f93317180c4377cd0047c338e05110a5afbf4f on graphflow:master.


Origins Crash Report

Im attempting to use origins alongside multiple other mods, but it continues to crash stating
"The game crashed whilst initializing game
Error: java.lang.RuntimeException: Could not execute entrypoint stage 'main' due to errors, provided by 'origins'!
Exit Code: -1"
I attempted taking out possible mods from my folder to fix this alongside origins itself, but nothing so far has worked.

A screenshot of the crash report I keep on receiving.

You should provide more information than that, in particular the crash report file (access it via "View crash report") would help a lot in figuring out what is causing the issue!


Document way to recover / reset OIDC and SAML auth providers from the CLI

If the OIDC config goes awry, the server will fail to start, preventing admins from fixing it. Document a command to reset this from the CLI.

Fixed in #1026


How to set/unset "Reset iOS Summary Rating" option?

Question Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Question Subject
Fastlane, App Store submission "Reset iOS Summary Rating" property setter?
Question Description
There is an ability in Spaceship to set Reset iOS Summary Rating property on a new App Store build with the following Feature Request: Reset iOS Summary Rating. After search in the docs and source code, I didn't find any way to set that particular option via Fastlane configuration files. Does one exist?

@fastlane-bot no idea why that's relevant, but here it goes
✅ fastlane environment ✅
Stack

Key
Value

OS
10.14.3

Ruby
2.3.7

Bundler?
true

Git
git version 2.20.1

Installation Source
~/.gem/bin/fastlane

Host
Mac OS X 10.14.3 (18D109)

Ruby Lib Dir
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib

OpenSSL Version
LibreSSL 2.6.4

Is contained
false

Is homebrew
false

Is installed via Fabric.app
false

Xcode Path
/Applications/Xcode-10.1.app/Contents/Developer/

Xcode Version
10.1

System Locale

Variable
Value

LANG
en_US.UTF-8
✅

LC_ALL
en_US.UTF-8
✅

LANGUAGE

fastlane files:
`./fastlane/Fastfile.swift`
// Customize this file, documentation can be found here:
// https://docs.fastlane.tools/
// All available actions: https://docs.fastlane.tools/actions
// can also be listed using the `fastlane actions` command

import Foundation

class Fastfile: LaneFile {
    // This is the minimum version number required.
    // Update this, if you use features of a newer version
    var fastlaneVersion: String { return "2.108.0" }

    func beforeAll() {
        // environmentVariables["SLACK_URL"] = "https://hooks.slack.com/services/..."
        // cocoapods()
        // carthage()
        // updateFastlane()
    }

    func deliverLane() {
        desc("Uploads metadata")
        deliver()
    }

    func testLane() {
        desc("Runs all the tests")
        runTests()
    }

    func betaLane(withOptions options:[String: String]?) {
        desc("Submit a new Beta Build to Apple TestFlight. This will also make sure the profile is up to date")

        ensureGitStatusClean()

        let changelog = prompt(text: "Changelog: ", multiLineEndKeyword: "END")

        buildApp(scheme: "App",
                 clean: true,
                 silent: true,
                 includeSymbols: true,
                 includeBitcode: true,
                 exportMethod: "app-store")

        uploadToTestflight(username: appleID,
                           changelog: changelog,
                           waitForUploadedBuild: true)
    }

    func releaseLane() {
        desc("Deploy a new version to the App Store")

        ensureGitStatusClean()

        buildApp(scheme: "App",
                 clean: true,
                 silent: true,
                 includeSymbols: true,
                 includeBitcode: true,
                 exportMethod: "app-store")

        //        captureScreenshots()
        //        frameScreenshots()

        precheck(appIdentifier: appIdentifier, username: appleID, teamId: teamID, includeInAppPurchases: false)

        uploadToAppStore(username: appleID, metadataPath: "Fastlane/metadata", screenshotsPath: "Fastlane/screenshots", force: true, app: appIdentifier)
    }

    // You can define as many lanes as you want

    func afterAll(currentLane: String) {
        notification(message: "\(currentLane): Successfully deployed.")
        // This block is called, only if the executed lane was successful
        // slack(
        //     message: "Successfully deployed new App Update.",
        //     slackUrl: "slackURL"
        // )
    }

    func onError(currentLane: String, errorInfo: String) {
        notification(message: "\(currentLane): Failed with \(errorInfo)")
        // slack(
        //     message: errorInfo,
        //     slackUrl: "slackUrl",
        //     success: false
        // )
    }

    // All available actions: https://docs.fastlane.tools/actions

    // fastlane reports which actions are used. No personal data is recorded.
    // Learn more at https://github.com/fastlane/fastlane/#metrics
}

No Appfile found
fastlane gems

Gem
Version
Update-Status

fastlane
2.116.0
✅ Up-To-Date

Loaded fastlane plugins:
No plugins Loaded
Loaded gems

Gem
Version

did_you_mean
1.2.1

bundler
2.0.1

io-console
0.4.6

CFPropertyList
3.0.0

public_suffix
2.0.5

addressable
2.6.0

atomos
0.1.3

babosa
1.0.2

claide
1.0.2

colored
1.2

colored2
3.1.2

highline
1.7.10

commander-fastlane
4.4.6

declarative
0.0.10

declarative-option
0.1.0

digest-crc
0.4.1

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20180417

dotenv
2.6.0

emoji_regex
1.0.1

excon
0.62.0

multipart-post
2.0.0

faraday
0.15.4

http-cookie
1.0.3

faraday-cookie_jar
0.0.6

faraday_middleware
0.13.1

fastimage
2.1.5

gh_inspector
1.1.3

jwt
2.1.0

memoist
0.16.0

multi_json
1.13.1

os
1.0.0

signet
0.11.0

googleauth
0.6.7

httpclient
2.8.3

mime-types-data
3.2018.0812

mime-types
3.2.2

uber
0.1.0

representable
3.0.4

retriable
3.1.2

google-api-client
0.23.9

google-cloud-env
1.0.5

google-cloud-core
1.3.0

google-cloud-storage
1.16.0

json
2.1.0

mini_magick
4.5.1

multi_xml
0.6.0

plist
3.5.0

rubyzip
1.2.2

security
0.1.3

naturally
2.2.0

simctl
1.6.5

slack-notifier
2.3.2

terminal-notifier
1.8.0

unicode-display_width
1.4.1

terminal-table
1.8.0

tty-screen
0.6.5

tty-cursor
0.6.0

tty-spinner
0.9.0

word_wrap
1.0.0

nanaimo
0.2.6

xcodeproj
1.8.0

rouge
2.0.7

xcpretty
0.3.0

xcpretty-travis-formatter
1.0.0

generated on: 2019-02-19

Hey @andreyz 👋
I had a look at AppStoreConnect.md docs and AppVersions section might help to answer to your question 🙂
v = app.edit_version # the version that's in `Prepare for Submission` mode
v.ratings_reset = true # reset the ratings

# Push the changes back to the server
v.save!

I think you could try to give it ☝a shot 🙂

Thanks @mollyIV, I'm aware that Spaceship exposes that property. What I'm looking for is a way to set the same property in a Fastfile or by any other mean beyond scripting Spaceship in Ruby...

Oh I see, sorry @andreyz  😬 in that case there's a deliver (part of a fastlane) which uses spaceship under the hood. Unfortunately I couldn't find the option to reset the ratings in the docs and in the code upload_metadata.rb.
I think we could make a pull request where we would add a new input param (option) for deliver like reset_ratings # reset your rating. I would be happy to help with it 🤚
As a workaround that you can do right now is just using a spaceship directly in your Fastfile:
# Fastfile
require 'spaceship'

...

lane :reset_ratings do
    Spaceship::Tunes.login
    Spaceship::Tunes.select_team

    v = app.edit_version
    v.ratings_reset = true
    v.save!
end

I'd like to also hear someone with more experience with deliver that could confirm the above findings 🙏

Thank you for such extensive reply @mollyIV !
I'm using Fastlane Swift (which is in eternal beta), so referencing Spaceship directly from there is out of the question.
On the other hand, if deliver would support reset_ratings option, then the Swift wrapper would pick that change up, I think.

@andreyz I create the pull request for this feature. Could you please help me out with manual testing? 🙏
To do so I believe you need to modify your Gemfile to get the fastlane from my branch:
source "https://rubygems.org"

gem "fastlane", git: "https://github.com/mollyIV/fastlane", branch: "deliver-reset-summary-rating"

Please let me know in a case of any problems 🙇

@mollyIV you're a super-⭐!


Reference the bug that's preventing an inline function

Follow-up to https://github.com/square/okhttp/pull/6779/files

@swankjesse Thanks!


[PE-D][Tester B] Valid format for email specified in UG does not match the description given in the error message in CareerSync

Taken from page 41 of the UG:

Message for invalid email input in CareerSync through the edit command:
1. The local-part should only contain alphanumeric characters and these special characters, excluding the parentheses, (+_.-). The local-part may not start or end with any special characters.
2. This is followed by a '@' and then a domain name. The domain name is made up of domain labels separated by periods.
The domain name must:
    - end with a domain label at least 2 characters long
    - have each domain label start and end with alphanumeric characters
    - have each domain label consist of alphanumeric characters, separated only by hyphens, if any.```

<!--session:<PHONE_NUMBER>548-be5c0e40-6216-42bc-a930-910a6e19fa60--><!--Version: Desktop v3.5.3-->

-------------
Labels: `severity.VeryLow` `type.DocumentationBug`
original: biinnnggggg/ped#8

Documentation bug. Update the UG (field summary) and message_usage of email.


TST: Port Sequence unittest classes

3 new xfails
3 xpasses removed
@KatherineCaley could you check if any old ArraySeq-related tests in the following classes should be ported? If not, I think it's the last of the Sequence tests!

DnaSequenceTests
CodonSequenceTests
DnaSequenceGapTests
SequenceIntegrationTests
ModelSequenceSequenceTests

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
3 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.02%) to 91.605%

Files with Coverage Reduction
New Missed Lines
%

src/cogent3/maths/scipy_optimize.py
3
84.92%

Totals

Change from base Build<PHONE_NUMBER>:
0.02%

Covered Lines:
32813

Relevant Lines:
35820

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
3 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.02%) to 91.605%

Files with Coverage Reduction
New Missed Lines
%

src/cogent3/maths/scipy_optimize.py
3
84.92%

Totals

Change from base Build<PHONE_NUMBER>:
0.02%

Covered Lines:
32813

Relevant Lines:
35820

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

2 of 2 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 91.614%

Totals

Change from base Build<PHONE_NUMBER>:
0.03%

Covered Lines:
32818

Relevant Lines:
35822

💛  - Coveralls


refactor(internal): add commitzen / update jsDocs / cut release

add commitzen and update internal util getDisplayedArea docs

@swederik I'm adding commitzen here like in cornerstone-tools for consistency + using this PR to cut a release.

:tada: This PR is included in version 2.6.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Revert "Merge pull request #1733 from hjdivad/hjdivad/merge-nested-fix"

This reverts commit 4a55742a01cfd1c7c7b1dc24b1c8ca6f5ef446ea, reversing changes made to 95040bb97bf318449cec333d07500a69a35186a0.

@larry-x-yu i think we can close this one as i merged #1749

No longer needed.


Cleanup GAP fill feature.

This PR contains a series of commit related to cleanup of the gap fill feature.
Generally gap filling is not required for hex / s19 files, and is always done for binary files as they contain no address information.
Clean up the code and clarify this by introducing a dedicated gap fill pattern setting.
The pattern is the important value if wanting to speed up flash times when flashing binary files.
To still support gap filling of hex / s19 files, then introduce dedicated settings for those formats.
Those flags are disabled per default as they are generally not required, but may be needed for special handling.
The PR also addresses the issue of llvm-objcopy not supporting a gap-fill argument.
Part of this PR follows the request for changes given in https://github.com/zephyrproject-rtos/zephyr/pull/77924#pullrequestreview-2279486779 but was never addressed by the author of that PR.

soc/ti/k3/am6x/Kconfig.defconfig:49 should be fixed too


refactor: switch typeof undefined checks with strict comparison of the value undefined

It may be handy to use a more semantic utility with a positive naming convention.
Something like
const isDefined = <Value>(value: Value | null | undefined): value is Value =>
  value !== null && value !== undefined;

We can't check for null because data can be null - it just can't be undefined


[WIP] Update HMC adaptation interface, remove spl.info, and more tests

TODOs

[x] Log statistics during adaptation

E.g. step size, accept rate, etc

[x] Run MH as a baseline for sanity check on ESS
[x] Adaptation interface
[x] Write scripts to test each of below

HMC + DA
HMC + PD
HMC + DA + PD
NUTS + DA + PD

Test cases:

[x] gdemo
m12.6
[x] m12.2
LDA

I reverted https://github.com/TuringLang/Turing.jl/commit/16d6bba492fb4c5ee347874f2360cb4e681b2bee temporarily to let tests pass. I will update the interfaces first so that it can be easily experimented. Then we can check that later.

@yebai When designing the interface for setting configuration for adaptation, I realised that we need to tie adaptation algorithm with sampling algorithm, otherwise the Gibbs interface is hard to work with.

I started playing around with the NUTS sampler trying to improve the performance. So far I didn’t had the time to make systemic tests on my changes. I’ll evaluate and possibly integrate the changes after this PR is merged.
Do you also plan to look at ignoring the samples from the adaption phase for the returned chain object?

I reverted 16d6bba temporarily to let tests pass. I will update the interfaces first so that it can be easily experimented. Then we can check that later.

@xukai92
Sorry for the confusion. This PR contains some temporary code to adapt the pre-conditioning matrix in the original space, instead of the transformed space  (i.e. Euclidean). This is no longer necessary after 16d6bba492fb4c5ee347874f2360cb4e681b2bee. Pls take a look at: https://github.com/TuringLang/Turing.jl/commit/29cd4c85bc1c0bb022ee44838a793947ba9bd6e1

Yes this is the current setting I'm experimenting, i.e. the original v.s. Euclidean space codes are removed (as in https://github.com/TuringLang/Turing.jl/commit/29cd4c85bc1c0bb022ee44838a793947ba9bd6e1). But with the modification in https://github.com/TuringLang/Turing.jl/commit/16d6bba492fb4c5ee347874f2360cb4e681b2bee, the inference results doesn't seem to be correct, e.g. the inference check of gdemo in test/nuts.jl/nuts.jl doesn't pass with diagonal pre-conditioner.

Yes this is the current setting I'm experimenting, i.e. the original v.s. Euclidean space codes are removed (as in 29cd4c8). But with the modification in 16d6bba, the inference results doesn't seem to be correct, e.g. the inference check of gdemo in test/nuts.jl/nuts.jl doesn't pass with diagonal pre-conditioner.

Hi Kai, I went through the code again and your version of momentum is correct. I was confused by the covariance of theta, and the covariance of momentum variables. However, it's strange the incorrect momentum gives better ESS. Any idea?

Correct me if I'm wrong: I think what happened was the incorrect momentum somehow resulted in a large step size after adaptation phase. Running NUTS with larger step size which would just lead to larger ESS regardless of whether or not the momentum sampler is correct.
Also the table seems to indicate that there is something wrong within NUTS alone, because with diagonal PC, other samplers do have a gain in ESS.
I looked into the NUTS code again and found these lines suspicious. I think we should also correct momentum variable by PC matrix here: https://github.com/TuringLang/Turing.jl/blob/kx/fix-pd-bug/src/inference/nuts.jl#L117 and https://github.com/TuringLang/Turing.jl/blob/kx/fix-pd-bug/src/inference/nuts.jl#L174
Also I haven't seen ESS gain by doing so neither, I believe this should be fixed.

Updated ESS using 300dda3

julia> posterior = sample(m12_2(d[:density], d[:tank], d[:surv]),
                          Turing.NUTS(4000, 500, 0.95),
                          # Turing.HMCDA(4000, 500, 0.95, 0.3)
                         )
[ Info: [Turing] looking for good initial eps...
[ Info: [Turing] eps = 0.1, accept ratio a = 1.1184742546043598
[ Info: [Turing] eps = 0.11000000000000001, accept ratio a = 1.1942888456178937
[ Info: [Turing] eps = 0.12100000000000002, accept ratio a = 1.3234513576835787
[ Info: [Turing] eps = 0.13310000000000002, accept ratio a = 1.555321884953187
[ Info: [Turing] eps = 0.14641000000000004, accept ratio a = 2.008758175882085
[ Info: [Turing] eps = 0.16105100000000006, accept ratio a = 3.029480023216812
[ Info: [Turing] eps = 0.17715610000000007, accept ratio a = 5.942575359627319
[ Info: [Turing] eps = 0.1948717100000001, accept ratio a = 18.5796781383723
[ Info: [Turing] eps = 0.2143588810000001, accept ratio a =<PHONE_NUMBER>2263613
[ Info: [Turing] eps = 0.23579476910000013, accept ratio a = 6286.38203235946
[ Info: [Turing] eps = 0.25937424601000014, accept ratio a = 1.4602934555244546e7
[ Info: [Turing] eps = 0.28531167061100016, accept ratio a = 2.927396270806854e14
[ Info: [Turing] eps = 0.3138428376721002, accept ratio a = 3.2374057724820245e28
[NUTS] Sampling...100% Time: 0:06:56
[NUTS] Finished with
  Running time        =<PHONE_NUMBER>230004;
  #lf / sample        = 0.0;
  #evals / sample     = 88.89675;
  pre-cond. metric    = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0,....
Object of type Chains, with data of type 4000×56×1 Array{Union{Missing, Float64},3}

Log evidence      = 0.0
Iterations        = 1:4000
Thinning interval = 1
Chains            = Chain1
Samples per chain = 4000
internals         = elapsed, epsilon, eval_num, lf_eps, lf_num, lp
parameters        = a_tank[25], a_tank[41], a_tank[18], a_tank[1], a_tank[27], a_tank[7], a_tank[15], a_tank[12], a_tank[6], a_tank[31], a_tank[26], a_tank[11], a_tank[8], a_tank[43], a_tank[48], a_tank[10], a_tank[20], a_tank[45], a_tank[47], a_tank[21], a_tank[37], a_tank[39], a_tank[13], a_tank[9], a_tank[29], a_tank[38], a_tank[40], a_tank[34], a_tank[19], a_tank[2], a_tank[14], σ, a_tank[36], a_tank[22], a_tank[5], a_tank[4], α, a_tank[42], a_tank[23], a_tank[35], a_tank[46], a_tank[32], a_tank[30], a_tank[17], a_tank[24], a_tank[33], a_tank[44], a_tank[28], a_tank[3], a_tank[16]

parameters
             Mean    SD   Naive SE  MCSE     ESS
 a_tank[1]  2.0874 0.8569   0.0135 0.0184 2179.6495
 a_tank[2]  3.0820 1.1454   0.0181 0.0273 1759.0694
 a_tank[3]  0.9737 0.6478   0.0102 0.0098 4000.0000
 a_tank[4]  3.0146 1.1085   0.0175 0.0254 1902.7665
 a_tank[5]  2.1342 0.8555   0.0135 0.0137 3910.0172
 a_tank[6]  2.1370 0.8716   0.0138 0.0218 1595.1616
 a_tank[7]  3.0481 1.1157   0.0176 0.0320 1214.4261
 a_tank[8]  2.1155 0.8624   0.0136 0.0170 2571.8287
 a_tank[9] -0.1722 0.6021   0.0095 0.0151 1583.3423
a_tank[10]  2.1045 0.8533   0.0135 0.0180 2243.0769
a_tank[11]  0.9942 0.6701   0.0106 0.0103 4000.0000
a_tank[12]  0.5854 0.6288   0.0099 0.0092 4000.0000
a_tank[13]  0.9743 0.6388   0.0101 0.0102 3909.9762
a_tank[14]  0.1790 0.6267   0.0099 0.0082 4000.0000
a_tank[15]  2.1105 0.8666   0.0137 0.0146 3539.0543
a_tank[16]  2.1146 0.8622   0.0136 0.0190 2056.2411
a_tank[17]  2.9000 0.7873   0.0124 0.0135 3387.2094
a_tank[18]  2.3901 0.6683   0.0106 0.0115 3374.4636
a_tank[19]  2.0135 0.5798   0.0092 0.0102 3200.4977
a_tank[20]  3.6400 1.0313   0.0163 0.0276 1394.2323
a_tank[21]  2.3826 0.6745   0.0107 0.0125 2907.9989
a_tank[22]  2.3759 0.6647   0.0105 0.0140 2254.5778
a_tank[23]  2.3883 0.6628   0.0105 0.0111 3559.9914
a_tank[24]  1.7034 0.5300   0.0084 0.0069 4000.0000
a_tank[25] -1.0049 0.4489   0.0071 0.0101 1975.0659
a_tank[26]  0.1652 0.3914   0.0062 0.0063 3870.3732
a_tank[27] -1.4201 0.5107   0.0081 0.0146 1219.1459
a_tank[28] -0.4797 0.3978   0.0063 0.0076 2758.4433
a_tank[29]  0.1706 0.3956   0.0063 0.0042 4000.0000
a_tank[30]  1.4346 0.5148   0.0081 0.0122 1782.7890
a_tank[31] -0.6248 0.4405   0.0070 0.0099 1970.4885
a_tank[32] -0.3114 0.3986   0.0063 0.0082 2388.5491
a_tank[33]  3.1679 0.7608   0.0120 0.0134 3209.3237
a_tank[34]  2.6963 0.6204   0.0098 0.0106 3438.3232
a_tank[35]  2.7023 0.6481   0.0102 0.0127 2589.1435
a_tank[36]  2.0551 0.5070   0.0080 0.0096 2815.1312
a_tank[37]  2.0538 0.5110   0.0081 0.0060 4000.0000
a_tank[38]  3.8388 0.9664   0.0153 0.0223 1873.4088
a_tank[39]  2.7135 0.6457   0.0102 0.0151 1834.0260
a_tank[40]  2.3467 0.5586   0.0088 0.0099 3176.7577
a_tank[41] -1.8039 0.4903   0.0078 0.0101 2354.3516
a_tank[42] -0.5749 0.3446   0.0054 0.0052 4000.0000
a_tank[43] -0.4566 0.3473   0.0055 0.0069 2539.3825
a_tank[44] -0.3335 0.3584   0.0057 0.0083 1869.6294
a_tank[45]  0.5716 0.3429   0.0054 0.0057 3681.3659
a_tank[46] -0.5607 0.3534   0.0056 0.0054 4000.0000
a_tank[47]  2.0666 0.5149   0.0081 0.0078 4000.0000
a_tank[48]  0.0037 0.3413   0.0054 0.0063 2943.2044
         α  1.2902 0.2421   0.0038 0.0038 3989.0677
         σ  1.6135 0.2246   0.0036 0.0067 1125.1501

So does it mean it's the initial step size was too bad?

yes; there is another subtle bug fixed by https://github.com/TuringLang/Turing.jl/pull/709/commits/cdf7e34911a0bae680ce4b69836d873ee58c43c0, which is the possible cause of turning stepsize to extreme values occationally.

Got it. That's great! Thanks for catching that!

Updated ESS using 8486578
 julia> describe(c[1])
Log evidence      = 0.0
Iterations        = 1:4000
Thinning interval = 1
Chains            = 1
Samples per chain = 4000
parameters        = a_tank[25], a_tank[41], a_tank[18], a_tank[1], a_tank[27], a_tank[7], a_tank[15], a_tank[12], a_tank[6], a_tank[31], a_tank[26], a_tank[11], a_tank[8], a_tank[43], a_tank[48], a_tank[10], a_tank[20], a_tank[45], a_tank[47], a_tank[21], a_tank[37], a_tank[39], a_tank[13], a_tank[9], a_tank[29], a_tank[38], a_tank[40], a_tank[34], a_tank[19], a_tank[2], a_tank[14], σ, a_tank[36], a_tank[22], a_tank[5], a_tank[4], α, a_tank[42], a_tank[23], a_tank[35], a_tank[46], a_tank[32], a_tank[30], a_tank[17], a_tank[24], a_tank[33], a_tank[44], a_tank[28], a_tank[3], a_tank[16]

Empirical Posterior Estimates
───────────────────────────────────────────────────
parameters
             Mean    SD   Naive SE  MCSE     ESS
 a_tank[1]  2.1003 0.8569   0.0135 0.0134 4000.0000
 a_tank[2]  3.0777 1.1224   0.0177 0.0206 2966.8863
 a_tank[3]  1.0122 0.7028   0.0111 0.0081 4000.0000
 a_tank[4]  3.0292 1.1208   0.0177 0.0256 1910.1714
 a_tank[5]  2.1234 0.8508   0.0135 0.0128 4000.0000
 a_tank[6]  2.1137 0.8874   0.0140 0.0141 3933.2352
 a_tank[7]  3.0676 1.1364   0.0180 0.0276 1699.9622
 a_tank[8]  2.1202 0.8758   0.0138 0.0149 3471.2253
 a_tank[9] -0.1771 0.6253   0.0099 0.0067 4000.0000
a_tank[10]  2.1044 0.8639   0.0137 0.0126 4000.0000
a_tank[11]  0.9947 0.6780   0.0107 0.0098 4000.0000
a_tank[12]  0.5707 0.6176   0.0098 0.0049 4000.0000
a_tank[13]  0.9857 0.6648   0.0105 0.0091 4000.0000
a_tank[14]  0.1955 0.6165   0.0097 0.0074 4000.0000
a_tank[15]  2.1081 0.8577   0.0136 0.0150 3286.1862
a_tank[16]  2.1171 0.8639   0.0137 0.0130 4000.0000
a_tank[17]  2.8987 0.8101   0.0128 0.0132 3751.5649
a_tank[18]  2.3839 0.6712   0.0106 0.0091 4000.0000
a_tank[19]  2.0169 0.5924   0.0094 0.0075 4000.0000
a_tank[20]  3.6290 0.9808   0.0155 0.0200 2410.0861
a_tank[21]  2.3844 0.6692   0.0106 0.0094 4000.0000
a_tank[22]  2.3876 0.6573   0.0104 0.0095 4000.0000
a_tank[23]  2.3754 0.6857   0.0108 0.0110 3902.7362
a_tank[24]  1.7012 0.5532   0.0087 0.0069 4000.0000
a_tank[25] -0.9958 0.4517   0.0071 0.0062 4000.0000
a_tank[26]  0.1580 0.3855   0.0061 0.0058 4000.0000
a_tank[27] -1.4286 0.4989   0.0079 0.0070 4000.0000
a_tank[28] -0.4718 0.4123   0.0065 0.0065 4000.0000
a_tank[29]  0.1631 0.4030   0.0064 0.0055 4000.0000
a_tank[30]  1.4453 0.4935   0.0078 0.0067 4000.0000
a_tank[31] -0.6408 0.4352   0.0069 0.0057 4000.0000
a_tank[32] -0.3140 0.3977   0.0063 0.0045 4000.0000
a_tank[33]  3.1940 0.7839   0.0124 0.0139 3191.3482
a_tank[34]  2.7118 0.6569   0.0104 0.0104 4000.0000
a_tank[35]  2.7100 0.6370   0.0101 0.0104 3768.0450
a_tank[36]  2.0645 0.5210   0.0082 0.0066 4000.0000
a_tank[37]  2.0538 0.5074   0.0080 0.0074 4000.0000
a_tank[38]  3.8684 0.9732   0.0154 0.0200 2365.1137
a_tank[39]  2.6992 0.6491   0.0103 0.0092 4000.0000
a_tank[40]  2.3446 0.5462   0.0086 0.0079 4000.0000
a_tank[41] -1.8085 0.4845   0.0077 0.0061 4000.0000
a_tank[42] -0.5761 0.3455   0.0055 0.0048 4000.0000
a_tank[43] -0.4552 0.3472   0.0055 0.0047 4000.0000
a_tank[44] -0.3321 0.3497   0.0055 0.0051 4000.0000
a_tank[45]  0.5760 0.3511   0.0056 0.0052 4000.0000
a_tank[46] -0.5664 0.3604   0.0057 0.0050 4000.0000
a_tank[47]  2.0663 0.5150   0.0081 0.0077 4000.0000
a_tank[48] -0.0036 0.3386   0.0054 0.0051 4000.0000
         α  1.2925 0.2536   0.0040 0.0042 3676.5900
         σ  1.6180 0.2251   0.0036 0.0049 2151.3881

How does this PR relate to AdvancedHMC?

@mohamed82008 This PR was originally created to fix some adaptation numerical bugs and update Turing's interface to pass different adaptation options. However, as

@yebai fixed the main bugs in another PR
some interface issues are resolved by AdvancedHMC
the new codes introduced by this PR is very limited - mainly it's just remove all in-sampler statistics from spl.info and instead returning them by each step function. To achieve this I introduced a new type for each sampler. E.g. HMCState for HMC that contains stats collected during each HMC iteration, to replace the original returned accepted probability (https://github.com/TuringLang/Turing.jl/pull/641/files#diff-b2000c195c9a7dd900b574dec178b527R224).

In fact, I'm thinking closing this PR first and do a new one later, when AdvancedHMC is stable to be integrated. Otherwise it's a bit tricky to maintain two copies of codes for the same functionality. How do you think?

Yes, currently I prefer minimal code addition to Turing. Code removal is fine. So if bug fixes already went into Turing, I recommend waiting for AdvancedHMC.jl. I finally have a working state of the type stability PR (at least for Hamiltonian), really not looking forward to a 3-hour rebase!

Nice! Neither do I was enjoying resolving all the conflicts yesterday :( Let's close it then

Thanks!


Update Filter when Area Type is changed (#5643)

What
https://user-images.githubusercontent.com/3719745/161991533-dbc8dd8a-af6e-4dbb-b3a4-aa144d461c46.mp4
Hooks up the button underneath the area-type radio selection screen so that it updates the dimension in Mongo (via the filter-flex API).
This is dependent on (in order to actually work):

https://github.com/ONSdigital/dp-cantabular-filter-flex-api/pull/46
https://github.com/ONSdigital/dp-cantabular-filter-flex-api/pull/54
https://github.com/ONSdigital/dp-filter-api/pull/236
https://github.com/ONSdigital/dp-api-clients-go/pull/235

Resolves: 5643
ID/Label mismatch
We still have the ID/Label mismatch with the  dimensions returned from the Dataset API.
Until this is fixed, I've changed the radio button value to be the label rather than the ID. H
Options list on review screen
Edit: The review screen doesn't display variables anymore since the model update, work is in progress to fix it
Since the returned dimension doesn't exist on the original dataset, there aren't any options displayed next to it (e.g. United Kingdom next to City). There is additional work in flight to add this in by querying the /areas endpoint of the Dimensions API.
How to review
Confirm the behaviour matches the ticket.
Who can review
Anyone.

@mr-nick17 Might be worth a look if you get a chance. The review screen will still be empty, but the change does still take effect.

@mr-nick17 Added validation, ready for a re-review 👍

Minor comments, please update the error string and then we're good to 🚢


RFP - VirtualDub2

Package Details
Software project URL : https://sourceforge.net/projects/vdfiltermod/
Direct download URL for the software / installer : https://sourceforge.net/projects/vdfiltermod/files/latest/download
Software summary / short description: A streamlined video editor with simple frame-by-frame timeline. Free, open-source, portable. The default package is ready for use with many essential plugins included, and can be further extended with 3rd party codecs and filters.  Previous name of this project: VirtualDub FilterMod. Maintained successor to VirtualDub.
Virtual Dub 2: http://virtualdub2.com/
Virtual Dub original: http://virtualdub.org/
Package Expectations

Add shims for CLI exe's - vdub64.exe, vdub.exe (just a single one for appropriate host OS type would be fine)
Add menu link to GUI - VirtualDub64.exe or VirtualDub.exe

Status: Review
https://chocolatey.org/packages/virtualdub2

Status: Published

https://chocolatey.org/packages/virtualdub2

Can be closed.


[DOCS] Azure DNS zone service principal required role can be more limited

The documentation states that the service principal used for Azure DNS Zone management has to have contributor rights on the resource group where DNS Zone is located. I just did a test -> "reader" on the resource group and "contributor" on the DNS zone is actually enough, no need to be overly permissive.
However, I am not 100% sure, although I did check the Azure provider source code as well. Can someone confirm this? I will be happy to provide a pull req for docs / tutorials if needed.

Ok, I've submitted a pull request for this


Behave more similarly to Angular's schematics

I'm submitting a...

[ ] Regression (a behavior that used to work and stopped working in a new release)
[ ] Bug report
[ ] Performance issue
[x] Feature request
[ ] Documentation issue or request
[ ] Support request => https://github.com/ngxs/store/blob/master/CONTRIBUTING.md
[ ] Other... Please describe:

Current behavior

ng g @ngxs/schematics:state states/test
CREATE src/states/test/test.state.spec.ts (685 bytes)
CREATE src/states/test/test.state.ts (337 bytes)

Expected behavior

ng g @ngxs/schematics:state states/test
CREATE src/app/states/test.state.spec.ts (146 bytes)
CREATE src/app/states/test.state.ts (22 bytes)

What is the motivation / use case for changing the behavior?

I'd like to be able to use NGXS' schematics the same way I use Angular's schematics. Right now, if I want the same result, I have to run a command like this :
ng g @ngxs/schematics:actions --name logout --sourceRoot src/app --path actions --spec true

With Angular's schematics, the commands looks like this :
ng g class classes/test

Additionaly I hope, that it also will behave more like angular schematics when using in IntelliJ. When I generate components they are generated inside the folder I want them to be created. When I to the same with ngxs/schematics I get the full path from root generated in the folder I want the files to be placed in. So from the example above when trying to generate files in src/app/states I get the files in src/app/states/src/app/states/test.

I added schematics configuration to angular.json and it partially solves the problem.
"@ngxs/schematics:action": {
	"spec": false,
	"sourceRoot": "src/app"
},
"@ngxs/schematics:store": {
	"spec": false,
	"sourceRoot": "src/app"
},
"@ngxs/schematics:state": {
	"spec": false,
	"sourceRoot": "src/app"
},
"@ngxs/schematics:actions": {
	"spec": false,
	"sourceRoot": "src/app"
}


Defer isInClient() call

Currently, iOS will show this when opening LIFF

Related discussion:
https://g0v-tw.slack.com/archives/C2PPMRQGP/p1583323511054400
This PR avoids this condition by moving isInClient API call to after liff.init().
Note: the "open links in Safari" toggle in iOS LINE does not matter. It seems that LIFFs are always opened in built-in browsers regardless of the toggle.

Pull Request Test Coverage Report for Build 828

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 98.438%

Totals

Change from base Build 825:
0.0%

Covered Lines:
359

Relevant Lines:
360

💛  - Coveralls


Navigation Bar Drop Down Menus and Accessibility

Been working on adding drop down menus to the navigation bar. Open to feedback or suggestions. I'll leave this open for a couple days so people have an opportunity to check it out and make sure it looks good.
Worked off comment from #1759 for accessibility. Tested with the following:

Firefox and Chromium on Windows with NV Access Screen Reader
Chrome on Android with Google TalkBack

fixes #1750
Here's some screenshots:

@violetcraze the navbar does not work on Mac using Safari. Could you cross check that?

I can confirm that it's not working on safari, don't seen anything in console.
Also, small thing but the "hand" doesn't appear when hovering the navbar as it does for other interactive elements on the page:

It doesn't work on Chrome either on my Mac. Only the Home button works as expected.
Looks like there was some regressions from the latest changes.

Great work, @violetcraze! ^^
It's working for me - but only when you actually click the element. @gruselhaus / @shiffman / @phucduong86 have you tested hovering or clicking? Also adding a onmouseover event would make it feel more natural to desktop users.
The "hand" could easily be fixed by applying cursor: pointer; to the respected HTML element in CSS. Then it also feels "clicky".