network policy doesn't work (even deny all)

Describe the bug
Simply define a deny-all network policy, but it doesn't work as expected
To Reproduce
in my environment, I delete Antrea add-on, Kubeadm reset on node, and re-init and apply add-on, having the same issue
Expected
block all traffic between pods
Actual behavior
NetworkPolicy is not configured via Antrea, and traffic doesn't block
Versions:
root@an01:/# antctl version
agentVersion: 0.10.0-dev-6be5e2f.clean
antctlVersion: v0.10.0-dev-6be5e2f
root@an01:/# antctl get networkpolicy
root@an01:/# antctl version
agentVersion: 0.10.0-dev-6be5e2f.clean
antctlVersion: v0.10.0-dev-6be5e2f
kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.0", GitCommit:"e19964183377d0ec2052d1f1fa930c4d7575bd50", GitTreeState:"clean", BuildDate:"2020-08-26T14:30:33Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.0", GitCommit:"e19964183377d0ec2052d1f1fa930c4d7575bd50", GitTreeState:"clean", BuildDate:"2020-08-26T14:23:04Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Additional context
barany@an01:~$ kubectl get pods --show-labels
NAME                         READY   STATUS    RESTARTS   AGE   LABELS
nginx10                      1/1     Running   0          15m   run=nginx10
web-96d5df5c8-bmfb2          1/1     Running   0          18m   app=web,pod-template-hash=96d5df5c8
web-96d5df5c8-fnk8g          1/1     Running   0          18m   app=web,pod-template-hash=96d5df5c8
web-96d5df5c8-q289m          1/1     Running   0          18m   app=web,pod-template-hash=96d5df5c8
web-app02-75cb584798-h4p9d   1/1     Running   0          18m   app=web-app02,pod-template-hash=75cb584798
web-app02-75cb584798-hwtkt   1/1     Running   0          18m   app=web-app02,pod-template-hash=75cb584798
web-app02-75cb584798-rgskp   1/1     Running   0          18m   app=web-app02,pod-template-hash=75cb584798
barany@an01:~$ kubectl get pods -o wide
NAME                         READY   STATUS    RESTARTS   AGE   IP            NODE   NOMINATED NODE   READINESS GATES
nginx10                      1/1     Running   0          16m   <IP_ADDRESS>   an03
web-96d5df5c8-bmfb2          1/1     Running   0          18m   <IP_ADDRESS>    an02
web-96d5df5c8-fnk8g          1/1     Running   0          18m   <IP_ADDRESS>    an03
web-96d5df5c8-q289m          1/1     Running   0          18m   <IP_ADDRESS>    an03
web-app02-75cb584798-h4p9d   1/1     Running   0          18m   <IP_ADDRESS>    an02
web-app02-75cb584798-hwtkt   1/1     Running   0          18m   <IP_ADDRESS>    an02
web-app02-75cb584798-rgskp   1/1     Running   0          18m   <IP_ADDRESS>    an03
barany@an01:~$ kubectl describe networkpolicy
Name:         denyall
Namespace:    default
Created on:   2020-08-30 04:48:32 +0000 UTC
Labels:
Annotations:
Spec:
PodSelector:      (Allowing the specific traffic to all pods in this namespace)
Allowing ingress traffic:
 (Selected pods are isolated for ingress connectivity)
Allowing egress traffic:
 (Selected pods are isolated for egress connectivity)
Policy Types: Ingress, Egress
barany@an01:~$ kubectl get  networkpolicy
NAME      POD-SELECTOR   AGE
denyall            5m10s
barany@an01:~$ kubectl exec -it web-app02-75cb584798-hwtkt -- curl -i <IP_ADDRESS>
HTTP/1.1 200 OK
Server: nginx/1.19.2
Date: Sun, 30 Aug 2020 04:54:08 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 11 Aug 2020 14:50:35 GMT
Connection: keep-alive
ETag: "5f32b03b-264"
Accept-Ranges: bytes

Welcome to nginx!

Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.

barany@an01:~$

@nsxdemo thanks for reporting. However I cannot reproduce it with master builds. Could you share the status and the logs of antrea-controller and antrea-agent Pods? You may use antctl to get them: https://github.com/vmware-tanzu/antrea/blob/master/docs/antctl.md#collecting-support-information

Hi, I'm seeing a similar issue with v0.9.1.
Networkpolicy is configured but there are no OVS flows to block traffic in EgressDefaultTable.

@yktsubo could you share logs? I cannot reproduce with 0.9.1 either.
This is the deny-all policy I used:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
spec:
  podSelector: {}
  policyTypes:
  - Egress

The applied policy is
 $ k -n elearning get netpol default-deny -o yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.k8s.io/v1","kind":"NetworkPolicy","metadata":{"annotations":{},"name":"default-deny","namespace":"elearning"},"spec":{"podSelector":{},"policyTypes":["Ingress","Egress"]}}
  creationTimestamp: "2020-08-28T09:13:54Z"
  generation: 1
  managedFields:
  - apiVersion: networking.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:kubectl.kubernetes.io/last-applied-configuration: {}
      f:spec:
        f:policyTypes: {}
    manager: kubectl
    operation: Update
    time: "2020-08-28T09:13:54Z"
  name: default-deny
  namespace: elearning
  resourceVersion: "219744"
  selfLink: /apis/networking.k8s.io/v1/namespaces/elearning/networkpolicies/default-deny
  uid: a081e326-d64d-4fe5-a896-988568949e48
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

Here is the list of pods running in the namespace
$ k -n elearning get pod -o wide
NAME                       READY   STATUS    RESTARTS   AGE     IP           NODE         NOMINATED NODE   READINESS GATES
course-655f9b8b69-rx9rb    1/1     Running   0          37m     <IP_ADDRESS>   k8s-node-1   <none>           <none>
debug                      1/1     Running   0          2d20h   <IP_ADDRESS>   k8s-node-1   <none>           <none>
rating-575dbf8478-k8zdl    1/1     Running   0          2d22h   <IP_ADDRESS>   k8s-node-2   <none>           <none>
student-6458d58469-w68rn   1/1     Running   0          52m     <IP_ADDRESS>   k8s-node-1   <none>           <none>
teacher-7c57f94f7b-shjv9   1/1     Running   0          2d22h   <IP_ADDRESS>   k8s-node-2   <none>           <none>

What I found is that table50 doesn't have correct flows to block traffic on both node-1 and node-2.
But all connection seems fine.
support-bundles_20200831T125822+0800.tar.gz

Here is the example from node1
$ kubectl -n kube-system exec -it $(kubectl -n kube-system get pod -o wide | grep k8s-node-1 | grep antrea-agent | cut -d ' '  -f 1) -c antrea-agent -- ovs-ofctl -O OpenFlow13 dump-flows br-int  --no-stats | grep 'table=50'
 cookie=0x1000000000000, table=50, priority=210,ct_state=-new+est,ip actions=goto_table:70
 cookie=0x1000000000000, table=50, priority=0 actions=goto_table:60

From my understanding, node-1 should block traffic from <IP_ADDRESS> and <IP_ADDRESS> to outside.

Thanks @yktsubo for sharing the information and providing a way to reproduce it.
After applying https://github.com/yktsubo/mock-demo several times, I can see that the networkpolicy reconcilers were not running any more (no Reconciling rule logs) while the agent can receive correct NetworkPolicy and AppliedToGroup events from the controller correctly. It should be the agent network policy workers were stuck by some tasks somehow.
After dumping the goroutines, there was indeed a deadlock when updating a ClusterNetworkPolicy rule:
sync.runtime_SemacquireMutex(0xc0006cc80c, 0x0, 0x1)
	/usr/local/go/src/runtime/sema.go:71 +0x47
sync.(*Mutex).lockSlow(0xc0006cc808)
	/usr/local/go/src/sync/mutex.go:138 +0xfc
sync.(*Mutex).Lock(...)
	/usr/local/go/src/sync/mutex.go:81
sync.(*RWMutex).Lock(0xc0006cc808)
	/usr/local/go/src/sync/rwmutex.go:98 +0x97
github.com/vmware-tanzu/antrea/pkg/agent/controller/networkpolicy.(*reconciler).uninstallOFRule(0xc000534ea0, 0x3000000011, 0x0, 0x0)
	/antrea/pkg/agent/controller/networkpolicy/reconciler.go:640 +0x4bf
github.com/vmware-tanzu/antrea/pkg/agent/controller/networkpolicy.(*reconciler).update(0xc000534ea0, 0xc0007e77a0, 0xc0003c8380, 0xc000736098, 0xc000736030, 0x0, 0x0)
	/antrea/pkg/agent/controller/networkpolicy/reconciler.go:589 +0xc43
github.com/vmware-tanzu/antrea/pkg/agent/controller/networkpolicy.(*reconciler).Reconcile(0xc000534ea0, 0xc0003c8380, 0x0, 0x0)
	/antrea/pkg/agent/controller/networkpolicy/reconciler.go:222 +0x35d
github.com/vmware-tanzu/antrea/pkg/agent/controller/networkpolicy.(*Controller).syncRule(0xc000534f00, 0xc0004ba1e0, 0x10, 0x0, 0x0)
	/antrea/pkg/agent/controller/networkpolicy/networkpolicy_controller.go:405 +0x114
github.com/vmware-tanzu/antrea/pkg/agent/controller/networkpolicy.(*Controller).processNextWorkItem(0xc000534f00, 0x407000)
	/antrea/pkg/agent/controller/networkpolicy/networkpolicy_controller.go:359 +0xf4
github.com/vmware-tanzu/antrea/pkg/agent/controller/networkpolicy.(*Controller).worker(0xc000534f00)
	/antrea/pkg/agent/controller/networkpolicy/networkpolicy_controller.go:348 +0x2b

The mutex will be acquired two times if there are any stale openflow rules need to be deleted for a ClusterNetworkPolicy:
https://github.com/vmware-tanzu/antrea/blob/68fa2210e408723332a95ec2397f52cb38c5f2b5/pkg/agent/controller/networkpolicy/reconciler.go#L207-L213
https://github.com/vmware-tanzu/antrea/blob/68fa2210e408723332a95ec2397f52cb38c5f2b5/pkg/agent/controller/networkpolicy/reconciler.go#L639-L642
Then all workers that were reconciling ClusterNetworkPolicies would be pending on the lock, causing normal NetworkPolicies cannot be reconciled as well.
The issue is only present after enabling AntreaPolicy (formerly called ClusterNetworkPolicy) feature.
@Dyanngg could you consider a fix and check if there are other similar issues?


Set DISABLE_WP_CRON from env var

This allows better control via configuration management like Trellis.
Also change the default to false. This setting requires external server configuration for cron to work and since Bedrock is a standalone boilerplate, we shouldn't require that by default.

docs will need updating: https://roots.io/bedrock/docs/wp-cron/

@retlehs no need for that doc anymore


Add Rector to CI Pipeline

Codecov Report

Merging #3 (f4e8632) into main (507ba33) will decrease coverage by 0.11%.
The diff coverage is n/a.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@             Coverage Diff              @@
##               main       #3      +/-   ##
============================================
- Coverage     67.55%   67.44%   -0.11%
  Complexity      287      287
============================================
  Files            22       22
  Lines           980      980
============================================
- Hits            662      661       -1
- Misses          318      319       +1

Files Changed
Coverage Δ

src/Detector/Strategy/DefaultStrategy.php
78.94% <ø> (ø)

...SuffixTree/ApproximateCloneDetectingSuffixTree.php
94.11% <ø> (-0.46%)
:arrow_down:

src/Detector/Strategy/SuffixTreeStrategy.php
95.00% <ø> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more
:loudspeaker: Have feedback on the report? Share it here.


Implement PaginatorTrait for SelectMany

The PaginatorTrait is implemented only for Selector and SelectorTwo types.
Implementation for SelectTwoMany is missing.
Implementation for SelectorRaw was added recently, which is not an alternative for us, since we don't want to construct raw SQL statements.
=> SeaORM contribution
The paginator implementation can be found here
A working example of paginator looks like:
let paginator = ManagedDevices::find()
    .paginate(conn, page_size);
paginator.fetch_page(page - 1).await

While adding find_with_related(...) doesn't compile , since it returns SelectTwoMany type, which doesn't implement the PaginatorTrait:
let paginator = ManagedDevices::find()
        .find_with_related(Agents)
        .paginate(conn, page_size);
paginator.fetch_page(page - 1).await

pagination is not support by intention, and SeaORM has plans to change the pagination concept
we can use the QuerySelect offset/limit features to achieve pagination functionality

Fixed


Add Haltscam project

Haltscam allows and provides a means of crowd-sourcing risky Cardano addresses, allowing community members to contribute to the database and prevent their fellow adopters from falling victim to scams or solicitation.

Thanks to the work we are doing in Haltscam we got CF delegation.

Hi @HT-Moh can you please provide us with your logo, and some profiles of the main individuals involved? Also if you have a blurb explaining your project and/or whitepaper/specification that would be great. Thank you!

Hi @nahern
Profiles:
Github: https://github.com/HT-Moh
Linkedin: https://www.linkedin.com/in/mohamed-habbat-41233428/
https://www.linkedin.com/in/flurina-giger-b09373204/
Github: https://github.com/HTHoussam
Linkedin: https://www.linkedin.com/in/habbathoussam/
Github: https://github.com/AdnCodez
Linkedin: https://www.linkedin.com/in/adnan-habbat-74bb9b53/
https://haltscam.com/roadmap

haltscam_project.pdf


Web3ApiClient Cache Garbage Collection

Currently, the Web3ApiClient caches each API that's queried for the lifetime of the client instance. This is not scalable, and will bloat application memory when too many APIs are queried (and downloaded).
In order to avoid this, we can add a garbage collection configuration which will allow the user to define rules for un-caching APIs. Some potential parameters may be:

Max number of cached APIs
Query staleness. For example, remove an API if it hasn't been used in the last 50 queries.

Or would it be something like:
// Update ClientConfig interface to include cachedApiLimit:
export interface ClientConfig<TUri = string> {
  redirects?: UriRedirect<TUri>[];
  tracingEnabled?: boolean;
  cachedApiLimit?: number;
}

And then in the loadWeb3Api method:
  public async loadWeb3Api(uri: Uri): Promise<Api> {
    const run = Tracer.traceFunc(
      "Web3ApiClient: loadWeb3Api",
      async (uri: Uri): Promise<Api> => {

// Return if the cache size is at the limit.
        if (this._apiCache.size === cachedApiLimit) return;
...

@kev-ngo Apollo's GC is certainly a great reference point, but I think its use-case is a bit different from ours, so our design goals may not be too compatible with their implementation. Still TBD though.
As for the psuedo-code you've added, I do think the GC config would be apart of the Client's configuration. However, I don't think we should "throw" if the "max number of APIs" has been reached. Instead it should try to offload one of the unused APIs so it can load the new one. How this will work if the max number has been reached, and all APIs are currently in use, I'm not sure.

Introduction
Polywrap currently caches each wrapper used for the entire lifetime of the client instance. This can cause scaling issues in the future as the cache bloats when many wrappers are queried and downloaded. To remedy this, a system will be needed to allow users to specify when the application should remove certain wrappers from the cache.
Background
This proposal views the introduced problem through two perspectives: at first glance, this seems to be a classic caching problem that, if approached as such, would allow maximum flexibility to end users; on the other hand, there is something to be said about the amount of static information that is known ahead of time, which may allow for certain optimizations to be made, even in such a highly dynamic environment. On this second view, we can instead consider usage of the client SDK as a kind of JIT-compiled program with dependencies as 'variables' having lifetimes, rather than dynamic libraries that are loaded on demand. The benefit to this perspective being that we can subsequently attempt to constrain the general problem of caching any data to a more narrow problem of garbage collecting shared variables in a concurrent system (which may provide us with better results, independently of any underlying caching perspective).
System Dynamism
All of the dependencies that a wrapper may use are known statically at "compile"-time, as they must be declared in the wrapper manifest. However, while the wrapper declares the set of all possible dependencies, the set of dependencies that will actually be used throughout the course of running the wrapper is some proper subset thereof — wrappers may have an "interface"-like dependency, where they expect to execute some functionality by contract, with the actual implementation not being determined until runtime — which is known only at runtime and depends upon the control flow execution of the wrapper itself.
Security Considerations
Understanding this problem as one related to caching provides us with insight into the potential attack vectors that would be familiar to malicious actors: timing attacks constitute the main class of potential attacks that can be executed against caches — by timing cacheable requests/operations, a wrapper may be able to interpret cache hits/misses and thus determine what other wrappers a client may be using, despite being sandboxed and unable to access them. What benefit an attacker might gain from such knowledge is not immediately clear, but we might imagine that it could serve some additional purpose to a clever actor, beyond the extant privacy concern.
Scope

Dependency caching solution generalized to any SDK/native languages (client language runtime: JS, Rust, etc.)
Applicable to multiple platforms and environments, including web and embedded (client system environment: browser, desktop, IoT)
NOT IN SCOPE: handling the caching of wrapper-internal elements (wrapper "heap" vs wrappers themselves)

Requirements

Must provide extreme developer flexibility of choice

Ex. Profiles: memory constrained (IoT), default (app, browser/desktop), performant (bounded memory space)
Users should be able to specify constraints as either "hard caps" or "soft caps", depending on whether they can be violated (and with what severity)

Must be highly configurable, but come with sane defaults for those users who aren't interested in GC-/cache-tuning

Users are welcome to manually specify certain cachable elements to be preloaded

Must provide general global options to describe the environment

Must provide appropriate options for highly constrained environments (e.g. allowing for limits on global memory/network usage)

Prior Art
Garbage Collection
Garbage collection, also sometimes known as automated memory management, has a long and storied history stretching back over sixty years. It is a well-researched and deeply explored field in both academia and industrial practice, and explaining it in any significant detail is both beyond the scope and ability of this document. For a good primer on garbage collection techniques and principles, consider [4]. For a more casual, practical description of a specific garbage collector implementation, consider [9].
In general, we may consider garbage collection to be the automatic process by which memory is allocated and reclaimed over the course of the lifetime of a program. This is often achieved in one of two popular ways (viz. tracing, reference counting), both having their own performance tradeoffs. In addition to this dichotomy, there is also the important distinction of when garbage is collected, and whether that occurs on the same thread as the main program, or concurrently, or in realtime.
Tracing Garbage Collectors
Tracing garbage collectors rely on the idea of a call graph, where certain elements of a program can be marked as "root" elements, against which all other elements are referenced. The garbage collector can then employ a strategy to traverse the graph and detect disconnected elements, removing any that weren't found to pertain to actively 'living' root elements. This analogy serves our case of dependencies well, as dependencies are often nested and interconnected in meaningful ways, where we are able to assume that they aren't needed right now if no one is using them.
Reference Counting Garbage Collectors
Similar to tracing garbage collectors, reference-counting garbage collectors concern themselves with keeping track of which elements can be disposed of. This is often done in a simple, greedy way: uses of an element increment a counter, and disuse appropriately decrements it; when the counter reaches zero, the element is immediately marked for garbage collection. A meaningful comparison of the two garbage collection techniques is beyond the scope of this document; however, it should be noted that popular programming languages which employ garbage collectors often allow the end users to select between possible algorithms, which sets a positive precedent (and is more ergonomic to developers who are familiar with the paradigm) for providing the same to users of a Polywrap SDK.
Cache Replacement Strategies
As with garbage collection, a full overview of cache replacement is outside the scope of this document. That being said, there exist many popular strategies with famous acronyms, (e.g. LRU, LFU, etc.), exhibiting different performance characteristics and each being suited better or worse for different use patterns. Amongst these, two are worth making points about directly, and an additional discussion will be had about the place of statistical learning in cache replacement policies.
ARC
Adaptive Replacement Cache (ARC) [8], is by-and-large considered to be one of the best general cache replacement policies. It combines the benefits of both the LRU (Least Recently Used) and LFU (Least Frequently Used) algorithms by maintaining two cache lists, one for each. ARC, in turn, can be improved upon by including further information, such as timing and interval sizes.
RRIP
An interesting approach worth noting is that of Re-reference Interval Prediction (RRIP) [3]. By attempting to predict the next time a dependency is used, a more intelligent decision can be reached regarding whether or not to evict it (dependencies which are not currently used but may be used in the near future would maintain preference over those that, despite also currently being unused are not likely to be used again soon). See the linked paper for further details.
Learning-based Cache Replacement
Modern advances in machine learning present us with an interesting opportunity to apply statistical reasoning to the question of what to evict from our cache next. For context, it is often considered that an "oracle algorithm" (Bélády's algorithm) is the theoretical limit of optimal cache eviction; that is, always correctly choosing the cache element which will be used furthest into the future (and thus needing to know the future). We can attempt to approximate this perfect understanding using machine learning, for which appropriate frameworks are beginning to appear [14].
One possible approach that has been theorized is to adopt a regret-minimization perspective. Popularized in game and decision theories, regret minimization is a strategy which posits that a good way of narrowing down which decision you should make is to determine how much you will regret every possible decision, and then minimizing across those regret variables (that is, pruning the most regrettable options until an optimal choice, or set of choices, is elucidated). This perspective can be used to attempt to integrate user specifications with statistically learned responses to the dynamic environment expected of a wrapper cache. It could also potentially provide an interesting input methodology for users of the client SDKs, vis a vis hard caps and soft caps.
Other Related Situations
Dynamically Loaded Libraries
Many modern operating systems provide functionality for dynamically loading additional libraries at runtime. This allows for programs to begin without having to link to every library that they expect to use, potentially saving on time and space (or simply being required to when the information is not known at link time). Linux, in particular, employs a reference-counting strategy to decide when to deallocate dynamically loaded libraries [6].
Proposals
To preface the following two discussions, it should be noted that in either the garbage collection (GC) perspective, or the cache perspective, we should uphold our goal of providing end-users with maximum flexibility to make the correct, informed decision for their individual usage pattern; consequently, we must acknowledge that different platforms/SDKs may be able to provide different options to their users, and may have different reasonable defaults as a result. Furthermore, users may provide global constraints (such as maximum memory usage, or strict deadlines) which can act as heuristics to determine the best choices on-the-fly.
1. Garbage Collection
In modern programming languages with automated memory management, a shared terminology exists regarding how information is stored: the "heap", and the "stack", with the general conception here being that information of a known size and lifetime will be stored on the stack and immediately evicted once the containing "frame", or query/function, is finished. However, not all data has a known size at "compile time": that data, of dynamic size, is instead treated as a (boxed) pointer to some memory space that has been allocated on the heap. This data needs to be handled in a reasonable way that prevents the heap from leaking memory, while still providing live pointers to the parts of the program that need them. Different languages have different approaches to what to do with this heap data: Rust employs a borrow checker which enforces certain invariants and determines the lifetimes of heap data, removing it once it has "expired" and will not be accessed by anything else in the future; Nim, Java, and many others offer several GC algorithms for users to choose from.
Keeping with the traditional terminology, we can understand garbage collection in Polywrap clients as an effort in static analysis that ultimately allows us to plan stack frames appropriately, as well as handle the dynamic dependencies on the "heap" by employing an appropriate GC algorithm.
Implementation
While much of the execution of a query is deterministic (cf system dynamism), the exact extent to which stack frames can be planned effectively (and, consequently, the cache managed efficiently) is dependent upon the level of static introspection possible into the underlying dependencies: it is not enough to know which wrappers are used, we must also know in what order they will be used if we are to plan optimal cache evictions. That being said, the more static analysis that can be performed, the better the caching can be planned; but, even without any static introspection, certain GC work can still be done.
Optimal Cache Replacement
In a perfect world, our goal would be to have every wrapper ready-to-go just before it needs to be used and gone just after it won't be needed any longer. To do that, we would need to know when each wrapper is needed and when it isn't anymore. Manifests tell us which dependencies a wrapper might use, but not when or in what order — using manifests alone to inform us would let us take the conservative approach of simply ensuring that every possible dependency is available before a query is run, the efficiency of which is the ratio of the cardinality of the sets of the declared vs. actually-used dependencies. If, however, we know more than just what the manifest tells us, we can become more precise in our layouts and allow for executions that would seem impossible on the conservative approach due to the memory constraints.
Suppose that we are executing a wrapper with declared dependencies a, b, c, d, and e. Suppose further than we know from the manifest that a and b depend upon c, that c in turn depends on d, and that e depends only on d; and, from abstract interpretation we have determined that the wrapper first makes a call to b, then to a, then to e, and then finally exits. Given this information, we have the runtime execute the wrapper "program" as such:

retrieve from cache or download d
retrieve from cache or download c
retrieve from cache or download b // we cannot execute out-of-order without knowing if there are side effects to the calls
execute b
evict b
retrieve from cache or download a
execute a
evict a and c
retrieve from cache or download e
execute e
evict d and e
exit

This execution plan would ensure that dependencies are present when needed and removed when they will no longer be used. However, importantly, constraints placed upon the cache size, upon network usage, and other criteria, can change the order of these steps (e.g. having a large cache size and no constraint on startup time would allow us to move steps (6) and (9) up towards the top, and to move steps (5) and (8) down to join step (11)). In this simplistic example we have a fairly linear approach, where only a few steps can be moved around (e.g. steps (1) through (3) can be executed in any order, ostensibly), without much depth. In a more likely scenario, however, we might imagine this graph becoming increasingly large as the branching factor stays above 1 in the deeper dependencies, fanning out the graph of possible paths immensely. In those cases, further analysis, in particular of the dependencies, would be necessary. This leads into an important caveat that must be addressed: we cannot analyze what we cannot see.
Indeed, though it may seem trivial to point it out, it is worth mentioning that we cannot statically analyze a wrapper until we download it, which in and of itself requires memory/network usage. This problem of foresight significantly narrows our planning horizon to only include those wrappers which we currently have downloaded at the time of the next "decision", and forces us to perform what are traditional offline, fully observable optimizations instead online, in a partially observable environment. On this understanding of the problem, the solution begins to look less and less like garbage collection and more akin to the kinds of "planning under uncertainty" that we see in traditional AI modelling and design.
Tracing & RC
While planning ahead using static analysis would be the preferred approach, the computational overhead may prove either undesirable or untenable (or both!) in certain situations. In those situations, it may be preferable to relax our efforts and engage in ad-hoc decision-making by employing more traditional techniques associated with garbage collections. To reason about this, during wrapper execution we might wish to either trace or perform reference counting on the dependencies as they are encountered. At every entry/exit of a (sub-)query, we increment/decrement a counter. When the counter reaches zero, we either immediately evict the offending wrapper, or else relegate it to a naïve cache replacement policy, depending on how conservative the user has set their configuration. Or, in the case of a tracing GC, we can trace the call graph at every entry/exit and again evict appropriately (or relegate, though some discussion of the need for that is had below).
 This overhead can prove negligible on the simple case, but can grow quickly to be intractable as programs and wrappers become more complex and deeply nested.
Dependencies

Garbage collection, at its core, requires some scheduling and computational overhead. As such, garbage collection would not be advisable/possible on systems that are not conducive to handling the multiple concerns (i.e. mutator & collector) of such a system in a reasonable fashion. Furthermore, modern garbage collection algorithms often rely on concurrency to operate efficiently and without imposing too severely upon the user experience; however, concurrency is not currently feasible on every platform (notably, in the browser), which can pose issues for implementation details on certain platforms.
Much of the value associated with static analysis depends upon gathering the most information about a potential execution at the lowest cost. To this end, introspectable source files and hierarchical dependency manifests would go a long way towards helping the cause of static analysis.

2. Cache Replacement Strategies
On the two perspectives in our framework of discussion, we can consider caching to be a degenerate case of garbage collection where nothing is known about the contents of the data being stored. This opacity requires us to treat all data equally and attempt to keep/evict it only on the basis of temporal/frequency patterns. As with the preface to the GC proposal, we should also note here that while the two aforementioned cache replacement policies are good defaults, it would make sense to offer as many other options as are available in the SDK native environment. In general this may mean that there may be some UX-fragmentation across different SDKs, as different policies are available in different environments, or may be of greater/lesser use depending on the memory accessibility thereof; however, this is something that ostensibly should be expected of cross-platform utilities, and is unlikely to severely affect developer ergonomics (although disclaimers are nevertheless important to placate belligerent users).
Implementation
Once a cache replacement policy is specified, the runtime can employ it to maintain a cache of dependencies. Whether any of those dependencies are preloaded, or if they are simply downloaded upon first request, can be determined by the user via configuration — the cache policy is meant to act with a certain naïveté, though one can imagine this to rest on a sliding scale of effort spent in planning, from none whatsoever in the simple cache policy, to complete forward planning in the aforementioned GC section.
Consequences
Cache replacement policies are a tried and tested methodology with known constraints and tradeoffs. The only comparable consequence in this analysis worth mentioning is that they may prove too naïve. If all you have is a hammer, everything begins to look like nails, which may be undesirable in certain environments or scenarios.
Dependencies

Some utility by which to enforce underlying memory handling (e.g. marking references as weak, calling an underlying GC in the SDK native language, manually cleaning up memory, etc.): it's hard to enforce a cache if we can't actually force something to be evicted when we say it was

3. Wizard
While not necessarily a separate concern, the complexity of the solution space discussed above may cause some concern to the average user (indeed, very few casual developers find themselves tuning GCs for performance, or analyzing flame graphs to determine their cache busts). Creating a wizard to help step users through the thought process required to grok the options available before them could prove very helpful. Whether it be a website or a CLI tool (or something else entirely), offering a way for the average user to gain insight into the various knobs and levers of the tuning apparatus can pay dividends in ensuring user viability and in lowering the barrier to entry for having performant programs.
 In general, following the earlier dictum of providing maximum flexibility to the end-user, we should consider all of the dimensions along which garbage collection can make tradeoffs. Three clear dimensions are technique, timing, and deadlines. Providing these dimensions to the end-user will allow us to determine a GC implementation that is best suited to their needs (and, if they so choose, the end-user is welcome to override the SDK's decision and force a particular GC algorithm, specifying its parameters as they best see fit for their particular usecase).
Along those three dimensions, we should consider separating GC algorithms as being either traced or reference-counted, sequential or concurrent, and conservative or liberal, respectively. Choices along these dimensions ostensibly have general defaults that can be assumed, which will inform the selection of a proper GC algorithm for a given situation. For example, if the user specifies that they are not concerned with memory usage, a generational garbage collector with a liberal eviction policy would keep weakly-referenced and phantom-referenced dependencies around for far longer, to the potential benefit of the user that may end up using those in the future.
Implementation
At its core, the wizard provides a questionnaire that steps users through the thought process required to fill out a cache config. Said cache config, shown below, would then be passed to the runtime. The questionnaire would ask the user about their global preferences, their usecase, and the kinds of data/operations that they would expect to see during the execution of their program. The answers to these questions would then inform the wizard in offering certain predetermined "profiles" with messages explaining the choices made (e.g. any conflicting choices, unreasonable expectations, etc.)
interface CacheConfig {
	network?: { // MB
    	    hard: number,
    	    soft?: number,
	},
	heap?: { // MB
    	    hard: number,
    	    soft?: number,
	},
	executionDelay?: { // ms
    	    hard: number,
    	    soft?: number,
	},
	gc?: {
    	    algorithm: string,
    	    options: Record,
	} | null,
	cachePolicy?: string,
	cacheWarmup: boolean,
}

const defaultCacheConfig: CacheConfig = {
	network: {
    	    hard: 10,
    soft: 8,
	},
	heap: {
    	    hard: 2048,
    	    soft: 1024,
	},
	executionDelay: {
    	    hard: 400,
    	    soft: 150,
	},
	gc: null,
	cachePolicy: 'ARC',
	cacheWarmup: false,
}

NB: this data model considers the problem to have a joint solution space between the GC approach and the cache replacement approach.
Consequences
Users who are unsure of exactly which GC or cache replacement algorithm they wish to employ, or who prefer to describe their situation in terms of their business constraints, would be well served by having access to a tool that can lead them through the necessary reasoning to determine the appropriate cache configuration for their usecase. This makes the system more accessible and more performant on the average case of user, as it allows those without the time or requisite knowledge to tune the runtime to meet their performance criteria.
Dependencies

CLI or website or other client-external tool that users can employ to help in their decision-making

Alternative Considered
None :)
Unknowns
¿Por qué no los dos?
Is there any benefit to attempting to employ both garbage collection and advanced cache replacement strategies? Following the epistemic principle that second-order knowledge of the unknown is true, valuable knowledge, it would seem that there may be a point that we find ourselves in where we can say with some certainty what actions to take up to a point, and from that point (until a potential return to clarity) we may instead downshift to naïve caching policies. However, whether this would bring with it too much overhead, or even potentially incur performance penalties over simply 'sticking to our guns' on one of the approaches, is at this time unclear.
Intermediary servers
Is there a need for having some sort of intermediary compilation server, one with fewer constraints than the actual runtime? A server that has many popular dependencies cached could potentially receive the "program" desired by the runtime, and attempt to calculate an appropriate cache strategy / collection policy ahead-of-time, which it can then return to the runtime as a 'plan' of sorts to follow.
References

 [1] Ali, W., & Shamsuddin, S.M. (2015). Intelligent Dynamic Aging Approaches in Web Proxy Cache Replacement. Journal of Intelligent Learning Systems and Applications, 07, 117-127.

 [2] Clark, L. & Ehrenberg, D. (2021). WebAssembly/ECMAScript module integration. In WebAssembly proposals. https://github.com/WebAssembly/esm-integration/blob/main/proposals/esm-integration/README.md

 [3] Jaleel, A., Theobald, K.B., Steely, S.C., & Emer, J.S. (2010). High performance cache replacement using re-reference interval prediction (RRIP). Proceedings of the 37th annual international symposium on Computer architecture.

 [4] Jones, R., Hosking, A., & Moss, E. (2016). The Garbage Collection Handbook: the art of automatic memory management. CRC Press.

 [5] Li, C., Song, M., Du, S., Wang, X., Zhang, M., & Luo, Y. (2020). Adaptive priority-based cache replacement and prediction-based cache prefetching in edge computing environment. Journal of Network and Computer Applications, 165, 102715.

 [6] The Linux man-pages Authors. (2021, March 22). dlclose(3). In Linux Programmer's Manual, (5.13). https://linux.die.net/man/3/dlclose.

 [7] Megiddo, N., & Modha, D. S. (2006). U.S. Patent No. 6,996,676. Washington, DC: U.S. Patent and Trademark Office.

 [8] Megiddo, N., & Modha, D. S. (2003, March). ARC: A Self-Tuning, Low Overhead Replacement Cache. In Fast (Vol. 3, No. 2003, pp. 115-130).

 [9] Nystrom, R. (2021). Garbage Collection. In Crafting Interpreters. https://craftinginterpreters.com/garbage-collection.html.

 [10] Rumpf, A. (2021). Nim's Memory Management. In Nim Programming Language Documentation. https://nim-lang.org/docs/gc.html.

 [11] The Rust Project Developers. (2021). Memory allocation and lifetime. In The Rust Reference. https://doc.rust-lang.org/reference/memory-allocation-and-lifetime.html.

 [12] Shi, Z., Huang, X., Jain, A., & Lin, C. (2019). Applying Deep Learning to the Cache Replacement Problem. Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture.

 [13] Sui, L., Dietrich, J., Emery, M., Rasheed, S., & Tahir, A. (2018, December). On the soundness of call graph construction in the presence of dynamic language features-a benchmark and tool evaluation. In Asian Symposium on Programming Languages and Systems (pp. 69-88). Springer, Cham.

 [14] Vietri, G., Rodriguez, L. V., Martinez, W. A., Lyons, S., Liu, J., Rangaswami, R., ... & Narasimhan, G. (2018). Driving cache replacement with ml-based lecar. In 10th {USENIX} Workshop on Hot Topics in Storage and File Systems (HotStorage 18).

Reposted @pwvpwvpwv's response above here for additional safe keeping: https://forum.polywrap.io/t/research-client-run-time-cache-management/144?u=dorgjelli


Ability to move FileSystem dock to bottom and drag resources across bottom docks

Allows moving the filesystem dock to the bottom
Added ability to drag resources across bottom docks

Animation, moving filesystem to the bottom dock:

Drag and Drop across bottom panels:

Production edit: This partially addresses https://github.com/godotengine/godot-proposals/issues/1508.

The "Make Floating" button should be expanded horizontally now, so its size isn't at odds with the new one.

That is really helpful for those that like the horizontal File System, in my case what i would love to have is the vertical panel but the feature to but collapse it. That would be increidibly helpful to have more space whenever i need it.
My work around is the NV addon just cuz i can collapse the file system to the bottom and hide it presing the tab, but i would love to be able to just collapse without send it to the bottom :)

Moving the dock breaks icons:
https://github.com/godotengine/godot/assets/2223172/7880afd8-3280-4a85-8ce0-e3f6c9740011
The minimum size is very small. This is relevant when your bottom dock is at minimum size. After moving file dock to bottom, it's awkwardly small:

The layout of elements needs some tweaking:

File path and filter do not need to be that big and the icons are misaligned. I'd just put every button/text field in the same row, there is enough space for that. The layout from the side dock does not look that good at the bottom.
Also any reason why Make Floating button does not appear when dock is at the bottom?

Regarding the minimum size if it's only for when it's first time dropepd to the bottom then I think it's ok to make it a bit larger but I wouldn't set absolute minimum size so the dock couldn't be minimized below that since this will often be opened while working with the viewport I can already imagine people just quickly dropping the size down while they work and pulling it up when needed (many people do this in Blender with animaiton and pretty much any separated views it's quick and reasonably comfortable to do so when you need more space). I understand you can make it go all away but that will unnecessarily inconvenient people.

Overal I like this, it makes sense since FS needs to be accessible with pretty much every dock at the same time so this will allow people to streamline their workflow, love it.

One more thing I'd like to see is a hide/show shortcut especially now since this (naturally) reduces the viewport size would it be possible to add a customizable shortcut for people (which as default could even be left empty) to show/hide the FS dock when it's at the bottom panel? Some practical examples for this would be Unreal Engine where I believe it's control + spacebar (or something like that) and it really seems very useful.

The icons at least the red one in the corner for how the FS should appear (list vs icons) if clicked on will get the icons back properly btw.

@KoBeWi

Also any reason why Make Floating button does not appear when dock is at the bottom?

It seems when something floats, it still belongs to one of the side docks, so making something from the bottom dock float would not have any reference to that. It would need to be something separate to make the bottom docks float eventually, hence out of the scope of this PR.

IMO each band-aid like this, while lovely for current users, decreases editor intuitiveness for new users, which in this case would expect to relocate tabs via tab dragging, both between areas and between windows, and the long term goal should be aiming for a flexible tabbed approach. Relevant proposals https://github.com/godotengine/godot-proposals/issues/7233 https://github.com/godotengine/godot-proposals/issues/1508 https://github.com/godotengine/godot-proposals/issues/28

@PLyczkowski To you it may be a band aid, but I look at it different:

For us contributors it keeps the code manageable because it is not a lot. If everything was super flexible as you suggest, it would be considerably more and more complex code.
If this flexibility is not really needed, then it is better to just allow what has specific demand, rather than go for a very flexible solution. This, again, keeps everything manageable. Flexibility has to be justified.

Just to clarify, detaching bottom panels may happen at some point in the future, but there is not nearly as much demand, so it's on low priority.

What about a key bind to toogle on and off the file system dock?

@jcostello I suggest opening a proposal for this that explains how it would work with the proper context. I am not very famliar with that area (how keyboard shortcuts are organized), and you can get better feedback from other people.

Okay I updated this pull request with the issues mentioned above fixed. I also did a bit of an extra effort to allow other docks to be moved to horizontal position and made the system more generic (though they have to explicitly allow this, only the filesystem one does currently).
I am having trouble figuring out why the icons break, because they don't when the dock is moved to other parts. Maybe someone more familiar with this area can lend a hand.

I would prefer if we implemented this after https://github.com/godotengine/godot/pull/84193 is merged. We need to de-bloat EditorNode, not add more code to it.

@YuriSizov That's fair, and the other one looks very nice, what's blocking it?

@reduz Nothing, just wasn't in scope for 4.2 and I didn't get a chance to review it yet. But I will!

I fixed the missing icon issue for you. The theming code in the FS dock was a mess. Also added missing docs, which failed the CI.

@YuriSizov Thanks hugely!

Well I did say the bottom dock was too small, but now it takes half of the screen 🙃

It can't be shrunk past this size.
The icon bug is fixed, and the layout adjustments I mentioned would require some more changes, so they can be done later.

I tested whether dock position and split mode are remembered between editor sessions and it seems to be working mostly correctly. The split mode is remembered for the current position (i.e. side/bottom), but not for the other position. The setting is lost when restarting the editor. It's a minor issue and can be fixed later, just wanted to note it.

Re: making the panel floating - Apparently it's possible to move bottom dock directly to floating dock if you change editor layout. So it's definitely possible to provide this option in the new popup.

I don't know what causes this, but something is wrong with drag and drop on buttons:
https://github.com/godotengine/godot/assets/2223172/bfdd2147-440b-4f75-8756-47d048f21dac

@KoBeWi whoa that is super weird, I can't reproduce it

I can reproduce it every time.
Windows 10
1920x1080
100% display scale
idk what could affect it. Also not sure how relevant is it to this PR; technically it's a bug in the new functionality.

@KoBeWi Nevermind I am blind, I also had the problem but did not notice it.

@KoBeWi Okay, it looks the problem is because the order can change of bottom panel items and the signal is being connected by item index. I changed the code to simply use the pointer, so it seems to work fine now.
I will clean up the rest of the things you mentioned and hopefully this should be ready to go.

Allright, all changes done, please give a final check and feel free to approve if you believe nothing else remains to be done.

Alright, done with the suggested changes.

Rebased after recent changes and also fixed another missing icon bug. Seems like a check was added in https://github.com/godotengine/godot/pull/23273 to avoid segfault on exit, but this should already be covered in 4.x where NOTIFICATION_THEME_CHANGED won't be sent on exit.

Sounds good! I need to give a pass to all the editors in the bottom panel and make sure they can be undocked and redocked at some point, so at some point we can start thinking how to customize their placement too.

Thanks!

@ismailgamedev Commenting on closed PR is unlikely to be seen. This should probably be a separate issue instead


Option to append rasa Mitie model

rasa NLU version (e.g. 0.7.3): 0.7.3
Used backend / pipeline (mitie, spacy_sklearn, ...): 'mitie'
Operating system (windows, osx, ...): Ubuntu 16.04
Issue: I have trained rasa, backend Mitie with 15 intents. Took me around 6-7 hours for the training completion (ran the training with 16 GB RAM system in Ubuntu). I have to add 2-3 additional intents. Is there a way to append to the existing model?

@PrashantKS unfortunately, mitie doesn't allow to resume the training after it has finished. currently there is no other way than to retrain the model. Might be a good idea to try the mitie_sklearn backend, should be faster to train, you should evaluate though if there is a performance hit for your use case.

Thank you tmbo ... I tried sklearn for training ... It's extremely fast compared to Mitie ... And during run time I am using Mitie ... Works well.


Can´t configure mailing list

Hi, i can´t find a way to confiure mailing lists.
I can enable mailing lists, but can´t see the tab to configure them.

Not sure if i missing somthing. Didn´t found anything if docs.
This is my environment
sh: 1: hg: not found
sh: 1: cvs: not found
sh: 1: bzr: not found
Environment:
  Redmine version                4.0.5.stable
  Ruby version                   2.4.0-p0 (2016-12-24) [x86_64-linux]
  Rails version                  5.2.3
  Environment                    production
  Database adapter               Mysql2
  Mailer queue                   ActiveJob::QueueAdapters::AsyncAdapter
  Mailer delivery                sendmail
SCM:
  Subversion                     1.8.8
  Git                            1.9.1
  Filesystem
  Xitolite                       1.9.1
Redmine plugins:
  additionals                    2.0.21
  redmine_agile                  1.5.0
  redmine_git_hosting            4.0.2
  redmine_hourglass              1.1.2+develop
  redmineup_tags                 2.0.8

Thanks ‼

i've just realized that notifications are handled through watchers.
But, looks like add or remove watchers is only available in sidebar, wich is hidden if there is only one repo y the project.


Some sensor names are not the same - not a complaint just logging so you know

Maybe you already know.
Written about it here
but basically eg the daily max temperature anymore is now daytime high temperature ?
I had to rename them from "dark_sky Overnight Low Temperature 2d" to "sensor.dark_sky_daily_low_temperature_2d" etc
THANK YOU for this integration

Thanks for highlighting this, I had noticed this before, and should have documented it. Unfortunately, as much as I strive for Dark Sky compatibility, it's not always possible. The issue here is that Home Assistant has changed a fair bit, and one of the changes relates to the sensor names. Since I'm ultimately trying to get this merged into mainline, I have to follow their guide.
I do have a legacy branch as part of this project, which should be 100% the same as the Dark Sky sensor. You'll lose the integrated updates, units, and some other more recent developments, but the data will all be the same.
I'm going to close this for now to try to keep my issues panel clean, but absolutely feel free to comment or open again if there's anything else that comes up here!


Importing collections : last destination folder improvement when importing several collections in a row

When importing postman collections, one by one, first you need to select a collection to import from somewhere (let's call it A), and then you need to chose where to import it (and where to keep the Bruno collection), possibly elsewhere (B).
The interface remembers where the last opened filesystem destination was (nice!) but when importing several collections in a row, it always opens back on B when you need to select collections from A, and A when you want to select the Bruno collections destination (B), and this can be annoying and tedious.
If it is possible, it would be nicer to keep track of the last A location (import folder) AND separately, the last B location (destination folder).

Electron provides an option to do this.
Need to cache the lastImportPath and reuse it
const { dialog } = require('electron');

dialog.showOpenDialog({
  defaultPath: lastImportPath
}, (filePaths) => {
    lastImportPath = filePaths[0];
  }
});

Will have this done for our next release.


feat(network): protobuf update

Pull Request type
Please check the type of change your PR introduces:

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build-related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

Issue Number: N/A
What is the new behavior?

Does this introduce a breaking change?

[ ] Yes
[ ] No

Other information

This change is

Current dependencies on/for this PR:

main

PR #1130

PR #1232   👈
PR #1173
PR #1152
PR #1147
PR #1131
PR #1123

This comment was auto-generated by Graphite.


Drill-3892 testcase

Abhishek,
I added a testcase for DRILL-3892. Can you kindly review?

Rahul

Looks good. +1

Thanks Abhishek!


[BUG] why in-game doesnt work?

Please provide more information where you are encountering issues and any error messages you see.

I believe they were referring to

Even when I'm in a game and in the match, it's the same thing for me


[UI] The text inside the button while pulling changes does not align vertical

It's a known issue with AvaloniaUI. It's not only affects the button.

Currently, you can change anthor font for Default Font in preference window.
Maybe they will fixed this in future.


Connect wallet modal

Fix modal header text and close button (this should be the same across all modal)
remove mouse hover from header text as it's not interactive
Add divider line in header
Button outline colour is wrong - should be borders colour #DBE1E7
Button text should be 16px semi bold
Vertically Centre button text
button text 24px from left side
buttons have 16px spacing from left and right edges of modal
New to ethereum? text - see design: https://www.figma.com/file/6y4nvjfeVZwzwKcoXB0neq/Simplified-UI?node-id=292%3A2833
Add hover states to buttons: https://www.figma.com/file/6y4nvjfeVZwzwKcoXB0neq/Simplified-UI?node-id=292%3A902

@bconfortin looks good, just a couple of small things
button text to be 24px from left side
Rounded corners of modal should be 16px

@bconfortin looks good, just a couple of small things
button text to be 24px from left side
Rounded corners of modal should be 16px


[WIP] Amp Start Configurator Container

Do Not Pull
Since today is my last day in the office, making this PR, as I started a container to display the live reloading configurator. So this is the code I had written for it, and couldn't get past trying to get URL Params in the iframe, as they are not allowed.
Still needs to be done:
[  ] Write some invalid <script> tags to grab the template from the configurator url.
[  ] Place the configurator URL in the iframe.
[  ] Use the template page.css json to create an amp-list.
[  ] Use amp-bind to bind the template variables to the configurator iframe.
[  ]  Style the inner configurator iframe to be the same width and height as the outer iframe.
Example

Hi @torch2424 - I may clean up this repo a bit. This PR is 2 years old, so I assume you do not plan to work on it further and it is described as a WIP. Should I reject it to get it off the books? Otherwise do you plan to keep working on it to complete it? Just cleaning house! (And I did not fully understand what this was for.)

@alankent Oh yeah! Yeah I'll go ahead and close this, it became very stale 😄
Thanks for reaching out and cleaning up the repo! 👍


Incident Counts differ when using limit param on the incidents endpoint

Using the limit query param with the incidents endpoint produces different results.
In hindsight ...  If so, you want to use --limit 1000, not -P limit=1000 which just passes that as a query param to the underlying HTTPS GET that is used by the fetch command. My bad, because I should not allow you to pass limit as a query param as it will mess up the pagination that fetch is doing. Can you try that and let me know what you observe? Thanks

--limit works as expected
#incidents: 604
#incidents_with_limit: 604

Thanks @BillSidney, I will fix the validation on that flag

Hi @BillSidney, I fixed that param validation in RC 0.1.7 just uploaded; I will probably promote it to stable soon, so if it's ok with you I'll be closing out this issue... thanks again for reporting it


Remove pod/exec privileges from virt-api

What this PR does / why we need it:
This PR mainly removes the exepart that executes sock-connector from virt-api. Alternatively, virt-api connects to a REST endpoint exposed by virt-handler via a websocket and virt-handler proxies that to the corresponding unix socket.
In addition, this PR slightly improves the console connection by reading 10226 bytes rather than reading 10240 and then sending via two messages (one of size 10226 and one of size 14). This adds another read on every 731 reads but may cut half of the amount of send calls.
Release note:

NONE

/cc @davidvossel  @rmohr @SchSeba

Update: I'm struggling with moving socat to the virt-handler container.
Without it, tests are failing:
(1) When trying to to connect multiple times through VNC
(2) When invoking serial console multiple times and assuming the new connection terminates the previous one
So I think about implementing the logic of (2) in console.go for both consoles for now (without calling socat).
@davidvossel thanks for the feedback. Would the above make sense?

Update: I'm struggling with moving socat to the virt-handler container.

Yes please remove socat. Just open the socket directly from virt-handler and copy io.

Update: I'm struggling with moving socat to the virt-handler container.

Yes please remove socat. Just open the socket directly from virt-handler and copy io.

Ack, so are we ok with limiting the VNC console to one connection per-VMI?

Ack, so are we ok with limiting the VNC console to one connection per-VMI?

Sounds good. There is not other option anyway.

(2) When invoking serial console multiple times and assuming the new connection terminates the previous one

:+1:  as Roman said, i actually don't think there's another option except to only allow a single connection.

Ack, so I dropped all use of socat and I'll remove it from the image of virt-launcher as well.

/lgtm

Update: I'm struggling with moving socat to the virt-handler container.

Why do you want to move it to virt-handler? It should stay where it is. Just leave socat and the script in virt-launcher and virt-api can still connect like always. After the next release we can remove it.

Update: I'm struggling with moving socat to the virt-handler container.

Why do you want to move it to virt-handler? It should stay where it is. Just leave socat and the script in virt-launcher and virt-api can still connect like always. After the next release we can remove it.

That's an old comment, irrelevant now

/retest

@rmohr @davidvossel thanks for the feedback. Before addressing the comments, I wonder if you could have some insights on a weird scenario I'm investigating:
The vnc functional tests fail when having multiple vnc connections. The first connection is properly established. The second fails. The reason is that "dialing" the unix socket fails because the socket file doesn't exist. I saw some reports that the underlying file may be removed when the listener is closed  but didn't find anything related to the client being closed. It works when commenting out this line. It is apparently unrelated to this line. I'm going to dig into go/net to see if the underlying file is unlinked anywhere else, but maybe you could save me that time if you faced this before...

@rmohr @davidvossel thanks for the feedback. Before addressing the comments, I wonder if you could have some insights on a weird scenario I'm investigating:
The vnc functional tests fail when having multiple vnc connections. The first connection is properly established. The second fails. The reason is that "dialing" the unix socket fails because the socket file doesn't exist. I saw some reports saying the underlying file may be removed when the listener is closed  but didn't find anything related to the client being closed. It works (the tests pass) when commenting out this line. It is apparently unrelated to this line. I'm going to dig into go/net to see if the underlying file is unlinked anywhere else, but maybe you could save me that time if you faced this before...

Or it may be related to the grpc server handling...

@ahadas I think you are hitting the qemu bug, where qemu (wrongly) deletes the socket after the first connection. @petrkotas solved that by making the qemu socket read-only. Maybe the logic introduced in https://github.com/kubevirt/kubevirt/pull/2171 got lost in the refactoring?

@ahadas I think you are hitting the qemu bug, where qemu (wrongly) deletes the socket after the first connection. @petrkotas solved that by making the qemu socket read-only. Maybe the logic introduced in #2171 got lost in the refactoring?

QEMU also unlinked it.. interesting
Looks like Petr's fix is still there. I think that I'm facing a different issue because without closing the the socket it works - seems it has something do to with the close operation

QEMU also unlinked it.. interesting

It must be QEMU. net.Dial does not unlink sockets. Only net.Listen (server side).

I think that the fix in place is a little bit racy. Now that we open the connections through virt-handler, I would do the following: After the check if the socket is there, and before opening the socket, ensure that the socket directory is read-only.

QEMU also unlinked it.. interesting

It must be QEMU. net.Dial does not unlink sockets. Only net.Listen (server side).

I tried to reproduce it manually in two scenarios:

Connecting to the VM via VNC from one terminal and then connecting from another terminal - the code seems to behave normally, virt-handler close the first connect and makes the second one.
Connecting to the VM via VNC from one terminal, closing that connection and then connect again - it also works.

I think that if it's QEMU fault, then those scenarios would have failed, right? but they seem to work

I think that the fix in place is a little bit racy. Now that we open the connections through virt-handler, I would do the following: After the check if the socket is there, and before opening the socket, ensure that the socket directory is read-only.

Ah you say that the connection is established too quicky? ok, that makes sense

I tested it with a go client and a server, just to be really sure. No sockets got removed when the client closed the connection. The symptoms you describe sound exactly like the qemu bug (note that the solution from petr may leave the directory as writeable for up to one second, so unit tests have plenty of time to break things). The bug is that QEMU deletes the socket if a connection gets closed.

@rmohr IMHO if we have such preparation stuff under the virt-launcher code, we should not move the VMI to the running state until all preparation is done.

I tested it with a go client and a server, just to be really sure. No sockets got removed when the client closed the connection. The symptoms you describe sound exactly like the qemu bug (note that the solution from petr may leave the directory as writeable for up to one second, so unit tests have plenty of time to break things). The bug is that QEMU deletes the socket if a connection gets closed.

Yeah, it didn't manage to reproduce it in a separate client-server application I wrote either.
You seem to be right - adding a sleep of 5 sec before the first connection solves that - I guess that's the time it takes to ensure the directory is changed to read-only.
So why do we set the VMI as running before those preparations are completed?

The fix should just have ensured on every connection, inside the exec script that the directory is read-only.

The fix should just have ensured on every connection, inside the exec script that the directory is read-only.

Petr's fix you mean?

Petr's fix you mean?

Yes.

@rmohr IMHO if we have such preparation stuff under the virt-launcher code, we should not move the VMI to the running state until all preparation is done.

In general yes. We should however be done when libvirt says the the VMI is running (and there the sockets already exist). The only issue is this bug. I think in this case it is by far the simplest solution to just ensure that the directory is read-only (check if read-only, if not setting it read-only) on every connection.

@rmohr ok, kudos on figuring it out :) but how come that without the changes in this PR the multiple-VNC test passes? is it because it takes more time to trigger the exec script? or because of the use of socat?

@rmohr ok, kudos on figuring it out :) but how come that without the changes in this PR the multiple-VNC test passes? is it because it takes more time to trigger the exec script?

I think that the exec script does not kill socat properly. It only kills it when you re-connect, so qemu always saw an open connection and when the second connection attempt happened, the directory was already read-only. I am very glad that we get rid of this bash script soon.

The fix was only meant as a temporary workaround and is designed in a way it does not block on IO too much.
Moreover, the QEMU 4 does not suffer from these issues and deems the fix deprecated. It is already released upstream. For downstream we are still waiting.
Once the QEMU is upgraded, these issues will go away.

I think that the exec script does not kill socat properly. It only kills it when you re-connect, so qemu always saw an open connection and when the second connection attempt happened, the directory was already read-only. I am very glad that we get rid of this bash script soon.

yep, this is a pretty reasonable explanation.
The "read-only" check before virt-handler connects solves the issue in that the socket won't disappear, but we'll still have flaky functional tests due to the timing of when read-only is set.
Like @cynepco3hahue pointed out, can we ensure read-only is set before marking the VMI as Phase: Running?

Like @cynepco3hahue pointed out, can we ensure read-only is set before marking the VMI as Phase: Running?

As mentioned before, the read-only fix which we had in place did its work asynchronous. There is not need to wait. The socket already exists when the VMI is reported by libvirt as running.

The "read-only" check before virt-handler connects solves the issue in that the socket won't disappear, but we'll still have flaky functional tests due to the timing of when read-only is set.

The "check" will ensure that it is set. It does not just check and return with an error if it is not-yet read-only.

Like @cynepco3hahue pointed out, can we ensure read-only is set before marking the VMI as Phase: Running?

As mentioned before, the read-only fix which we had in place did its work asynchronous (and left a time window open to let something like this happen). There is not need to wait. The socket already exists when the VMI is reported by libvirt as running.

Already coded but in light of the last part I wonder if it's needed to check whether the socket exists - we can try to Chmod the parent directory, and fail if it doesn't exist, and if the socket itself doesn't exist then the Dial operation will fail, no? (@rmohr)

@ahadas the prometheus metrics tests have issues (Probably because of TLS verification?). Apart from that everything looks good. To be honest I did not think about the fact that we need to allow insecure TLS connections for prometheus right now ...
So back to two servers I guess. Sorry for that.

@ahadas the prometheus metrics tests have issues (Probably because of TLS verification?). Apart from that everything looks good. To be honest I did not think about the fact that we need to allow insecure TLS connections for prometheus right now ...
So back to two servers I guess. Sorry for that.

Sure, I thought it's debatable so I implemented it in an independent commit - I can just drop it and modify the one that makes the port configurable at deployment phase to inject the port to virt-handler (and not only to virt-api). On it

/retest

interestingly, make client-python started to fail although it passes locally...

interestingly, make client-python started to fail although it passes locally...

Let me retrigger travis.

/retest

/retest

WOW - So many green lanes.


Screen resolution - again! See comments.

Max screen needs to NOT maintain the initial aspect ratio. Doggedly maintaining aspect ratio stops true full screen.

Works now all of a sudden. Closing.


bind: factor out RecursiveUnmount

Factor out logic for recursively unmounting our bind mounts, and make use of the parent/child relationship between mountpoints to decide which order to use, rather than sorting based on their mountpoint paths.

Seems rather brutal.  But LGTM

Rebased.

LGTM, thanks for the funtion name change to keep my OCD at bay!  :^)

@rh-atomic-bot r+


Remove scalar/scalar comparisons

Scalar == scalar is unlike vector == scalar and vector == vector in that it doesn't filter. To keep the language orthogonal and consistent, I propose we drop support for scalar/scalar comparisons. Any case where it would be useful should be doable via vectors.

Another option would be to make it filter.

How would it filter? By returning an empty vector? What purpose would that serve? (but removing sounds good so far)

It'd do the same as vector/vector, return the left argument.
Hmm, if we do that we'd want to remove support for scalar/vector and only allow vector/scalar.
I'm thinking removing is the way to go.

A use case has appeared for filtering scalar/scalar comparisons: restricting times when alerts fire via math on time(). Accordingly let's filter and take the left argument.

As complains about breaking changes are getting louder with time, we might want to make a decision on this in the next few weeks.

@brian-brazil I'm still not clear on how scalar/scalar would filter: if the return type is scalar as well, what would the return value be for something like 5 < 3? NaN?

Hmm, yeah you'd need to make the result of scalar/scalar comparisons to be vectors for it to work - which would break other things.
That'd imply dropping support for it, and using absent({}) or maybe a one/iota/unity() function that returns a vector with a single no-label sample with value 1 for when you need to filter.

Devil's advocate - would it make sense to drop scalars all together and parse a number literal as a single-element vector without labels?
Except for arguments in functions such as topk.
We'd have to allow unary expressions for vectors. Label matching simply wouldn't make sense as the label set is empty.

Devil's advocate - would it make sense to drop scalars all together and parse a number literal as a single-element vector without labels?

That'd require group_left/group_right on "scalar"/vector operations, which'd be too overbearing considering how common that is.

Another thought is that a scalar/scalar comparison could return a vector, either empty or {}:left value

To summarize, we've two options to clean this up:

Drop support for non-filtering scalar comparisons
Filter, and return a no-label vector with the value

I'm tending towards the first, as that'll leave us the option of doing the second in future.
I'm thinking this should be done the release after #1049 is released, to give users a chance to transition.

0.16 will include bool, so we can drop scalar/scalar in 0.17.


Embed styles (etc) breaking if nested in a table

e.g.
Here is a w2c page with the issue from the Week 1 module. The issue occurs across all the modules.

Actually a CAR problem


Agree on data structures for Gawel's problem case

The case Gawel's is currently working on requires defining problems and geometries in ways different than the cases for suppercompressibles and hyperelastic.
We should discuss and agree on how to handle such cases and what changes to DoeVars.py are required.

Based on our discussion on 3-November. We pass data to simulator.abaqus as a pickled file. We still need to agree on the content and structure of such file


db caching error "mysqli_num_fields() expects parameter 1 to be mysqli_result, boolean given"

mysqli_num_fields() expects parameter 1 to be mysqli_result, boolean given in /wp-includes/wp-db.php on line 3013
it seems to have occurred right after upgrading to WordPress 4.5.
If you turn W3TC db caching off, the problem disappears, but no more caching.

I noticed Wordpress dev stopped using the "@" error suppressor for mysqli_num_fields() in wp-db.php starting at 4.5.  So it's very possible this error msg was occurring prior to 4.5 but was being suppressed.
So this db cache fix might simply need a condition check or @ in the corresponding w3tc file (Db.php? DbCache.php?) that is indirectly calling the load_col_info() function of wp-db.php (the function that is throwing  this error message).
From a quick cursory glance it might be w3tc calling the __get() overload.  When i have time  I'll have a closer look.

you're right...

I decided to have a look at this bug and for the life of me i cannot recreate it on my end.  I get no error warnings and w3tc's Database Caching works perfectly for me (its enabled).  What is a sure way for me to see this bug?
For my test system i currently have:

WP 4.6 installed (fresh install)
PHP 7.0.9 active
Latest fix-w3tc build
Enabled WP_DEBUG via wp-config.php (have it redirecting to /wp-content/debug.log instead of on-screen -- nothing is ever written out to it)
w3tc is enabled and Database Caching (set to: "Disk") is enabled.  Note: I didn't enable anything else; everything is at their default value.

Based on the that setup i see no php warnings/errors of any kind and database caching is working -- a "db" directory is generated and several cached sql/result files are made (and being made).
Thanks for the assist,
Kimberly

I also investigated but could not find anything wrong in chache db.

I can also confirm there are no errors with this. Can we look at suppressing the error again?

@Volnus, I disagree with simply suppressing a error message.
I perfer to add error_log before the command, and in case mysqli_result is boolean then dump a bunch of debug information to help with the mater for the next time. Maybe the problem is a simple change between PHP version when in old version you will get a boolean, but in the newer version you get an object.

I think i'll refocus back on this old elusive issue...if it is even an issue (the db cache still works fine).  Last i checked i was not even able to get any warning or error message of any kind.    I'll put back the wordpress <IP_ADDRESS> on a live server  and compare it against fix-w3tc to see if the error message only occurs for that older version.  This will hopefully put to rest if this issue had indirectly been resolved long ago (so no warning messages either) with all our updates to fix-w3tc.  Will be fun to get back into this issue.
Kimberly

I have already investigate, try tk replicate without success. For me is not a real bug.

Just as an unrelated follow up. I get this issue with WP-CLI errors and WP-Migrate-DB on a cron-sync. I have modified wp-db and dump the var to logs before the call that creates the warning. So far, it outputs objects with arrays inside. PHP 5.6 here, apache 2.4 on CentOS.

@mkormendy thanks for the info!

No problem.
I was able to capture cases where, for whatever reason, the mysqli_query() call would produce a boolean true value, and in some cases a boolean false value.
To fix it on my end (while this isn't a suggested permanent solution) I ended up modifying the wp-db.php file in the wp-includes folder to accommodate when the query returns a boolean:
/**
 * Internal function to perform the mysql_query() call.
 *
 * @since 3.9.0
 *
 * @access private
 * @see wpdb::query()
 *
 * @param string $query The query to run.
 */
private function _do_query( $query ) {
	if ( defined( 'SAVEQUERIES' ) && SAVEQUERIES ) {
		$this->timer_start();
	}
	if ( ! empty( $this->dbh ) && $this->use_mysqli ) {
		$this->result = mysqli_query( $this->dbh, $query );
	} elseif ( ! empty( $this->dbh ) ) {
		$this->result = mysql_query( $query, $this->dbh );
	}
	$this->num_queries++;
	if ( defined( 'SAVEQUERIES' ) && SAVEQUERIES ) {
		$this->queries[] = array( $query, $this->timer_stop(), $this->get_caller() );
	}
}

Hi, any news on this one? Getting the issue only when not logged-in. Using WP 4.8 latest and W3 Total Cache up to date too.
The issue disappears if I uncheck then check again the database caching in general settings, but sometimes reappears for no apparent reason.

Hey @nicooprat:

Are you using the 0.9.5.x branch of w3tc?
What cache storage are you using? (File-based, memcached, redis, ...)?
What other cache types do you have active (object cache?)
Plugins? Themes?

Thanks for your quick answer.

Have to admit that I didn't see at the moment that I was on the repo of a fork. So no, I'm using the latest official release.
Disk: enhanced
Page cache + minify + database cache + object cache (disk)
Custom theme (based on Roots Sage 9) + complete list of extensions

Next time the issue appears I'll try to replace W3TC by this community fix!

No problem!
This particular issue is one of those mysteries that have went unsolved, and somehow disappeared at some point.
If you do end up having this issue again in the future, feel free to post about it and I'd look into it :)

Just had this issue again. Tried to switch Page cache from advanced to basic and it disappeared. I wanted to try every setting one by one but I guess it just regenerated the cache by changing this setting, so it might not be related. Pretty frustrating!

I've the same error, wordpress 4.8 multisite

Thanks for the info @nicooprat . I will try to replicate on my side.
@Derevko : which w3tc version are you using, and what cache storage engine?

Got this error in the middle of the night.
E_WARNING: mysqli_num_fields() expects parameter 1 to be mysqli_result, null given
Stack trace
…at /html/wp-includes/
wp-db.php (3091)
…at /html/wp-includes/
wp-db.php (682)
… html/wp-content/plugins/w3-total-cache/
DbCache_WpdbInjection_QueryCaching.php (161)
….html/wp-content/plugins/w3-total-cache/
DbCache_Wpdb.php (152)
…at /html/wp-includes/
wp-db.php (2280)
… /html/wp-includes/
comment.php (634)
… /html/wp-includes/
comment.php (1943)
… /html/wp-includes/
comment.php (3095)
… /html/
wp-comments-post.php (25)
WP 4.8.1
Not multisite
PHP 7.0
MySql 5.6
Memcached database in W3 Total Cache

We have the same issue.
Wordpress v4.8.1
W3CTC v<IP_ADDRESS>
PHP v5.6
The warning appears only if Database cache is enabled (and method set to disk, I didn´t tested any other methods). And only if user is not logged in.

Same here...
Wordpress v4.9.1
W3CTC v0.9.6
PHP v7.1
The warning appears only if Database cache is enabled (and method set to disk, I didn´t tested any other methods). And only if user is not logged in.

Same issue here:
WordPress 4.9.6
W3TC 0.9.7
Database cache is enabled
The warning appears for non-logged Users and when logged in, it disappears.

Me as well. Suddenly it started appearing...

I'm getting the following errors randomly
[21-Aug-2018 14:49:31 UTC] PHP Warning:  mysqli_query(): (HY000/126): Incorrect key file for table '/tmp/#sql_2736_5.MYI'; try to repair it in /home/xxxx/public_html/yyyy/wp-includes/wp-db.php on line 1924
[21-Aug-2018 14:49:31 UTC] Incorrect key file for table '/tmp/#sql_2736_5.MYI'; try to repair it
[21-Aug-2018 14:49:31 UTC] PHP Warning:  mysqli_num_fields() expects parameter 1 to be mysqli_result, boolean given in /home/xxxx/public_html/yyyy/wp-includes/wp-db.php on line 3182

Hi,
Got the same problem "mysqli_num_fields() expects parameter 1 to be mysqli_result, boolean given in" EveryWHERE.
appear when DB cache "ON"
WP 4.9.8
W3TC Version 0.9.7
hope you can find any solution
thanks!

found the problem. If you are using w3tc you need to pay attention to the use of lazyloading. Lazyload gives errors.


Bug: Arbitrary "Kernel died" on jupyter

What happened?
When I am trying to infer the embeddings for universal sentence encoder, arbitrary "The kernel appears to have died" dialog appears, and the runtime is destroyed. Generally, we infer embeddings for, say 1M unigram to tri-gram phrases, and we have provisioned big machines for that.
Software:
tf_hub 0.12
tensorflow 2.1.0
python 3.6
Compute:
Amazon Ec2, g3.8xlarge,
AMI Deep learning Linux 2 Instance
244 Gb RAM
8 GB GPU
One thing to note is that, tf doesn't detect GPU, so it might be a contributing factor, but this behavior is quite arbitrary, which is bothering the teams. Some insights into this would be great.
Relevant code
import tensorflow as tf
import tensorflow_hub as hub
embed = hub.load("https://tfhub.dev/google/universal-sentence-encoder/4")
print('Successfully imported tensorflow hub')

'''
we have three lists, a,b,c, containing the phrases.
the task is to find the closest out of b and c to a, so we infer embeddings two at a time and calculate dot product
'''
def get_dot_product(list1, list2):
    em1 = embed(l1) ######## CRASH POINT - FAILS WHEN PASSED THE SAME LIST ARBITRARILY
    ##### STOPPAGE POINT
    em2 = embed(l2)
    sim1 = np.sum(em1*em2,axis=1,keepdims=False)
    return sim1

### ...

Relevant log output
The Kernel appears to have died. It will restart automatically

tensorflow_hub Version
0.12.0 (latest stable release)
TensorFlow Version
2.1.0
Python Version
3.6
OS
Linux centOS (AMazon AMI)

Hey @captSteveRogers,
Can you please share the error you are getting in terminal while the kernel dies and restarts? This will help us in debugging the issue.
Also, you can try explicitly cast the inputs as a string as shown here.
And you can go through this article to see if it resolves you issue.

Closing due to inactivity.


Catching an error caused by a faulty config/settings file

It would be extremely useful if there was a way to trap an error arising from a misconfigured settings file e.g. the parser requires user but the file has the setting username.
I would like to be able to do this:
try:
    parser.parse_args()
except Exception as err:
    logger.error('Unable to load settings:%s' % err)

At present, the script just "dies":
usage: arg.py [-h] [-s SETTINGS] [-n NAME] [-p PORT] [-o HOST] [-u USER]
arg.py: error: unrecognized arguments: --username derek

the current behavior is based on the behavior of argparse for unexpected args.
You might want to use parse_known_args() instead of parse_args()


time: NewTicker will not emit ticks at a frequency greater than 1/sec on qemu user mode ppc64le

What version of Go are you using (go version)?
Found on go version 1.12.15 and 1.13.6 but not 1.11.13.
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
ppc64le (run via Qemu user mode emulation in a docker container on an AMD64 machine provided by travis-ci.org).

Ran the following test:
func TestTickerPerformance(t *testing.T) {
	testTicker(t, time.Microsecond*100)
	testTicker(t, time.Microsecond*200)
	testTicker(t, time.Microsecond*500)
	testTicker(t, time.Millisecond)
	testTicker(t, time.Millisecond*10)
	testTicker(t, time.Millisecond*100)
	testTicker(t, time.Millisecond*1000)
}

func testTicker(t *testing.T, d time.Duration) {
	count := 0

	ctx, cancel := context.WithTimeout(context.Background(), time.Second*4)
	defer cancel()

	ticker := time.NewTicker(d)
	defer ticker.Stop()

loop:
	for {
		select {
		case <-ticker.C:
			count++
		case <-ctx.Done():
			break loop
		}
	}

	t.Logf("%v ticker emitted %v ticks/sec", d, float64(count)/4.0)
}

Available here also as a playground link (code slightly modified to run from main() instead of as a test): https://play.golang.org/p/-zk7GR62pse (please note this produces the 'expected' output on playground rather than the broken output obtained from the ppc64le emulator).
What did you expect to see?
The playground output:
100µs ticker emitted 10000 ticks/sec
200µs ticker emitted 5000 ticks/sec
500µs ticker emitted 2000 ticks/sec
1ms ticker emitted 1000 ticks/sec
10ms ticker emitted 100 ticks/sec
100ms ticker emitted 10 ticks/sec
1s ticker emitted 1 ticks/sec

Or performance figures close to it (noting that this test isn't perfect and there are good reasons why the 100µs ticker may be observed as emitting somewhat fewer than 10000 ticks/sec).
What did you see instead?
On go 1.12.x on ppc64le (see this travis CI run, scroll down to "Running test suite under ppc64le", then to "TestTickerPerformance", line ~1077): https://travis-ci.org/godbus/dbus/jobs/637768665)
100µs ticker emitted 1 ticks/sec
200µs ticker emitted 1 ticks/sec
500µs ticker emitted 1 ticks/sec
1ms ticker emitted 1 ticks/sec
10ms ticker emitted 1 ticks/sec
100ms ticker emitted 1 ticks/sec
1s ticker emitted 0.75 ticks/sec

This go 1.13.x run has the same result: https://travis-ci.org/godbus/dbus/jobs/637768666
The go 1.11.x run does not have this issue:
100µs ticker emitted 9702.75 ticks/sec
200µs ticker emitted 4995.25 ticks/sec
500µs ticker emitted 1999 ticks/sec
1ms ticker emitted 1000 ticks/sec
10ms ticker emitted 100 ticks/sec
100ms ticker emitted 10 ticks/sec
1s ticker emitted 0.75 ticks/sec

This suggests to me that this may be a regression in golang and not due to a flawed emulator.
Comments
I wanted to independently verify this issue on an actual PowerPC device (rather only test on an emulator), but this is difficult for me to come across, so I have not done this.

Doesn't this CL need to be backported?

@gopherbot please consider this for backport to 1.13, it's a regression.
/cc @ianlancetaylor


CRL - Verification Failed

Hello. I am unable to move past the Captcha authentication. I have verified the URLs I am trying to use are valid and have tried several different links. The only switches I use are "p" and "l" for user credentials. I complete the Captcha but it seems like the cookie(s) are not being saved as It asks to authenticate every time. Any help would be welcomed.
C:\Users\ME\Desktop\FAKKU Downloader\fakku-downloader-master>python main.py -l myusernameishere -p mypassword
Cookies file(cookies.pickle) are not detected. Please, login in next step for generate cookie for next runs.

DevTools listening on ws://<IP_ADDRESS>:58086/devtools/browser/0e0f0ad4-f89e-42e3-a350-2fdfb74041a6
[27952:28036:0714/063155.195:ERROR:device_event_log_impl.cc(214)] [06:31:55.195] USB: usb_device_handle_win.cc:1058 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)
[27952:28036:0714/063155.203:ERROR:device_event_log_impl.cc(214)] [06:31:55.203] Bluetooth: bluetooth_adapter_winrt.cc:1072 Getting Default Adapter failed.
Traceback (most recent call last):
  File "C:\Users\Jake\Desktop\FAKKU Downloader\fakku-downloader-master\main.py", line 131, in <module>
    main()
  File "C:\Users\Jake\Desktop\FAKKU Downloader\fakku-downloader-master\main.py", line 120, in main
    loader.init_browser(headless=False)
  File "C:\Users\Jake\Desktop\FAKKU Downloader\fakku-downloader-master\downloader.py", line 158, in init_browser
    self.__auth()
  File "C:\Users\Jake\Desktop\FAKKU Downloader\fakku-downloader-master\downloader.py", line 190, in __auth
    self.browser.find_element_by_class_name('js-submit2').click()
  File "C:\Users\Jake\AppData\Roaming\Python\Python39\site-packages\selenium\webdriver\remote\webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\Users\Jake\AppData\Roaming\Python\Python39\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "C:\Users\Jake\AppData\Roaming\Python\Python39\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Users\Jake\AppData\Roaming\Python\Python39\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
  (Session info: chrome=91.0.4472.124)

[27952:26504:0714/063156.844:ERROR:cast_crl.cc(384)] CRL - Verification failed.

The element name changed again. Edit line 190 of downloader.py and change it from this:
self.browser.find_element_by_class_name('js-submit2').click()
To this:
self.browser.find_element_by_class_name('js-submit-login').click()
I just did this a few minutes ago and restarted the login process, and was able to get it working again.


Fix: better story ssg

Better story SSG
This PR addresses some issues related to SSG, specifically returning a true 404 (see notFound here) instead of statically generating an error page and returning a fallback page the first time a user hits a story which was not built at build time through getStaticPaths(). I tried to keep the styling of the fallback page coherent with the rest of the app and with the 404 page.
Let me know what you think!
Proposal: we could extend this PR to #126 and think about a way for statically generating and paginating the high number of stories that we have on the website.
This PR is related and closes #141.

I'm glad to see the removal of the custom error page / 404 handling, but I'm not sure about the fallback switch from 'blocking' to 'true'. blocking is certainly a bit slower on the first load of an unrendered / uncached page, but it eliminates all the fallback handling logic and components. So the complexity trade-off doesn't feel worth it. What do you think?

In my opinion it's a fair "small complexity increase for better UX" trade, because it only adds few lines of code and gives faster feedback to the user. I also think that fallback: 'blocking' might be the reason behind the issue I was having in #126.

Alright yeah, I'm down to give it a try. Can you clean up the conflicts here so we can get a test deploy? I'll test by manually adding a new entry to the staging DB and we'll see how it handles.

Sure, let me know what you think after you've tried it @mwickett!


Population data in PIP are not in millions

HI Samuel,
Do we need it in millions? Is it ok if pop is in million using the option  povclnet_format, so that if the user types pip, povcalnet_format clear population will be in millions, but if they only type pip, clear then it will be in the current format?
Best,
@tsamuel321 , @tonyfujs , @Aeilert, @Tefera19

The population data are still not in millions, but I don't think anyone now wants us to change it to millions.


Documentation Request: Debugging with V8 Inspector

I'm submitting a feature request
Webpack version:
1.10.x/2.x
Current behavior:
We don't have any documentation on how to use the new node-nightly --inspect --debug-brk feature with webpack. There are a few little details to implement so it is worth having documentation on it.
Expected/desired behavior:
Having a small section in documentation explaining how to leverage the power of node-nightly which contains the --inspect flag letting you use the full chrome dev tools to profile, find memory leaks, and debug your webpack builds.

What is the motivation / use case for changing the behavior?
There are small leaky abstractions from miss use of certain webpack features. Mine being one of them. A useful profiler that people are familiar with like ChromeDevTools can be invaluable

see blog post

@bebraw we should try and move this issue over if we can. Then we can have atleast a tracker for it.

Done, https://github.com/webpack/webpack.io/issues/90 .


Question: is context used when translating with a neural engine?

Hello:
I'm performing several experiments training phrase based and neural engines with the same corpora and then translating a test set with a without context. Then I evaluate the results using NIST and BLEU. When using phrase based engines, giving a context to the segment to be translated slightly improves the results. But when using a neural engine, I get the same evaluation results without and with context. Is this normal? Is context used in neural engines?
Thank you very much again
Antoni

Hi @aoliverg !
this is expected for small-medium sized engines because there are some deep differences on how the context vector is used in phrase-based and neural engines.
In phrase-based engine if no-context is passed, the engine won't adapt at all, while in neural this is not the case: neural engines always adapt to data stored in the internal memory.
So for neural engines the context-vector is used to weight different candidates to choose from for adaptation. In small-medium engines there is little chances to find two candidates with very similar source-part and different target-part; so there are few chances that the context-vector will be crucial in order to decide which option to choose between two similar suggestions coming from two different memories.
For example:
[candidate 1] source="Ciao" target="Hi" memory=1
[candidate 2] source="Ciao" target="Hello" memory=2

The score of each candidate is 0.5 * similarity(query, source) + 0.5 * context_vector(memory), so if the sentence to translate is "Ciao" and no context is provided:
[candidate 1] score = 0.5 * 1 + 0.5 * 0 = 0.5
[candidate 2] score = 0.5 * 1 + 0.5 * 0 = 0.5

the engine will randomly pick one of the two options, but if we pass a context vector 2:1.0:
[candidate 1] score = 0.5 * 1 + 0.5 * 0 = 0.5
[candidate 2] score = 0.5 * 1 + 0.5 * 1 = 1

the engine will pick always the second candidate because, even if it has the same source, it has a better context score.


Fixed getBaseQuery select to call the correct absolute path (not relative)

As per my understanding this needs to be an absolute path in this case to make it callable.

hi @mitelg
I use the command /opt/remi/php70/root/bin/php bin/sw plugin:create --haveBackend --backendModel=RmTestModel --haveApi RmTest
First to ensure that i use php 7.0, execute bin/sw from the sw-cli-tools folder and create the plugin with backend, model and api

@mitelg I created an issue for plugin creation bug a while ago: #108 😉

thanks for your PR @ray-magini :+1: :blue_heart:
@larsbo I will have a look :blush: