SSL passthrough for Ingress container

the current ingress container cannot do L4 routing. I dont want to terminate my ssl at the ingress container because my ssl config is complicated (with lua rules and all) and i would rather not port them over to annotations.
I see that there is a readme for proxy protocol as well plans to use Caddy as the ingress container.
Is there a long term plan to add SSL pass through for the ingress container ?

Is there a long term plan to add SSL pass through for the ingress container ?

@sandys yes, please check https://github.com/kubernetes/kubernetes/issues/24145
That being said features like "SSL pass through" must be supported by the software that provides the reverse proxy (for instance this is not possible with nginx)

That being said features like "SSL pass through" must be supported by the software that provides the reverse proxy (for instance this is not possible with nginx)

@aledbf NGINX Ingress controller (nginx-ingress-controller:0.8.3) seems to be running nginx/1.11.3. I believe this would be possible with nginx/1.11.5 which has the ngx_stream_ssl_preread_module?

@Sushant yes and no. It is possible IF you use the new stream feature but if you do that means TCP only because can't do L7 balancing without terminating TLS (ie all the Ingress with TLS termination in the controller won't work).
Check this issue https://github.com/nginxinc/kubernetes-ingress/issues/57#issuecomment-255208477 (already ask if is possible to use conditional routing for SSL pass through)

@aledbf
Right. For a moment, I thought I could get away with having two instances of the Nginx ingress controller on the same cluster (1 with and 1 w/o SSL passthrough). But then I would need something like this: https://github.com/kubernetes/contrib/pull/1880 to support both types of apps.

hi manuel
thanks! btw, the nginxinc bug was also filed by me ;)
do you think we can get nginx 1.5 as the ingress controller and see if tcp
pass through is working? if you check the k8s slack channels... there are a
lot of people who want to do this (we dont want to move our ssl termination
to ingress... because that would mean porting over our configs).
if all i can do is use ingress as a dumb pipe that passes ssl (and source
ip preservation) to my nginx pods... ill be a very happy man!
On Oct 21, 2016 02:32, "Manuel Alejandro de Brito Fontes" <
<EMAIL_ADDRESS>wrote:

@Sushant https://github.com/Sushant yes and no. It is possible IF you
use the new stream feature but if you do that means TCP only because can't
do L7 balancing without terminating TLS (ie all the Ingress with TLS
termination in the controller won't work).
Check this issue nginxinc/kubernetes-ingress#57 (comment)
https://github.com/nginxinc/kubernetes-ingress/issues/57#issuecomment-255208477
(already ask if is possible to use conditional routing for SSL pass through)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/contrib/issues/1854#issuecomment-255227209,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEsU1Pdc9liDh-n9mattM0YiVNoWibdks5q19bTgaJpZM4KRtn-
.

there's a proposal open for tcp ingress, and a bug open for multiple tls modes. We should combine those two to give us what we want. Please comment on  https://github.com/kubernetes/kubernetes/issues/32928 and https://github.com/kubernetes/kubernetes/pull/25821 to make sure it does what this bug requires, and re-open if we can't leverage those features for passthrough

@bprashanth im commenting here - because i dont want to sound stupid and pollute one of the other bugs. in 25821 - the load balancers that everyone is talking about is cloud based load balancers. for example the readme mentions all the rackspace, vsphere, etc.... but i dont see haproxy/nginx anywhere.
is that going to be added to the bug ? because clearly this bug (and all the people in here) are talking about ingress with a K8s loadbalancer - nginx or haproxy or one of the alternatives

you can use any controller behind an ingress. If we surface the behavior in the ingress, it doesn't matter what fulfills it. We will still need N bugs for "implement ssl passthrough in [backend of choice] ingress" , but typically that's after "surface ssl passthrough in ingress".

@bprashanth sorry to bother you again, but I also am following https://github.com/nginxinc/kubernetes-ingress/issues/57
Do you need to "surface" anything if we are talking nginx or haproxy ingress ? It may true for the different cloud based ingresses/load balancers. I'm trying to figure out whether bug 25821 is blocking the implementation of all these various features (L4, SSL passthrough) for nginx or haproxy ingresses.
For example, nginx version need to be upgraded in the ingress controller to take advantage of  ngx_stream_ssl_preread_module, but im not sure if that will be done until 25821 is done.. which in turn is blocking for the superset of all the various cloud services.

@sandys this is already working. Please wait while the new version is published in a new repository in the kubernetes organization.
This is how it works:

create an Ingress with the annotation ingress.kubernetes.io/ssl-passthrough: "true"

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/ssl-passthrough: "true"
  name: kubernetes
  namespace: default
spec:
  rules:
  - host: kubernetes.foo-bar.com
    http:
      paths:
      - backend:
          serviceName: kubernetes
          servicePort: 443
  tls:
  - hosts:
    - kubernetes.foo-bar.com

that's it :)
That annotation generates this code in the configuration file:
stream {
    # map FQDN that requires SSL passthrough
    map $ssl_preread_server_name $stream_upstream {
        foo-898.bar.com         default-kubernetes-443-0-pt;
        foo-997.bar.com         default-echoheaders-x-80-1-pt;
        kubernetes.foo-bar.com         default-kubernetes-443-2-pt;
        # send SSL traffic to this nginx in a different port
        default                         nginx-ssl-backend;
    }

    log_format log_stream '$remote_addr [$time_local] $protocol [$ssl_preread_server_name] [$stream_upstream] '
        '$status $bytes_sent $bytes_received $session_time';

    access_log /var/log/nginx/access.log log_stream;
    error_log  /var/log/nginx/error.log;

    # configure default backend for SSL
    upstream nginx-ssl-backend {
        server <IP_ADDRESS>:442;
    }

    upstream default-kubernetes-443-0-pt {
        server <IP_ADDRESS>:443;
    }
    upstream default-echoheaders-x-80-1-pt {
        server <IP_ADDRESS>:8080;
    }
    upstream default-kubernetes-443-2-pt {
        server <IP_ADDRESS>:443;
    }

    server {
        listen                  443;
        proxy_pass              $stream_upstream;
        ssl_preread             on;
    }

# TCP services
# UDP services

}
```

> For example, nginx version need to be upgraded

Is a more complicated than that but this is the PR that I will open to [update nginx](https://github.com/kubernetes/contrib/compare/master...aledbf:enable-stream?expand=1)

[Here's](https://github.com/aledbf/ingress-controller/blob/master/backends/nginx/rootfs/etc/nginx/template/nginx.tmpl) the new template

@aledbf you are a lifesaver !
just to double check because we have been blocked on this

does this need a special version of kubernetes ?
i assume the public DNS of kubernetes.foo-bar.com is pointing to the host with the ingress controller ?
can i do both port 80 and port 443 ?
the way it works for us (atleast in the past with nginx) is that we redirect http -> https. so port 80 gets rewritten to port 443. In this particular case, since the ingress controller will pass it through to the nginx ... will the rewrite still work ?

thanks! this is awesome. Just in time for Diwali ..

does this need a special version of kubernetes ?

No

i assume the public DNS of kubernetes.foo-bar.com is pointing to the host with the ingress controller ?

Yes (or just use your /etc/hosts file :) )

can i do both port 80 and port 443 ?
the way it works for us (atleast in the past with nginx) is that we redirect http -> https. so port 80 gets rewritten to port 443. In this particular case, since the ingress controller will pass it through to the nginx ... will the rewrite still work ?

Yes but you need to add a redirect annotation (here's the syntax
Why is this annotation required? I cannot decide for the user what behavior is the "correct" one so if you don't add the annotation the port 80 will not be redirected automatically.
Writing something like:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/ssl-passthrough: "true"
  name: kubernetes
  namespace: default
spec:
  backend:
    serviceName: kubernetes
    servicePort: 443
  tls:
  - hosts:
    - kubernetes.foo-bar.com

means traffic in port 80 for host kubernetes.foo-bar.com redirect to 443

@aledbf why do i need to annotate it at all ?
can i not let the traffic go through the port 80 of ingress controller to my nginx pod and then get a 301 redirect ?
I dont necessarily want to do this at the ingress controller level. I actually have many, many redirects in my nginx pod.. and i was wondering if I can simply use my ingress controller to pass traffic to my actual nginx pods.
will that work ? that's the configuration im looking for.

can i not let the traffic go through the port 80 of ingress controller to my nginx pod and then get a 301 redirect ?

I need to test that. Let me see if it works

thanks! really appreciate it!

@aledbf did this get merged into the k8s ingress repo yet? I didn't see a PR from your repository, and the config seems similar to yours, but not sure if it's the same.

@paultiplady yes. Here's the code

@aledbf your link is 404

Hello Is there an examp;e of all config infrastructure for ingress controller set up as daemon without external lb and deployments with pods that will be terminate ssl.
Ingress contoller must don't terminate ssl pnly pass it to pods on other worker nodes


test: adding native image configuration required for tests

Trying to fix "Kokoro - Test: Java GraalVM Native Image" build.
Failures (12):
  JUnit Vintage:ITSystemTest:detectLabelsTest
    MethodSource [className = 'com.google.cloud.vision.it.ITSystemTest', methodName = 'detectLabelsTest', methodParameterTypes = '']
    => java.io.FileNotFoundException: src/test/resources/wakeupcat.jpg (No such file or directory)
       com.oracle.svm.jni.JNIJavaCallWrappers.jniInvoke_VA_LIST:Ljava_io_FileNotFoundException_2_0002e_0003cinit_0003e_00028Ljava_lang_String_2Ljava_lang_String_2_00029V(JNIJavaCallWrappers.java:0)
       java.io.FileInputStream.open0(FileInputStream.java)
       java.io.FileInputStream.open(FileInputStream.java:219)
       java.io.FileInputStream.<init>(FileInputStream.java:157)
       java.io.FileInputStream.<init>(FileInputStream.java:112)
       com.google.cloud.vision.it.ITSystemTest.getResponsesList(ITSystemTest.java:223)
       com.google.cloud.vision.it.ITSystemTest.detectLabelsTest(ITSystemTest.java:263)
       java.lang.reflect.Method.invoke(Method.java:566)
       org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
       org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
       [...]
...

https://source.cloud.google.com/results/invocations/cc541c2e-5da6-48bf-acb1-dd0d90f340b3/targets/cloud-devrel%2Fclient-libraries%2Fjava%2Fjava-vision%2Fpresubmit%2Fgraalvm-native/log
The test execution requires some configuration for native image to load resource files.

{"pattern":"*.png"} failed:
Fatal error:java.util.regex.PatternSyntaxException: Dangling meta character '*' near index 0
*.png
^

{"pattern":".*.jpg"}, didn't make any effect.

{"pattern":"\\Qcity.jpg\\E"}, didn't work.
https://source.cloud.google.com/results/invocations/18c9ab4e-2225-4531-b777-146d0e61a1c4/targets/cloud-devrel%2Fclient-libraries%2Fjava%2Fjava-vision%2Fpresubmit%2Fgraalvm-native/log

It's from
      ByteString imgBytes = ByteString.readFrom(new FileInputStream(RESOURCES + image));

where RESOURCES is "src/test/resources/" and image is "wakeupcat.jpg" for example. It seems the problem is the working directory.

I think this is a limitation of using resource configs with Graal: https://github.com/mpeddada1/native-resources-config/blob/main/child-module/src/test/java/com/example/ResourceExampleTest.java. FileInputStream/FileSystems doesn't recognize the resources-config.json. What happens if we reference the resource using the ClassLoader?

I recognize this is the problem of working directory https://github.com/googleapis/java-vision/pull/856#issuecomment-1079195775. JVM execution and GraalVM native-image execution show different output for working directory.
I think the integration test is intended to use file as input, rather than resources, for documentation purpose on how to use this library.


BL Scripts Crash (0.14)

The new BL crash when I enable scripts

My mod doesn't work either on it, I'm trying to find the culprit function, but no luck so far.
It crashes on world launch and just force closes. I turned the crash log on, it saves them but they are empty.

@Black-DeathPE @zhuowei for me it does that when u activate ModPE Scripts

@officialmcpro32 @zhuowei It must be function that you run straight away and I run on new level

@officialmcpro32 Try again with the latest beta: it fixes at least two crashes at startup

As for me, it stopped crashing when 0.14.0 full update released

@zhuowei close this issue it works fine now


3d_show not working

I am attempting to get the 3d_show example to work.
however, when I attempt to connect on port 9998, the connection is reset. I see that there is a file
"3d_show.html" in the project that contains the html and javascript that presumably should be sent by the server but I cannot find any references in the code where this file is accessed. Is there something missing here?

The problem is becoming clearer. I was mistaken when I thought that there should be a way for the example to serve the '3d_show.html' file. All you have to do is to load it in your browser. Doing that, I was faced with a 'reference error: webSocket not defined'. I corrected this by adding a script to load requirejs.
<script src="https://requirejs.org/docs/release/2.3.6/r.js"></script>
and
const webSocket = require('ws');
Now,  the websocket handshake logic works and webSocket gets created and I can begin to debug why the MPU9250 logic I  added is not yet working. My changes to the project, so far,  can be reviewed at: here

The MPU9250 logic is working but it's not getting to the browser. problem seems to be these errors:
Error: Mismatched anonymous define() module: function(l){function oa(){}function C(a,b){this.x=a||0;this.y=b||0}function ea(a,b,c,d,e,f,g,h,k,m){Object.defineProperty(this,"id",{value:Oe++});this.uuid=Q.generateUUID();this.name="";this.image=void 0!==a?a:ea.DEFAULT_IMAGE;this.mipmaps=[];this.mapping=void 0!==b?b:ea.DEFAULT_MAPPING;this.wrapS=void 0!==c?c:1001;this.wrapT= void 0!==d?d:1001;this.magFilter=void 0!==e?e:1006;this.minFilter=void 0!==f?f:1008;this.anisotropy=void 0!==k?k:1;this.format=void 0!==g?g:1023;this.type=void 0!==h?h:1009;this.offset=new C(0,0);this.repeat=new C(1,1);this.generateMipmaps=!0;this.premultiplyAlpha=!1;this.flipY=!0;this.unpackAlignment=4;this.encoding=void 0!==m?m:3E3;this.version=0;this.onUpdate=null}function ga(a,b,c,d){this.x=a||0;this.y=b||0;this.z=c||0;this.w=void 0!==d?d:1}function Db(a,b,c){this.uuid=Q.generateUUID();this.width= a;this.height=b;this.scissor=new ga(0,0,a,b);this.scissorTest=!1;this.viewport=new ga(0,0,a,b);c=c||{};void 0===c.minFi… r.js:417:17     makeError https://requirejs.org/docs/release/2.3.6/r.js:417     intakeDefines https://requirejs.org/docs/release/2.3.6/r.js:1503     localRequire https://requirejs.org/docs/release/2.3.6/r.js:1695     requirejs https://requirejs.org/docs/release/2.3.6/r.js:2046     <anonymous> file:///home/allen/Downloads/esp32_snow/example/3d_show/3d_show.html:22

Hi，allenck. first of all. you must put these html file into sd card, the simple webserver lode file from SD card and use chunked way to send data to broswer. my test environment is Chorme. websocket and three.js is necessary.

That worked. I had not considered that option because it is a lot of
trouble to make changes to the html because of the problem of updating
the sd card.
On 5/11/19 8:23 PM, frankie wrote:

Hi，allenck. first of all. you must put these html file into sd card,
the simple webserver lode file from SD card and use chunked way to
send data to broswer. my test environment is Chorme. websocket and
three.js is necessary.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/whyengineer/esp32_snow/issues/3#issuecomment-491553912,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYZORJBQQWX2IPDDRSEJQ3PU5PPDANCNFSM4HMFNHYQ.


Are Pinebook Pro DTBs up to date in EFI mode ?

Hello,
I just moved my Pinebook Pro's Manjaro from legacy syslinux-like boot to EFI boot.
I works well, but I've noticed that my system has become very notceably slower when booted if EFI mode compared to when booted in legacy mode - using the same kernel version.
I've also noticed that the devices (eMMC and SD card) are not numbered in the same order.
So I was wondering if the hardware description and settings provided by tow-boot EFI are the same as those provided by DTBs when using legacy boot ans if there are up-to-date ?
Seeing that tow-boot is apparently based upon an u-boot release that is already one year old.
Is there some update planned ?
Should I consider normal that the system would be slower when booted in UEFI mode compared to legacy boot ?
Thanks in advance.
Best regards.

A quick search in the u-boot repository, reveals that the Pinebook Pro DTS found in the 2021.10 release of u-boot, which is what the latest release of tow-boot is based on, was last synced with mainline linux 5.14-rc1 on August 12th 2021.
See https://github.com/Tow-Boot/Tow-Boot/pull/178 for an planned update to 2022.07 in the works. This will bring in a DTS synced with 5.18 in June 2022.

I try syncing newer and "more exciting" platforms on top of the usual syncs by upstream U-Boot.
For the Pinebook Pro I had it synced to 20 Feb 2022. Annoyingly I didn't note the Linux commit.

https://github.com/Tow-Boot/Tow-Boot/blob/2f8fff7be70ebfd22098ee6a351b0075f63955fc/boards/pine64-pinebookPro/0001-arch-arm-dts-Sync-rk3399-pinebook-pro-with-Linux.patch

The Pinebook Pro, though is thankfully less "exciting" these days. Or in other words, we're seeing basically no functionality changes. AFAIK there should be functionally no difference with the current state as of right now. The three tipmost current commits are mostly cleanup.

Well I tried both and so far the “legacy” boot gives correct performance (I mean what you would expect given the machine's specs) and the EFI one really, really crawls.
But I would be very interested in your own independant impressions on the matter.


Pytest warning

With the recent pytest version (5.4.1) there is a warning now:
src/schemathesis/extra/pytest_plugin.py:46: PytestDeprecationWarning: direct construction of SchemathesisFunction has been deprecated, please use SchemathesisFunction.from_parent
I think it should be easy to resolve

+1
I am getting this also. It produce some noise in the test output. I'd appreciate if it is resolved.

The fix is released in 1.0.2 https://pypi.org/project/schemathesis/1.0.2/

There is one place, where I forgot to put a fix. I'll add it and will make a release today

Resolved in 1.0.5!


the effect on cardview show behind it

when i add it as parent and try it it just show the effect in the border of the card view behind it not the card view itself

Add a RelativeLayout into CardView  then MaterialRippleLayout.on(RelativeLayout)

hii I had the same problem. have you find solution ?

Hii @Abdelsattar I found solution . in xml file set this attribute card_view:mrl_rippleOverlay="true"  . It will do proper effect .

@Abdelsattar all part of parent has ripple , we want to set ripple just for cardview with it's radius

@Abdelsattar it's working perfect for me. In adapter item layout card view  is used . so this is worked for me. Have you also added this property ?  card_view:mrl_rippleInAdapter="true"   if set then in adapter it's effect will work. Hope this Help.

@Abdelsattar RecyclerView is not an AdapterView

i have an idea , extend MaterialRippleLayout to CardView instead of FrameLayout
example :
public class MaterialRippleLayout extends CardView {
but it has a problem , left a dot color in your item during scroll !

i have other idea too 😆
Edit this line (777) of MaterialRippleLayout    My mean ---> public MaterialRippleLayout create() {
edit it to public CardView create() {
then
remove these lines (805 ~ 811) ---> my mean 👇
layout.addView(child, new ViewGroup.LayoutParams(MATCH_PARENT, MATCH_PARENT));

            if (parent != null) {
                parent.addView(layout, index, params);
            }

            return layout;

then , paste these code 👇
 CardView cardView = new CardView(context);
            cardView.addView(layout);
            layout.addView(child, new ViewGroup.LayoutParams(MATCH_PARENT, MATCH_PARENT));

            if (parent != null) {
                parent.addView(cardView, index, params);
            }

            return cardView;

now in ViewHolder in Adapter
   public static class ViewHolder extends RecyclerView.ViewHolder {
        CardView cardView;
        View yourView;

        public ViewHolder(View itemView) {
            super(itemView);
            cardView = (CardView) itemView;
            MaterialRippleLayout rippleLayout = (MaterialRippleLayout) cardView.getChildAt(0);
            yourView = (ViewGroup) rippleLayout.getChildAt(0);
        }
    }


make clear that user must reload browser to see avatar update

I go to Settings. In "Your Account" I choose "Customize avatar". I upload an image from my local machine. I click "Save changes". I get confirmation that the update has been saved ("Updated settings!"). But: if I view Home (or a stream or something) and then go back to Settings, the old avatar appears. And if I send a new message, I see the old avatar (I presume because it's cached).
If I hit Reload, then the new avatar displays (in Settings and in the new messages I've sent since changing the avatar). But it was confusing for a bit, and it would be great if we could force a cache empty of that particular resource after the avatar is changed.
(I'm running Firefox 47.0. I don't think I have any particularly aggressive caching settings locally. The issue is slightly different in the Brave browser: if I change my avatar and send a new message, then the new message does use the new avatar, but if I go to Settings, I still see the old avatar. Again, a plain refresh of the Settings page updates the image.)

@brainwane this is essentially a duplicate of #229, but I think maybe we should turn this issue into updating the message from Update settings! in this case to add `You will need to reload to see your changes take effect", like we do for the display settings like language that we don't support dynamically swapping out.

Agreed @timabbott. (I searched for avatar issues before filing this -- must have missed #229, whoops.)

Fix merged, thanks @brainwane for the report (And @brockwhittaker for the fix)!


Fix syntax error in README

Coverage remained the same at 100.0% when pulling 1b9c89fa33c5b00e67a1d0426bb89ad2391fc4e2 on GeorgeTaveras1231:patch-1 into 24128bb96129b74ad8a11b17924158f1372d3661 on evcohen:master.

ah good catch, thanks!


ARXIVNG-279 Implemented indexing agent for populating test index

I refactored populate_test_metadata to use search.agent.consumer.MetadataRecordProcessor, which now has everything except the Kinesis glue. This should be sufficient to populate the test index for beta deploy, once we've stabilized the search document.

Coverage increased (+1.2%) to 69.686% when pulling 1f2f13c66498836a200ba0c3145c93d91aee1d8c on task/ARXIVNG-279 into b9e0f8e48353c3ccc3dee70206b35e9a8a86a90a on develop.

Coverage increased (+1.2%) to 69.686% when pulling 1f2f13c66498836a200ba0c3145c93d91aee1d8c on task/ARXIVNG-279 into b9e0f8e48353c3ccc3dee70206b35e9a8a86a90a on develop.


Triangulated surface constraint using geograd and multiple DVGeos per DVCon

Purpose
This PR adds the KS-based triangulated surface constraint from the Brelje et al AIAAJ paper
Type of change
What types of change is it?
Select the appropriate type(s) that describe this PR

[ ] Bugfix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (non-backwards-compatible fix or feature)
[ ] Code style update (formatting, renaming)
[x] Refactoring (no functional changes, no API changes)
[ ] Documentation update
[ ] Maintenance update
[ ] Other (please describe)

Testing
Reg tests with good coverage have been added. All previous DVConstraints tests pass after the refactor.
Checklist
Put an x in the boxes that apply.

[ ] I have run flake8 and black to make sure the code adheres to PEP-8 and is consistently formatted
[x] I have run unit and regression tests which pass locally with my changes
[x] I have added new tests that prove my fix is effective or that my feature works
[x] I have added necessary documentation

Don't merge it just yet. I updated geograd with the ability to pass comm from Python layer so I need to update DVConstraints.py real quick

This looks good in general. One discussion; you are currently allowing multiple DVGeos kicking around with a single DVCon. I was doing this in a slightly different way with DVGeoMulti, where I had an upper-level dvgeo object, that kept track of all of the individual DVgeos for each component. that way we can also define intersections etc. I think we can come up with a common way to handle multiple dvgeos and use a consistent approach.
Here is what I recommend: I understand you don't want to make changes at this point. I also do not want to work on geometry stuff. BUT, I think we can come up with a much better scheme if we sit down together and do a mini hackathon with you. I am guessing we can take care of this work within a few hours on a call.
What do you say @bbrelje? If you don't want to make changes, that is totally fine by me and the PR is good as far as I am concerned.

This looks good in general. One discussion; you are currently allowing multiple DVGeos kicking around with a single DVCon. I was doing this in a slightly different way with DVGeoMulti, where I had an upper-level dvgeo object, that kept track of all of the individual DVgeos for each component. that way we can also define intersections etc. I think we can come up with a common way to handle multiple dvgeos and use a consistent approach.
Here is what I recommend: I understand you don't want to make changes at this point. I also do not want to work on geometry stuff. BUT, I think we can come up with a much better scheme if we sit down together and do a mini hackathon with you. I am guessing we can take care of this work within a few hours on a call.
What do you say @bbrelje? If you don't want to make changes, that is totally fine by me and the PR is good as far as I am concerned.

I talked to ben and we decided to merge this PR now, and we will refactor these stuff for harmony when we merge the dvgeomulti code. We estimate to do this in April, when ben will be done running the cases for his thesis work. I will create an issue about this and also move my dvgeomulti code to my public fork because the paper is accepted and in press now.

I talked to ben and we decided to merge this PR now, and we will refactor these stuff for harmony when we merge the dvgeomulti code. We estimate to do this in April, when ben will be done running the cases for his thesis work. I will create an issue about this and also move my dvgeomulti code to my public fork because the paper is accepted and in press now.

I have created the issue that details the refactoring in #57.

I have created the issue that details the refactoring in #57.

I forgot I never included the triangulated volume constraint when I ported from the tensorflow-based geograd to the fortran one. I will need to deprecate that function or add it to geograd this weekend (probably the former). Block merge for now.

I forgot I never included the triangulated volume constraint when I ported from the tensorflow-based geograd to the fortran one. I will need to deprecate that function or add it to geograd this weekend (probably the former). Block merge for now.

@bbrelje I converted the PR to a draft. I understand you want to work on this more? I did the draft thing to prevent somebody accidentally merging this since both reviewers approved.

@bbrelje I converted the PR to a draft. I understand you want to work on this more? I did the draft thing to prevent somebody accidentally merging this since both reviewers approved.

geograd tests failed previously, this has now been fixed. I've verified locally that all tests pass, so I'll merge manually.


daemon: Add wildcard support to --devices ("eth+")

This adds support for device name wildcards to the "--devices"
option, e.g. if user specifies "--devices=eth+,en+" the agent
will match devices eth0, eth1, en0, etc.
Fixes: #15693

test-me-please

test-me-please

test-me-please

test-net-next

test-gke

test-runtime

test-1.20-4.19

test-1.21-4.9

test-net-next

test-gke

test-runtime

test-1.21-4.9

test-1.20-4.19


docs: state-compatibility documentation

Closes: #XXX
What is the purpose of the change
Add documentation for state-compatibility safety, patch and minor releases.
Brief Changelog
This change is a trivial rework / code cleanup without any test coverage.
Documentation and Release Note

Does this pull request introduce a new feature or user-facing behavior changes? no
Is a relevant changelog entry added to the Unreleased section in CHANGELOG.md? no
How is the feature or change documented? not applicable

Todo: mention historical queries that can be pruned

Pretty sure I covered all suggestions in the updates. Please let me know what you all think


[Improve](inverted index) improve match performance without index

Proposed changes
Issue Number: close #xxx
improve performance for match without index, from 2 min 34.54 sec to 9.16s, on httplogs dataset.
mysql> select COUNT() from wc_httplogs_without_index where request match "GET";                                                                  +-----------+
| count(*)  |
+-----------+
| 245552656 |
+-----------+
1 row in set (29.29 sec)

mysql> select COUNT() from wc_httplogs where request match "GET";
+-----------+
| count(*)  |
+-----------+
| 245552656 |
+-----------+
1 row in set (0.51 sec)

mysql> select COUNT() from wc_httplogs_without_index where request match "GET";
+-----------+
| count(*)  |
+-----------+
| 245552656 |
+-----------+
1 row in set (2 min 34.54 sec)

mysql> select COUNT() from wc_httplogs_without_index where request match "GET";                                                                 +-----------+
| count(*)  |
+-----------+
| 245552656 |
+-----------+
1 row in set (9.16 sec)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall

run buildall

run buildall


Grafana & Prometheus board for Lighthouse

Problem
A clear and concise description of what the problem is
Ideas to solve this
A clear and concise description of what you think could solve this problem.

Number of transfers ( forcedSlow, Normal)
Avg time for each domain
frequency of each domain

Acceptance Criteria

[ ] X happens
[ ] Y happens

Other

out of scope


Geolocation can not watch the position.

Geolocation can not watch the position. It only updates when you go to homescreen and reopen the app. To recreate this problem, just use the example on React Native docs:
http://facebook.github.io/react-native/releases/0.31/docs/geolocation.html#watchposition
componentDidMount(){
  this.watchID = navigator.geolocation.watchPosition((position) => {
      var lastPosition = JSON.stringify(position);
      this.setState({lastPosition});
    });
}

Using React Native version 0.31.0 & 0.30.0 on iOS.

I just tested with the latest RN on the iOS simulator, and it works fine in this case. (I haven't tried on a real device)
Are you still having this issue?

Closing for now. Please re-open if this problem is still here!

@ericvicenti
I am experiencing this issue with:
react 15.4.2
react-native 0.41.1
xcode 8.2.1
os sierra
ios 10.2

It only loads the first location on boot, but then does not register any change in location when using Debug -> Location. Any help at all would be greatly appreciated.

yes I'm facing same issue @jaredtibs

I'm having the same issue as well. @ericvicenti
It works fine on emulator, stops updating on real device. Do you guys have a guess about it?

The same issue is happening to me. It only refreshes when you refresh the app. The location does not refresh real time. See this expo snack and test with a real device. https://snack.expo.io/SJb5ZGMd-

I'm using a real android device and watchPosition is not event getting invoked.
"react": "16.0.0-alpha.12",
"react-native": "0.45.1",

same here, should we open a new issue? seems like a regression? @ericvicenti

Little update here. watchPosition inside of componentDidMount does get invoked.
After about 5~10 mins after the component gets mounted.

Remove enableHighAccuracy: true option and try it. It works for me.

@ckim16 what if we need that option enabled?

testing in the ios simulator even with this disabled doesn't works... 😫

@edencakir I'm sorry I really do not know what to do. I just wanted to make that watchPosition part work in my android device and it worked after I removed that enableHighAccuracy option. For now it will stay in that way and hopefully someone will insight us with the better answer.
@leonardodino That's really weird and I'm sorry to hear that. My ios, both simulator and device, has no trouble using watchPosition. Hopefully you have found your answer by now!

Any update on this?  I'm having the same issues.

Same here. It only calls when the app opens. So suspending it (going to home) and opening the app again also triggers the listeners. But it doesn't update at all otherwise.

I have the same issue, watchPosition() is not updating. In my project, it gets called 2 or 3 times after my screen is mounted and then stops.

Having the same problem with watchposition running the snack posted above by @nos-joel.  But if I run watchposition in a simple PhoneGap app, it works fine, with updates once per second while I'm driving.  Really frustrating.  I wanted to use React Native for the UI benefits, but I can't make it work.

I have the same problem which @briangainer has. @ericvicenti I think this issue should be open.

Someone please provide a solution. Watchposition invokes only once inside ComponentDidMount () when homepage/screen starts


Bottom Overflowed By x Pixels

Hi all,
I've created a list of suggestions I get from a query, but I get bottom overflowed.
How to avoid this?

Hi @pitanni  Could you provide a sample code? Hard to imagine where it's coming from.

Sorry, my mistake.
The error was due to another library.
Thanks @nelstein


refactor: cleanup re-exports for easier scripting

This should make work on #750 easier and there is no reason to not re-export like this because it comes down to the same anyway

[!IMPORTANT]
Rebase & Merge

all of these except the name change of baseVariants will be resolved by #750 anyway, so i would recommend rescoping this to just the name change. other than that, world's easiest lgtm


Questions about the model

Thank you for sharing your work. I have two questions.
First, I found a undefined class as 'BAM' and undefined type as 'BAM'  in  the line 118-121 at the \models\cbam\model_resnet.py. Do you mean 'CBAM' here?
Second, when I tried to train one-class-ood on ResidualNet with depth of 18 and 34, which used the BasicBlock defined in your code, the model run fine. However when using ResidualNet with depth of 50 and 101, based on the Bottleneck, the model seemed to have some problem. The size of "out" and "residual" in  the line 96 at \models\cbam\model_resnet.py  is not the same shape. Here is the Traceback:
Traceback (most recent call last):
7   File "train.py", line 224, in
8     train()
9   File "train.py", line 164, in train
10     x = net(2 * data - 1)
11   File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call
12     result = self.forward(*input, **kwargs)
13   File "/data/workplace/ss-ood-master/models/cbam/model_resnet.py", line 169, in forward
14     x = self.layer1(x)
15   File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call
16     result = self.forward(*input, **kwargs)
17   File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
18     input = module(input)
19   File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call
20     result = self.forward(*input, **kwargs)
21   File "/data/workplace/ss-ood-master/models/cbam/model_resnet.py", line 96, in forward
22     out += residual
23 RuntimeError: The size of tensor a (256) must match the size of tensor b (64) at non-singleton dimension 1

I used CBAM code from https://github.com/Jongchan/attention-module/tree/master/MODELS
I think the code in the repo might need to be
class Bottleneck(nn.Module):
    expansion = 4

for larger ResNets to work. Let me know if that works.


[FEATURE REQUEST] Listing original filenames in list-view / Downloading to original filename

Is your feature request related to a problem? Please describe.
I'm running a chibisafe instance for multiple users and it's sometimes very time consuming trying to find a specific file. For example, in list view, clicking on all 30 something files with the .zip extension until you find what you are looking for. Same goes for downloading uploaded files. If you don't rename them and organize them actively, you will quickly lose track of which file is which.
Describe the solution you'd like
It would be nice to have a column for the original filenames in the list view for uploads and to have a toggle in the settings to be able to download files as their original filenames. The latter could be achieved by setting the filename in the Content-Disposition header.
Additional context
For the filenames it could look something like this:

Thank you for bringing this up.
We have to think more about being able to let users download files with the original filename as it can be sensitive data so it has to be limited to the owner, but then again it would be nice to be able to share a file with the original filename too. So we'll see how to tackle this soon.

would love to see original filename download happening! if possible could it be a per-upload setting? like a toggle, on by default, on the upload page, or smth?


hi  in android it can not run

my environment:
"react": "16.6.3",
"react-native": "0.58.5",
"react-native-navigation": "1.1.484",
"react-redux": "^5.0.6",
"redux": "^3.7.2",
"redux-actions": "^2.2.1",
"redux-thunk": "^2.2.0",
"react-native-linear-gradient": "^2.5.3",
"react-native-image-zoom-viewer": "^2.2.25",
"eventemitter2": "^5.0.1",
"immutable": "^3.8.2",
"moment": "^2.20.1",
"react-native-root-toast": "^3.0.0",
"react-native-datepicker": "^1.7.2",
"react-native-fileupload": "^1.2.0",
"react-native-image-picker": "^0.26.7",
"react-native-keyboard-spacer": "^0.4.1",
"react-native-zss-rich-text-editor": "^1.1.0",
"react-native-webview-bridge": "^0.40.1"
and my setting:
include ':app', ':react-native-webview-bridge'
project(':react-native-webview-bridge').projectDir = new File(rootProject.projectDir, '../node_modules/react-native-webview-bridge/android')
import com.github.alinz.reactnativewebviewbridge.WebViewBridgePackage;
protected List<ReactPackage> getPackages() {
	// Add additional packages you require here
	// No need to add RnnPackage and MainReactPackage
	return Arrays.<ReactPackage>asList(
    new LinearGradientPackage(),
    new ImagePickerPackage(),

new WebViewBridgePackage()
);
}
compile project(':react-native-webview-bridge')

and when id react-native run-android
it error :

but when i dose not use react-native-webview-bridge it can run

it is rn version problem


stucked frequently

Mutate is very good because of sample and extendable. However my pc was stucked frequently by mutate (cpu usage > 200%). OS version is ubuntu 14.04

Hi Godfreyhe.  What version are you using, and is it a particular script that seems to do this?  I noticed for myself I had the same problem and it was related to scripts I had created not working correctly.

I'm also seeing the same issue on my system. Screenshots below.
http://i.imgur.com/cLOzHq6.png
http://i.imgur.com/tBWZe0X.png


Update vboxrun.sh to specify "Other_64" as ostype

IncludeOs now builds 64-bit images, so ostype needs updating (#1613)

Can one of the admins verify this patch with one of the following commands:

"jenkins test please" for a one time test run
"ok to test" to accept latest and future commits on this pull request for testing
"add to whitelist" to add the author to the whitelist


STORM-1292

@kamleshbhatt can you add the full JIRA title to the PR . Also squash your commits.

STORM-1292: port backtype.storm.messaging-test to java

@kamleshbhatt please take a look at other prs and edit the title of the PR to add the full JIRA id and title. Also squash your commits.


Error while bootstrapping data directory

Hi there,
when deploying a cluster, I receive this error all the time and cannot sucessfully create one.
Cloudnative-PG is installed using the current helm chart:
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: postgres

helmCharts:
  - name: cloudnative-pg
    repo: https://cloudnative-pg.github.io/charts
    version: 1.18.1
    releaseName: cnpg
    namespace: postgres

Full log (cluster-example-1-initdb-):
Defaulted container "initdb" out of: initdb, bootstrap-controller (init)
{"level":"info","ts":1675094246.6801445,"msg":"Creating new data directory","logging_pod":"cluster-example-1","pgdata":"/var/lib/postgresql/data/pgdata","initDbOptions":["--username","postgres","-D","/var/lib/postgresql/data/pgdata","--encoding=UTF8","--lc-collate=C","--lc-ctype=C"]}
{"level":"info","ts":1675094247.1110559,"logger":"initdb","msg":"The files belonging to this database system will be owned by user \"postgres\".
This user must also own the server process.

The database cluster will be initialized with this locale configuration:
  provider:    libc
  LC_COLLATE:  C
  LC_CTYPE:    C
  LC_MESSAGES: en_US.utf8
  LC_MONETARY: en_US.utf8
  LC_NUMERIC:  en_US.utf8
  LC_TIME:     en_US.utf8
The default text search configuration will be set to \"english\".

Data page checksums are disabled.

creating directory /var/lib/postgresql/data/pgdata ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 20
selecting default shared_buffers ... 400kB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ","pipe":"stdout","logging_pod":"cluster-example-1"}
{"level":"info","ts":1675094247.1112525,"logger":"initdb","msg":"Bus error
child process exited with exit code 135
initdb: removing data directory \"/var/lib/postgresql/data/pgdata\"
","pipe":"stderr","logging_pod":"cluster-example-1"}
{"level":"error","ts":1675094247.1113946,"msg":"Error while bootstrapping data directory","logging_pod":"cluster-example-1","error":"error while creating the PostgreSQL instance: exit status 1","stacktrace":"github.com/cloudnative-pg/cloudnative-pg/pkg/management/log.(*logger).Error
	pkg/management/log/log.go:127
github.com/cloudnative-pg/cloudnative-pg/pkg/management/log.Error
	pkg/management/log/log.go:165
github.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/instance/initdb.initSubCommand
	internal/cmd/manager/instance/initdb/cmd.go:146
github.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/instance/initdb.NewCmd.func2
	internal/cmd/manager/instance/initdb/cmd.go:103
github.com/spf13/cobra.(*Command).execute
	pkg/mod/github.com/spf13/cobra@v1.6.1/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
	pkg/mod/github.com/spf13/cobra@v1.6.1/command.go:1044
github.com/spf13/cobra.(*Command).Execute
	pkg/mod/github.com/spf13/cobra@v1.6.1/command.go:968
main.main
	cmd/manager/main.go:64
runtime.main
	/opt/hostedtoolcache/go/1.19.4/x64/src/runtime/proc.go:250"}
Error: error while creating the PostgreSQL instance: exit status 1

The cluster:
---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: cluster-example
spec:
  instances: 3
  primaryUpdateStrategy: unsupervised

  storage:
    size: 1Gi
    storageClass: "mayastor-2"

Storage Class:
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: mayastor-2
parameters:
  repl: '2'
  protocol: 'nvmf'
  ioTimeout: '60'
  local: 'true'
provisioner: io.openebs.csi-mayastor
volumeBindingMode: WaitForFirstConsumer

Thank you and best regards

Why are you using replicas at the Postgres level in conjunction with replicas at the storage level?
In any case, I suggest you check with OpenEBS community for more appropriate help on this matter, as we are storage agnostic.

@gbartolini  As far as I understood, CNPG only provides HA on the application level - if the storage fails, I would have needed to replace it manually and install a backup (or setup a failover).
Nevertheless, I switched from Mayastor to a NFS Share (Democratic-CSI) and still have the same issue - so I do not think it is storage related, if the error occurs with two different providers.

Turned out that this issue came from enabled hugepages - I created a PR to document the necessary steps.

Closing since it's solved


Provide a textual based terminal user interface for the command centre

A completely unnecessary but cool request but following from #13 working so nicely and @textualize offering cool libraries like trogon and textual I thought would't it be nice to provide an additional terminal based interface which isn't exactly a CLI but not a traditional GUI.
Moreover this will extend out Python skills on the CLI and allow us to value add the library.
The why of this request really is around bringing richer CLI tools for the Gallagher ecosystem.
Note that this request and the CLI should be additional poetry installables. The API client should be a product of its own.

See also textual-plotext


Multiple metrics in training

💡 Your Question
Hi, Is there a way to assign multiple metrics in training for a network with multiple output branches?
I changed the segmentation network to include a regression head, and I want to assign metrics such that the segmentation head uses IOU and the regression head uses MeanSquaredError.
(The single output trainer currently uses "IOU" alone.)
Thank you!
Versions
No response

As of now there is no easy way to achieve this.


Extract User#manual_records to DRY up controller methods

The logic for scoping the manual records to be used by the various repositories was repeated in each of the three controllers.
I paired with @chrislo on this change which extracts the duplicate logic into User#manual_records. While User#manual_records may not be the best long-term home for this logic, we think it's definitely an improvement. On a similar note, it's not ideal that we're using the PermissionChecker inside User#manual, but we think it'll do for now.
One benefit of this change is that we've been able to write a unit test for the extracted logic.

@chrislo: Thanks. I've just re-worked this so the new method is called User#manual_records instead of User#manuals, because I think the former is clearer. I've force-pushed my changes. Also I opened #937 to remind us to aim to use Pundit in the future.

Rebasing against master and force-pushing in preparation for merging.


Searches that return a large data set cause out of memory.

A search term that return a very large amount of parts cause an out of memory error. This may be unfixable without a server upgrade or implementing pagination (this solution may not work either).

Pagination implemented


Report Message or the Report Phishing add-ins for shared mailboxes in EXO.

Hello,
I'm reaching out for some specifications and details about how it's supposed to work between Report Message or the Report Phishing add-ins and shared mailboxes, please help me with this one and if possible to update the article as well so it's becomes clearer for the public:
The article is explicitly mentioning: "Both add-ins are not available for shared mailboxes."

As per my testing I can report junk or phish messages using OWA with the feature "Open another mailbox"

But nothing happens when I try using the add-ins (the add-ins are there displayed):

Few days ago the add-ins moved the message to junk folder only but didn't report it.

When accessing the shared mailbox as a shared folder only this option is currently available and working, add-ins have no effect when clicking on them:

In Outlook the add-ins aren't even displayed when selecting a shared mailbox. For some other users the add-ins are there but have no effect.

For a newly auto-mapped shared mailbox the add-ins are displayed for few seconds, can be used, then disappears.

What is the exact behavior we should be expecting with the Report Message or the Report Phishing add-ins when managing/dealing with shared mailboxes ?

What is the difference between "Security Options" and add-ins ?

BR
Ribo

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 445892a6-5948-33e4-ce60-72670829d8ef
Version Independent ID: a7407a6f-6563-9a4b-4a95-7ed908ab322b
Content: Enable the Report Message or the Report Phishing add-ins - Office 365
Content Source: microsoft-365/security/office-365-security/enable-the-report-message-add-in.md
Product: m365-security
Technology: mdo
GitHub Login: @Dansimp
Microsoft Alias: dansimp

@riohedfield Thank you for your feedback. I was able to test all the methods and you are right, we are able to mark it is Junk and Phishing when opened as another mailbox and these are security options that will update your internal anti-spam policy Only.
The Report Message and Report Phishing add-ins for Outlook and Outlook on the web are used to report false positives (good email marked as bad) or false negatives (bad email allowed) to Microsoft and its affiliates for analysis. In other words, if you report a message as junk using the security options, You are updating just the internal policy. However, if you mark it using the Report Message add-in the email is sent to Microsoft for analysis, and your internal policies are also updated.
I could not make the add-in to work on a shared mailbox, so that statement from the document is right, However, there seems to be a bug with an auto-mapped mailbox where it lets the users use the add-in for a few minutes.
If you find such a bug in design, you can report it to our product group via the main feedback portal - https://feedbackportal.microsoft.com/ or raise a support ticket with Microsoft via the need help section in the admin center.
Hope this helps!
Thanks
Sri

Hi @msbemba,
Thank you so much for your detailed answer; on the other hand when you say:
" if you report a message as junk using the security options, You are updating just the internal policy"
By internal policy you mean the MailboxJunkEmailConfiguration of the shared mailbox right ?
Also I've tested again using the Security Options then Mark as Junk/ Mark as not Junk:

and I receive this pop up window:

I click report, and as an admin I receive a copy of the report because my address is used as the Organization's Mailbox, and I guess the Message is as well reported to Microsoft, because the User Reported Message Settings are configured this way:

I was aiming for some specifics to be added to the documentation when it comes to shared mailboxes to avoid  confusion and have a better idea how the Add-ins vs Security Options are working.
Thank you
BR

@RatulaC and @chrisda shared/delegate reporting is not supported. we are working to fix this no ETA. So when someone reports from it - it moves the email to the folder as we normally do but there is no report to custom mailbox or Microsoft or both.
Security options vs addin should work in the same way in the future. I cannot provide an ETA as different clients are doing development work in different time.

I added the following bullet to the What do you need to know before you begin? section. Feel free to modify:

Currently, reporting messages in shared mailboxes or other mailboxes by a delegate using the add-ins is not supported. Messages are not sent to the custom mailbox or to Microsoft. Built-in reporting in Outlook on the web sends messages reported by a delegate to the custom mailbox and/or to Microsoft.

Thanks for the feedback.
Closing


fluentd failed to process log after Kubernetes upgrade from 1.19.x to 1.20.x

Our log system for business is EFK(elasticsearch+fluentd+kibana). The fluentd daemonset is running on AKS(Microsoft Azure Kubernetes service). I haven't done any change to the fluentd configuration. And after I upgrade the kubernetes from 1.19.x to 1.20.x, the fluentd has not sent log to Elasticsearch anymore.
From the logs of fluentd I can see it did grab logs which are located in directory /var/log/containers on kubernetes node. But then it did not send the log data to Elasticsearch. And I can see that the buffer directory is empty. When it worked well, there were buffer files written to the buffer directory.
Is there some change of Kubernetes 1.20.x that affect fluentd running?

What is the image of fluentd you are running and what do you have set for the conf?
we had to update
image: fluent/fluentd-kubernetes-daemonset:v1.4-debian-elasticsearch-1

to
image: fluent/fluentd-kubernetes-daemonset:v1.12-debian-elasticsearch6-1

As well as update the kubernetes.conf source:
format /^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$/


Add option to force s3 virtual host addressing

You can force virtual style addressing through the config file:
s3 =
    addressing_style = virtual_host

You can also specify it in the Config object passed in when a client is instantiated. The virtual host addressing no longer defaults to the s3 global endpoint. Instead, it will use the endpoint url determined from the region or if the endpoint url that is specified upon instantiation of a client:
session = botocore.session.get_session()
s3 = session.create_client('s3', config=Config(s3_addressing_style='virtual'))

If you force the virtual style addressing and the bucket name on an operation you call is not DNS compatible it will throw an error. Another important note to emphasize is that if you force the virtual style addressing, it will use the endpoint of the region you specified so you may hit PermanentRedirect errors. Otherwise if it is not forced, the requests will still be sent to the the global s3 endpoint.
As for testing, the s3 integration tests pass for both botocore and the CLI. I even tried running the integration tests with virtual style forced and the botocore tests passed and most of the CLI test passed (their were a few tests that randomly generated '.' in their name which would cause an error to be thrown as expected because the name is not DNS compatible).
Also, thanks to testing it in this fashion I fixed some bugs where if virtual style is forced and it is sigv4, we would get signature errors. We did not run into these bugs before because we would turn off virtual style if sigv4 is being used.
cc @jamesls @mtdowling @rayluo @JordonPhillips

Looking at this now I have some initial high level feedback before digging into this:
Given the scoped config is {'s3': {'addressing_style': 'virtual_host'}}, I think we should map as closely to that in the client config, so something like Config(s3={'addressing_style': 'virtual_host'}).  It also makes it a little easier to see all the s3 specific config options rather than having to iterate over config.__dict__ and check if it starts with s3_.

That sounds good. It would provide a precedence to prevent the Config file from becoming to bloated. Maybe name it to something more explicit like s3_options or s3_configuration?

Looks good, just had some small feedback.

@kyleknap I would prefer s3 instead of s3_configuration.  I think the fact that it's in the Config object makes it clear that it's configuration data.

Ok that's fair.

@jamesls
I updated based on feedback. I ended up adding support for 'path'. I also set it and ran it on the s3 CLI integration tests and all of those tests passed. So there should not be any auth issues.

Changes look good.  Just wondering what we should be doing about merging config.  It makes it slightly more complicated for the user once they have various S3 options in their config and they want to override only specific options, but I suppose it's not that hard for them to manually merge the s3 config dicts before creating a client.  What do you think?

@jamesls
We talked about this. Going with the merging of dictionaries approach. Should be good to look at again.

:shipit:

This broke some CLI integration tests reverting.


Using multiple PrismaClient instances

Context
https://prisma-company.slack.com/archives/C5Z9TH6N9/p1624863600370600 (@janpio will provide a use case)
Suggested action

A new page under this section named 'Using multiple instances of PrismaClient' or similar: https://www.prisma.io/docs/concepts/components/prisma-client/working-with-prismaclient (nav title can be 'Multiple instances'
In every location where we mention using one instance, link to this page with a short explanation of when you might use more than one

Another data point that this would be useful: https://github.com/prisma/prisma/discussions/17855#discussioncomment-4952250


Get WCT working with Appveyor

@justinfagnani this PR wasn't quite ready for review; I've been using it as a vehicle to try and coerce appveyor to succeeding in running tests on Windows as its behavior is different than the Windows VM I was developing on.  I'll send a review request once I can get CI green for IE11.


Advertising: Sample AVD - Azure DevOps project using CAF

Hi,
together with some colleagues I did a DevOps project for automating the deployment of AVD using CAF principles
aka.ms/avdcollege

hth,
Bernhard

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 516d6288-2c04-3934-04a9-ca95dca66d3d
Version Independent ID: 049eeabb-25b0-9d96-7bba-1aff3f7e5015
Content: Platform automation and DevOps for Azure Virtual Desktop - Cloud Adoption Framework
Content Source: docs/scenarios/wvd/eslz-platform-automation-and-devops.md
Service: cloud-adoption-framework
Sub-service: scenario
GitHub Login: @BrianBlanchard
Microsoft Alias: brblanch

This may make a good contribution, I'll follow up offline with some links for that option.


Work

Hi @Acino01, thanks for opening this issue. As there's currently no description provided, I would be thankful if you can clarify what this issue is about.

Hi @Acino01, as I've not heard back I'm closing for now. Feel free to open a new issue if you feel there is something to discuss.


/ before separator for getCSV

What version of Hugo are you using (hugo version)?
Hugo v0.70.0 Mac install with brew
Does this issue reproduce with the latest release?
Yes
The bug
I am using getCSV for importing a csv from the web and this file launch an error
 ERROR 2020/05/08 00:06:05 Failed to get CSV resource "https://raw.github.com/Gelma/LugMap/master/db/veneto.txt": failed to parse CSV file https://raw.github.com/Gelma/LugMap/master/db/veneto.txt: record on line 2: wrong number of fields
`
This error is when you have the separetor after a / .
You can reproduce this bug with this code in a template file
`
{{ $url := print "https://raw.github.com/Gelma/LugMap/master/db/veneto.txt" }}
{{ $sep := "|" }}
{{ range $i, $r := getCSV $sep $url }}
{{ index $r 0 }}
{{ index $r 1 }}
{{ index $r 2 }}
{{ index $r 3 }}
{{ index $r 4 }}

{{ end }}
`

Your CSV file is invalid. See:
https://gohugo.io/templates/data-templates/#specs-for-data-formats
https://tools.ietf.org/html/rfc4180
Section 2, item 4 of RFC4180 states:

Each line should contain the same number of fields throughout the file.

Some of the records in your file contain a phone number, others do not. Here's a clean file with the phone numbers removed.

As @jmooring noted, your CSV file is improperly formatted.


Stop putting env vars into BoundPods.

They will still show up in etcd.  They never were available
through the API.
A subsequent PR(s) will rip out all BoundPods code.
Working in small increments.
This PR will cause users on lagging cloud providers
to not get env vars in their pods if they update to this code.
They have already been warned via email.
Removed unit tests of BasicBoundPodFactory.
There is adequate coverage in pkg/kubelet/kubelet_test.go.

LGTM
So nice.

Do the Red Hat folks have concerns?
@smarterclayton

None.

not sure why  integration test failed on go1.4 on travis.  Works for me and shippable and travis+1.3

@erictune Could you rebase please

@saad-ali rebased

Does this close #4176?

no
On Thu, Feb 12, 2015 at 11:19 AM, Zach Loafman<EMAIL_ADDRESS>wrote:

Does this close #4176
https://github.com/GoogleCloudPlatform/kubernetes/issues/4176?
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/pull/4359#issuecomment-74133792
.

and by "no" I mean yes.
On Thu, Feb 12, 2015 at 11:21 AM, Eric Tune<EMAIL_ADDRESS>wrote:

no
On Thu, Feb 12, 2015 at 11:19 AM, Zach Loafman<EMAIL_ADDRESS>wrote:

Does this close #4176
https://github.com/GoogleCloudPlatform/kubernetes/issues/4176?
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/pull/4359#issuecomment-74133792
.


Text UI (CLI) support in Designer

Is your feature request related to a problem? Please describe.
Sometimes I need to connect into my TcMenu powered device remotely. While embedControl works, I believe the solution could be easily extended to generate text UI interface besides the regular TcMenu UI. Perhaps this could be done using a new plugin?
Describe the solution you'd like
I'd like to reuse what each project already has, a menu tree structure. From there we can generate a CLI hierarchy.
CLI> // root menu, regular access
CLI# // root menu, privileged access, e.g. through entering a password
Let's say we have a submenu called Settings, user can get there from the root:
CLI> Settings
CLI Settings>
In nested submenus we could use pipes:
CLI Settings | Display> ...
Tab completion would be also great!
I believe from the menu tree we already have enough information to generate menu like this. However, we should allow user for extending this menu using custom code so they can add custom callbacks or methods not needed in regular TcMenu. Also, they should be able to implement the body of such functions. For example:
CLI Settings | Display> show info
(Prints display settings, a custom method implemented by a user)
Each submenu should have an implicit help method which lists all available commands in the subtree.
Additional context
I already have a PoC with some 3rd party library and I can provide the sample project. The difference is that I had to recreate the whole menu from scratch instead of using generated TcMenu code. The CLI works via serial/tty and allows me to control my device from all around the world. Since a terminal app is widely available, it works even on places where embedControl cannot be installed (e.g. no graphical terminal).

I can also imagine using this in cases where the device has no display but it could still use TcMenu, the same way as many network devices like routers operate.


[GSK-1321] Integration with MLflow

   1. mlflow Plug-in via evaluate
Description
Integration of mlflow via the model_evaluator plugin.
Installation requirements:
pip install mlflow # mlflow is needed for .evaluate() not mlflow-skinny shipped with giskard
pip install "git+https://github.com/Giskard-AI/giskard.git@gsk-1321/mlflow-integration#subdirectory=python-client" -q

Code example:
import mlflow

from giskard import demo
model1, df = demo.titanic(max_iter=5)
model2, df = demo.titanic(max_iter=100)

with mlflow.start_run(run_name="model1") as run1:
    model1_uri = mlflow.sklearn.log_model(model1, "sklearn_model1", pyfunc_predict_fn="predict_proba").model_uri
    mlflow.evaluate(model=model1_uri, model_type="classifier", data=df, targets="Survived", evaluators="giskard", evaluator_config={"classification_labels": ["no", "yes"]})

with mlflow.start_run(run_name="model2") as run2:
    model2_uri = mlflow.sklearn.log_model(model2, "sklearn_model2", pyfunc_predict_fn="predict_proba").model_uri
    mlflow.evaluate(model=model2_uri, model_type="classifier", data=df, targets="Survived", evaluators="giskard", evaluator_config={"classification_labels": ["no", "yes"]})

Running mlflow ui in the terminal, one gets:

the html results of scan embedded as Artifacts.
the test suite result of scan logged as Metrics.
model and dataset artifacts

Two notebooks to test this feature:
Run them locally in order to run the mlflow ui

titanic demo (sklearn): https://colab.research.google.com/drive/1j2w7MtTThXej8fEagdYUxwbeSnMrg9Cc?usp=sharing
text classification (TensorFlow): https://colab.research.google.com/drive/1MbWVLYZeryKtv_VrTJ_zlYdNoB_BFlaf?usp=sharing

   2. Giskard API via to_mlflow
Description
logging artifacts and metrics from giskard to mlflow
Code example:
scan_results = giskard.scan(giskard_model, giskard_dataset)
test_suite = results.generate_test_suite("My first test suite")
test_suite_results = test_suite.run()

import mlflow

# Option 1 (via the fluent API)
with mlflow.start_run() as run:
    giskard_model.to_mlflow()
    giskard_dataset.to_mlflow()
    scan_results.to_mlflow()
    test_suite_results.to_mlflow()

# Option 2 (via MlflowClient)
from mlflow import MlflowClient

client = MlflowClient()
experiment_id = "0"
run = client.create_run(experiment_id)

giskard_model.to_mlflow(client, run.info.run_id)
giskard_dataset.to_mlflow(client, run.info.run_id)
scan_results.to_mlflow(client, run.info.run_id)
test_suite_results.to_mlflow(client, run.info.run_id)

notebook to test this feature:
Run them locally in order to run the mlflow ui:

titanic demo (sklearn): https://colab.research.google.com/drive/1cnYTOmFQnQXRGTlYif3MMYe7aOkCwC5d?usp=sharing

   Open questions

what if the model logged into MLflow is not giskard-compliant?
how does MLflow handle data preprocessing?

MLflow allows the custom creation of models if pre- or post-processing is needed, see: https://mlflow.org/docs/latest/python_api/mlflow.pyfunc.html?highlight=preprocessing

Todo:
Tech

[x] better html rendering -- fixed in https://github.com/Giskard-AI/giskard/pull/1209
[x] better metrics logging
[x]  use native mlflow saving when possible -- implemented inhttps://github.com/Giskard-AI/giskard/pull/1189/commits/e83667df6674c229bb922c797f338b2a5d2b4bd3

[x] remove save_model from MLflowBasedModel and introduce a dataclass class to create flags and disable the saving/loading validation during the scan. (PyFunc models are not meant to be saved, and the model going through mlflow.evaluate is a PyFunc model)

[x] implement the above notebooks as functional-tests
[x] tempfile.NamedTemporaryFile doesn't seem to delete files after context
[x] add text as model_type
[x] ability to compare two models (via scan_summary.json)
[x] log scan results as json file to be rendered in artifact view of mlflow and allow comparison
[x] implement to_mlflow() on (see doc):

[x] ScanResult
[x] TestSuiteResult
[x] giskard.Model
[x] giskard.Dataset

[x] artifact view similar to doc
[x] Allow people to add any Model.__init__() argument to evaluator_config not only classification_labels
[x] Better error handling + their unit-tests
[x] run pdm lock -G:all
[x] Implement EvaluationResult
[x] implement telemetry tracking
[x] Implement an output message pushing for mlflow ui

Writing
   LLM models comparison
import pandas as pd
from langchain import PromptTemplate, LLMChain
from langchain.llms import OpenAI
import mlflow
import openai
import os

df = pd.read_csv('https://raw.githubusercontent.com/sunnysai12345/News_Summary/master/news_summary_more.csv')
df_filtered = pd.DataFrame(df["text"].sample(10, random_state=11))

prompt = PromptTemplate(template="Create a reader comment according to the following article summary: '{text}''",
                        input_variables=["text"])

openai.api_key = os.getenv("OPENAI_API_KEY")

llm1 = OpenAI(openai_api_key=openai.api_key,
             request_timeout=20,
             max_retries=100,
             temperature=0,
             model_name="text-ada-001", ) # Possibility to select another model

chain1 = LLMChain(prompt=prompt, llm=llm1)

with mlflow.start_run(run_name="text-ada-001") as run2:
    model_uri = mlflow.langchain.log_model(chain1, "langchain").model_uri
    mlflow.evaluate(model=model_uri, model_type="text", data=df_filtered, evaluators="giskard")

llm2 = OpenAI(openai_api_key=openai.api_key,
              request_timeout=20,
              max_retries=100,
              temperature=0,
              model_name="text-embedding-ada-002", ) # Possibility to select another model

chain2 = LLMChain(prompt=prompt, llm=llm2)

with mlflow.start_run(run_name="text-embedding-ada-002") as run1:
    model_uri = mlflow.langchain.log_model(chain2, "langchain").model_uri
    mlflow.evaluate(model=model_uri, model_type="text", data=df_filtered, evaluators="giskard")

[ ] doc
[ ] Write an article

Optional

[ ] Populate MLflow's evaluation examples with the giskard evaluator.
[ ] Capture giskard.scanner.logger ERROR    Detector LLMToxicityDetector failed with error: 'PyFuncModel' object has no attribute 'rewrite_prompt' and output instead something like giskard.scanner.logger ERROR    Detector LLMToxicityDetector is not supported by giskard plug-in through mlflow.evaluate

@andreybavt I am currently wrapping the PyFunc model I have access to in mlflow.evaluate with a custom giskard CloudPickleBasedModel. I only needed to customise model_predict, which was the advantage, since PyFunc standardise predict().
The drawback of not having access to the underlying model object, is that we can't run all scan detectors. This mainly happened only once, for the toxicity detector where we needed the underling langchain model object to regenerate prompts, see: https://github.com/Giskard-AI/giskard/blob/116e1fd4f652cedcc0497a54eb7a78ecba089c89/python-client/giskard/models/langchain/init.py#L51-L71.
We can put a pin in this, but I think we might be forced to re-think the unwrapping of the model-object from the artifact logged by MLflow for our own purposes (scan).

@andreybavt I am currently wrapping the PyFunc model I have access to in mlflow.evaluate with a custom giskard CloudPickleBasedModel. I only needed to customise model_predict, which was the advantage, since PyFunc standardise predict().
The drawback of not having access to the underlying model object, is that we can't run all scan detectors. This mainly happened only once, for the toxicity detector where we needed the underling langchain model object to regenerate prompts, see:
https://github.com/Giskard-AI/giskard/blob/116e1fd4f652cedcc0497a54eb7a78ecba089c89/python-client/giskard/models/langchain/__init__.py#L51-L71
We can put a pin in this, but I think we might be forced to re-think the unwrapping of the model-object from the artifact logged by MLflow for our own purposes (scan).

I think we can look at model_type argument (or the value of model_type in model_config) to either wrap it with just a PyFuncModel or a LangchainModel and then be able to call rewrite_prompt in case it's a text_generation model

@andreybavt I am currently wrapping the PyFunc model I have access to in mlflow.evaluate with a custom giskard CloudPickleBasedModel. I only needed to customise model_predict, which was the advantage, since PyFunc standardise predict().
The drawback of not having access to the underlying model object, is that we can't run all scan detectors. This mainly happened only once, for the toxicity detector where we needed the underling langchain model object to regenerate prompts, see:
https://github.com/Giskard-AI/giskard/blob/116e1fd4f652cedcc0497a54eb7a78ecba089c89/python-client/giskard/models/langchain/__init__.py#L51-L71
We can put a pin in this, but I think we might be forced to re-think the unwrapping of the model-object from the artifact logged by MLflow for our own purposes (scan).

I think we can look at model_type argument (or the value of model_type in model_config) to either wrap it with just a PyFuncModel or a LangchainModel and then be able to call rewrite_prompt in case it's a text_generation model

As it's not a blocker, I think it's a good idea to merge this branch and to take care of this point in a new one. This will unlock some marketing actions. WDYT?


Option for Matrix Based Cell Coloring

Provide the option (perhaps checkbox) to allow users to color the matrix plots based on whether the cell comes from the R, U, V, or Y matrix.

Implemented and tested. Most of implementation resides in utils/matrix.py


Fixed button NeedsImageProp property.

Add test for ownerdraw button.
Add tests for static control.

Current coverage is 96.55%

Merging #91 into master will increase coverage by +0.04% as of 4daf47c

@@            master     #91   diff @@
======================================
  Files           22      22
  Stmts        16332   16331     -1
  Branches         0       0
  Methods          0       0
======================================
+ Hit          15763   15769     +6
  Partial          0       0
+ Missed         569     562     -7

Review entire Coverage Diff as of 4daf47c

Powered by Codecov. Updated on successful CI builds.


agentstates: Reformat construction of returned dictionary

How about the actual error on test / lint ? Shouldn't we deal with it first? Not too worried about the jsonschema warning btw

How about the actual error on test / lint ?

Can you restart the linter via the github web interface. That's what I would try first.


[SONARMSBRU-183] Added MSBuild task to add merge result sets

This commit can't be manually tested yet; it just adds the new task and tests. The targets haven't been changed to call it yet. I'm sending this for review now to make the next review which will change the targets simpler.

LGTM

FYI I'll push this change now as it has no effect on the existing product code.


dx12: Descriptor Pool limitations

Descriptor pools map 1:1 to a slice in the global descriptor heap (1M descriptors, 2k samplers).
Overallocation would be troublesome, but Vulkan allows to have more. Even some CTS tests relay on it as well as some real world applications.
Emulating this is quite costly and I would love to avoid this. At the end this depends on what Khronos Working Group decides on.

FTR, Khronos TSG didn't decide to change these in any way since we said that it's possible on D3D12. I thought we had a rough plan on supporting multiple pools in play, and it's more a matter of implementation complexity than resulting performance.

well, yes, it's doable but requires a lot of descriptor juggling. Performance impact is hard to estimate I would say. At the end, exposing a feature to restrict stuff might give the user the best of both worlds?

Do you have an idea of how this could sound in the spec? Saying something straightforward like "at no point should a draw call use bindings from multiple descriptor pools, unless there is one for samplers specifically" :)

Exposing an upper limit of concurrently allocated descriptors types by descriptor pools coupled with error on descriptor pool allocating due to fragmentation etc.
This would allow to hide the heap as implementation detail. Maybe in far future Vulkan 3 will move towards Mantle definition of descriptor sets :P

Interestingly, the number of samplers is already capped in Vulkan: maxSamplerAllocationCount

right, nonetheless, ordered array of samplers as found in descriptor sets are more of an issue. Even though we can create 2k samplers easily (still less than 4k guarenteed by vulkan) and would be able to store them in gpu heaps we still need to deal with the issue of the user being able to arbitrarily combine them.

Sampler limitations are addressed in  #3393


train model

how to can train the models ? please help me for train dnn models tanx

It is explained here:
https://github.com/tamarakatic/sentiment-analysis-in-amazon-reviews/blob/master/README.md#deep-learning-models
But first, install all dependencies and process dataset:
https://github.com/tamarakatic/sentiment-analysis-in-amazon-reviews/blob/master/README.md#installing
If you have any questions please let me know!


Failed to verify bitcode issue

I get the following error message while trying to do an Ad-Hoc export of project that uses DynamicColor framework:
Failed to verify bitcode in DynamicColor.framework/DynamicColor:

description = "Failed to verify bitcode in DynamicColor.framework/DynamicColor:\nerror: Cannot extract bundle
level = ERROR;
type = "malformed-payload";

It happens in Compile Bitcode expert step.

Hi Damir,
Which version are you using? Do you work with Cocoapods?

Latest version, Xcode 8, Swift 3, Carthage.

Ok, thank you, I'll try to take a look tonight. 👍

Thanks!

I've tested it with a sample project and it works fine for me. When the error occurred? During the archive or in debug mode?

During archive for Ad-Hoc export.
Side note: I have no other dependency in the project.

I've created an empty project and I only import the DynamicColor lib using carthage (and I put some code to use the lib) then I Archive in Ad-hoc with the bitcode enabled and it works fine.
Does it work with an empty project in your side? Or have you an example project?

@yannickl Seems to work now all of a suden. 🙈 Thank you anyway!


Add a note re: timezone in Sending an Email

To do:
Add a note to the Sending an Email page informing the customer that they need to make sure their account is set to the correct timezone to ensure that their emails are indeed sending when they want them to. Add simple instructions for navigating to their timezone in their account settings.

Completed in #3010

@hmbrown64 and @madhurgupta10 - I'm so sorry, but it looks like multiple people submitted pull requests for this issue last night! I really appreciate your efforts on this, but I am accepting the first one (https://github.com/sendgrid/docs/pull/2986) - please do check back for more open issues though! We are adding new ones often.

@Whatthefoxsays - I would love to contribute to other projects.

@madhurgupta10 awesome! check back here often for issues, because we'll continue to add in new ones often. We also have 7 libraries that you could contribute to as well! https://github.com/sendgrid


[WIP] api docs: Implement reading parameters from OpenAPI files.

Right now we can only read parameters from the arguments.json in order to display them in the REST API docs.
With these changes, the parameters can be injected into the docs like this:
{generate_api_arguments_table|../../../static/openapi/zulip.yaml|/<api_endpoint>/<method>}

(yeah, once we have everything migrated to OpenAPI I'll change the base path so we don't have to ../../.. all the way to static/openapi).
In order to achieve this in a cleaner way, the following changes have been made:

Renamed Swagger to OpenAPI.
Created a new file that uses the new OpenAPI format, so we can start migrating the old stuff with the new 3.0 specification.

Testing Plan: All tests passed locally + the old tables show properly + the table in /api/update-message looks great as well.

The works for me structurally (though it'd be really nice to not have the hardcode the ../ part of the path to zulip.yaml); I'll merge the first few commits once CI passes.  But the actual arguments data in the OpenAPI file doesn't match what we have in the old version, and we should resolve that.

I merged the first 2 commits; might also want to do a commit or two renaming docs/subsystems/swagger-api-docs.md to mention OpenAPI (wouldn't bother updating the content until we figure out what we're doing)

Made a small fix that we should remove once everything has been moved to YAML files, that keeps us from having to do the ../../.. trick. Also renamed the docs as you mentioned.
By the way, I didn't quite get this part, could you ellaborate a bit more?

[...] the actual arguments data in the OpenAPI file doesn't match what we have in the old version, and we should resolve that.

@timabbott Let's wait before finally merging this, if it isn't a problem. I just realized that we may have to change the pyyaml dep in order to fetch the fixtures from the spec.

Sounds good.  That last bit is that we should figure out which of the OpenAPI docs vs. the arguments.json docs is better where they differ before switching this over (they are definitely different for this endpoint).

Ah, gotcha. I just took the arguments.json ones in this case (which looked better to me), and made a small addition to render_table's logic in zerver/lib/bugdown/api_arguments_table_generator.py. This way, it displays all the possible values if there's a list you have to choose from (i.e. an enum in the argument's schema).
I'll let you know as soon as we can safely merge this.

This should be ready to review and merge now! Removing the WIP status.

I merged 3 commits, and posted comments on the remaining one.

Hmmm, apparently static/openapi/zulip.yaml isn't reachable in production.
At first I thought that this could be like when we were trying to import KaTeX without including it in the static asset pipeline, but this file shouldn't be processed/minified/etc in any way, so I'm a bit confused about what the problem is. Mind giving any pointers?

I merged the first 2 commits.
The issue is that static/ doesn't get included in production at all (because everything in there is supposed to be shipped via minification).  Possibly we should just move it to somewhere else, e.g. zerver/openapi/ or something.

Thanks for the guidance, @timabbott! It should be fixed now.

Nice, merged, thanks @YagoGG!  Let's check in on where we are with the roadmap now that this is done.


"Segmentation Fault (Core Dumped)" Error using train.py

Hi, When I try to run train.py. I get the following error.. FYI: Demo works fine for me..
Segmentation fault (core dumped)

@sandeshshrestha45 If you were successful in training using train.py, Can you help me with the following error..
Segmentation fault (core dumped)

@silymarins If you were successful in training using train.py, Can you help me with the following error..
Segmentation fault (core dumped)

I found the error. The issue was in the matplotlib.pyplot version. I had the latest version which was 3.1.2
I uninstalled it and installed an older version which is 2.0.0 It is working now..


Check Dokcer repo and tag?

@BorjaEst @vykozlov
Maybe it makes sense to have a step in the cookiecutter preprocessing to check that the repo+tag submitted by the user do indeed exist in DockerHub.
This can be done easily using the DockerHub API.

We did something similar in EOSC Perf, the main constrain was when users want to use images located at nvidia hub or other personal hubs, let me take a look.

Yes. Well, we can always detect if the repo belong to DockerHub and if so, perform this check. Otherwise, skip it.

Just to mention, draw back is the cookiecutter becomes dependent on the user internet connection when creating the project (no problem from Templates Hub I hope) and from that docker hub is also up. I suppose nevertheless it should be ok.

Yes. Maybe add a timeout, and if DockeHub does not respond, proceed anyway without errors.

Hi @BorjaEst and @IgnacioHeredia ,
do we talk about "docker_baseimage"? and "baseimage_tag"?
If so, do we really need to check them? hub.docker.com in the meantime is not the only Docker repository, e.g.:

ghcr.io - from Github
nvcr.io - from Nvidia
quay.io - from RedHat

in the end, there are custom deployed Harbor instances..
Also "docker_baseimage" appears only in the Dockerfile, i.e. it will immediately clear for the user why the build fails, if Docker image name:tag is wrong.
Like that, I do not see a value in having this check, or?

It's true that is is a fixable error, as it does not spread much.
So I'm fine not checking it, especially since, as you say, can potentially come from anywhere.
Feel free to close the issue.


use 'preloadedState' instead of 'initialState'

Coverage remained the same at 93.939% when pulling 054ef957386f9b1913c1a594fbcc62c05694b9e5 on state into b7daf5eca8a15934f0647220153f96b6f732009a on master.


remove broken links

fix this issue #614

Hey, thanks. Can you send the PR to the dev branch?

@PierfrancescoSoffritti  done

Thanks!


TestCafe not reporting back to console

What is your Test Scenario?

I've been experiencing an issue where TestCafe stops reporting on which test/fixture it's currently running. We have over 100 tests which we run nightly, and a subset of these during our CI/CD on CircleCi. Often we see build failures due to CircleCi's 10m time limit on getting a response "Too long with no output (exceeded 10m0s)".
What is the Current behavior?

I added in some logging, for when a test starts and completes, and I found that TestCafe stops reporting back to the console, despite tests continuing to run (and complete). Looking through the log it appears that 1 of our tests seems to be started 3 times, but never completes (and quarantine mode is disabled). Eventually TestCafe crashes with FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - Clearly something to do with our test.
What is the Expected behavior?

I would expect TestCafe to continue to report back to the console, whenever a test / fixture is completed. If there is a fault with a test, which is unable to complete for some reason, then this should be flagged as failed "Something occured and testcafe was unable to complete this test" etc.
What is your web application and your TestCafe test code?

Your website URL (or attach your complete example):

Your complete test code (or attach your test files):

This is just one of our fixtures, but it appears that the first test is the one causing the issue. Whenever I try to debug it, it works, so I'm unable to locate where the fault lies.
 import { authenticatedUserWithOrgRole } from '../../../../support/custom_commands/getRoles';
import { baseUrl, httpAuthCredentials } from '../../../../support/helpers';
import { generateNewUserInfo } from '../../../../support/helpers/newUserHelper';
import loginPage from '../../../../support/page_objects/loginOrJoin';
import logout from '../../../../support/custom_commands/logout';
import makeUserIdentified from '../../../../support/custom_commands/makeUserIdentified';
import newUser from '../../../../support/page_objects/newUser';
import sapFlow from '../../../../support/page_objects/startPetitionFlow';

fixture('Start A Petition Flow')
  .page(`${baseUrl}/?country_code=CA&lang=en-US`)
  .httpAuth(httpAuthCredentials);

test('Start a petition as identified user', async t => {
  const user = await newUser.createNewUser();

  await sapFlow.startAPetitionAskAndDm();
  await makeUserIdentified();
  await sapFlow.completeDescriptionStep();
  await sapFlow.completeMediaUpload();

  await t
    .expect(loginPage.signUpModal.visible)
    .ok()
    .typeText(loginPage.passwordInput, user.password)
    .click(loginPage.signUpContinueButton);
  await loginPage.completeYourProfile();
});

test('Start a petition as Authenticated user - with Org', async t => {
  await t.useRole(authenticatedUserWithOrgRole);
  await sapFlow.startAPetitionAndUploadMedia();
  await t
    .expect(loginPage.completeYourProfileModal.visible)
    .ok()
    .click(loginPage.jsDone);
  await sapFlow.completeStartAPetitionFlow();
});

test('Start a petition as Org user', async t => {
  await t.useRole(authenticatedUserWithOrgRole);
  await sapFlow.startAPetitionAndUploadMedia();
  await t
    .expect(loginPage.completeYourProfileModal.visible)
    .ok()
    .click(loginPage.loginAsOrg)
    .click(loginPage.jsDone);
  await sapFlow.completeStartAPetitionFlow();
});

test.meta({ smoke: 'true' })('Start a petition as guest user - sign in', async t => {
  const user = await newUser.createNewUser();
  await logout();
  await t.navigateTo(`${baseUrl}/?country_code=CA&lang=en-US`);

  await sapFlow.startAPetitionAndUploadMedia();
  await t
    .expect(loginPage.signUpContinueButton.visible)
    .ok()
    .click(loginPage.modalLoginLink);
  await loginPage.loginViaModal(user.email, user.password);
  await loginPage.completeYourProfile();
});

test.meta({ smoke: 'true' })('Start a petition as guest user - create new user', async t => {
  const user = generateNewUserInfo();

  await sapFlow.startAPetitionAndUploadMedia();
  await t
    .expect(loginPage.signUpContinueButton.visible)
    .ok()
    .typeText(loginPage.signUpFirstName, user.first_name)
    .typeText(loginPage.signUpLastName, user.last_name)
    .typeText(loginPage.emailInput, user.email)
    .typeText(loginPage.passwordInput, user.password)
    .click(loginPage.signUpContinueButton);

  await loginPage.completeYourProfile();
});

test.page(`${baseUrl}/?country_code=US&lang=en-US`).meta({ smoke: 'true' })(
  'Start a petition as US guest user - create new user',
  async t => {
    const user = generateNewUserInfo();

    await sapFlow.startAPetitionAndUploadMedia();
    await t
      .expect(loginPage.signUpModal.visible)
      .notOk()
      .typeText(loginPage.emailInput, user.email)
      .click(loginPage.signUpContinueButton);
    await t
      .expect(loginPage.completeYourProfileModal.visible)
      .ok()
      .typeText(loginPage.signUpFirstName, user.first_name)
      .typeText(loginPage.signUpLastName, user.last_name)
      .typeText(loginPage.passwordInput, user.password)
      .click(loginPage.existingUserContinueButton);
    await sapFlow.completeStartAPetitionFlow();
    await sapFlow.editPetitionAndUploadImage();
  }
);

test.meta({ smoke: 'true' })('Start a petition as an authenticated user', async t => {
  await newUser.createNewUser();
  await sapFlow.startAPetitionAndUploadMedia();
  await t
    .expect(loginPage.completeYourProfileModal.visible)
    .ok()
    .click(loginPage.existingUserContinueButton);
  await sapFlow.completeStartAPetitionFlow();
});

Your complete configuration file (if any):

{
  "browsers": ["chrome", "firefox"],
  "src": "./tests/**/*.js",

  "assertionTimeout": 10000,
  "pageLoadTimeout": 30000,
  "selectorTimeout": 10000,

  "quarantineMode": false,
  "skipJsErrors": true,

  "disablePageCaching": true,

  "reporter": [
    {
      "name": "xunit",
      "output": "artifacts/test-results/results.xml"
    },
    {
      "name": "spec"
    }
  ],

  "screenshots": {
    "fullPage": true,
    "takeOnFails": true,
    "pathPattern": "${TEST_ID}/${RUN_ID}_${FILE_INDEX}.png",
    "path": "artifacts/screenshots"
  },

  "videoOptions": {
    "singleFile": false,
    "failedOnly": true,
    "pathPattern": "${TEST_INDEX}/${RUN_ID}_${FILE_INDEX}.mp4"
  },
  "videoPath": "artifacts/recordings"
}

Your complete test report:

This is the copy from the console log. Despite only Start A Petition Flow being reported as the current fixture, 4 other fixtures were executed before the JS OOM error.
It appears that 2 tests were started prior to TestCafe reporting the fixture starting. And also looks like Start a petition as identified user was started 3 times, but never completed.
Start a petition as identified user - Started
Start a petition as Authenticated user - with Org - Started

`Start A Petition Flow` _- This is what TestCafe reports as it "starts" the Start Petition Fixture_
Start a petition as Org user - Started
Start a petition as Authenticated user - with Org - Completed
Start a petition as guest user - sign in - Started
Start a petition as Org user - Completed
Start a petition as guest user - create new user - Started
Start a petition as guest user - create new user - Completed
Start a petition as US guest user - create new user - Started
Start a petition as identified user - Started
Start a petition as guest user - sign in - Completed
Start a petition as an authenticated user - Started
Start a petition as US guest user - create new user - Completed
Start a petition as an authenticated user - Completed

Guest user becomes a member and selects the 4th amount option - Started
Guest user enters custom amount - Started

Authenticated user with saved credit card becomes a member - Started
Indecisive existing user changing payment methods before finally paying with CreditCard - Started
Start a petition as identified user - Started
Indecisive existing user changing payment methods before finally paying with CreditCard - Completed

Guest paying with credit card makes a OTC successfully, sees the thank you page, and a membership is not created - Started
Guest using an existing email and paying with credit card is prompted to log in, makes a successful OTC, and sees the thank you page - Started
Authenticated user with saved credit card becomes a member - Completed

Authenticated user with saved credit card details promotes a petition and it succeeds and redirects to the thank you page - Started
Authenticated user with saved credit card details promotes a petition and accepts membership upsell - Started

==== JS stack trace =========================================

Security context: 0x12004c41e6e9 <JSObject>
    0: builtin exit frame: captureStackTrace(this=0x12004c409719 <JSFunction Error (sfi = 0x1200db216e01)>,0x12001097d669 <JSFunction module.exports (sfi = 0x12001097d411)>,0x1200b931f771 <Error map = 0x1200eff4cbd9>,0x12004c409719 <JSFunction Error (sfi = 0x1200db216e01)>)

    1: /* anonymous */ [0x1200cd74f321] [/Users/rcooper/work/regression-qaa/node_modules/testcafe/lib/api/test-run-tracker...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10003c597 node::Abort() [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
 2: 0x10003c7a1 node::OnFatalError(char const*, char const*) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
 3: 0x1001ad575 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
 4: 0x100579242 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
 5: 0x10057bd15 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
 6: 0x100577bbf v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
 7: 0x100575d94 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
 8: 0x10058262c v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
 9: 0x1005826af v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
10: 0x10055f884 v8::internal::Factory::NewJSArray(v8::internal::ElementsKind, v8::internal::PretenureFlag) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
11: 0x10055fe2e v8::internal::Factory::NewJSArrayWithElements(v8::internal::Handle<v8::internal::FixedArrayBase>, v8::internal::ElementsKind, int, v8::internal::PretenureFlag) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
12: 0x100637551 v8::internal::Isolate::CaptureSimpleStackTrace(v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::FrameSkipMode, v8::internal::Handle<v8::internal::Object>) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
13: 0x100637c36 v8::internal::Isolate::CaptureAndSetSimpleStackTrace(v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::FrameSkipMode, v8::internal::Handle<v8::internal::Object>) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
14: 0x10025f987 v8::internal::Builtin_Impl_ErrorCaptureStackTrace(v8::internal::BuiltinArguments, v8::internal::Isolate*) [/Users/rcooper/.nvm/versions/node/v10.15.3/bin/node]
15: 0x269a5a6dbf9d
Abort trap: 6

Screenshots:

Steps to Reproduce:

I'm not sure if there's an easy way to reproduce this issue. As mentioned, I think 1 (or more) of our tests seems to cause a memory leak somehow, which is throwing this FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory.
As a suggested way to reproduce, create a set of tests (and copy these in to different Fixtures). Set one test to cause the OOM error and the log the test name at the start and end of each test. Run in the console and see if TestCafe hangs reporting on the faulty Fixture, while the remaining tests still continue to run.
Your Environment details:

testcafe version:                   1.6.0
node.js version:                    10.15.3
command-line arguments:  testcafe -c 3 chrome:headless
browser name and version: Chrome 78.0.3904.70
platform and version:          "macOS"

I'm afraid we cannot do much in the situation when the environment has limited memory. You can try to increase the memory size, avoid running tests concurrently, or divide your tests into several tasks.

My concern is that current state of the tests are misleading. Fixture 1 is displayed as the currently running Fixture, whereas multiple other Fixtures have completed during this time.
Not to mention that it appears tests are actually being triggered before TestCafe reports the Fixture starting. Would it be possible for TC to report to the console which test it's currently running?

Hi @rob4629
Recently, we've added the reportTestStart method to the TestCafe reporting subsystem.  The built-in reporters doesn't use this method.
Implement the custom reporter plugin, add the necessary logging into the reportTestStart and add the reporter to TestCafe.
See more information in the Reporter Plug help topic.