Quantization Aware Training + FeatherMap

If I wanted to use Quantization Aware Training (QAT) in conjunction with structured hashing, should I quantize before or after FeatherMap?
i.e, (before, intuitively seems correct to me):
# model has appropriate torch.quantization.DeQuantStub()
model = ResNet50()

model.train()
model.qconfig = torch.quantization.get_default_qat_qconfig('fbgemm')
model_fp32_prepared = torch.quantization.prepare_qat(model_fp32)

# feather map stuff
model_fp32_prepared = FeatherNet(model_fp32_prepared, compress=0.10)

# train loop

# eval stuff
model_fp32_prepared.eval()
model_int8 = torch.quantization.convert(model_fp32_prepared)
model_int8.deploy()

To be specific, I'm interested in the low-rank decomposition being INT8.

Yep your intuition is correct - ideally one would aim to quantize during training (or emulate). I would surmise that post-quantization would perform poorly on a model that's been compressed via structured multi-hashing (SMH) due to the nonlinear mapping of weights to the reduced weight matrices (V1 and V2).
In theory, there shouldn't be any issue doing both SMH and quantization - however I haven't dug into the implementation internals of QAT to confirm whether its compatible with FeatherMap out the box. Let me know how it goes!

On further thought, I think one should apply QAT after wrapping in FeatherMap. FeatherNet as a layer will just expose self.V1 and self.V2 as the weights to be updated, which should then be quarantined (or emulated quantization) and then trained. E.g, something like
base_model = ResNet50()
f_model = FeatherNet(base_model, compress=0.10)
# now apply quantization awareness to f_model and thus V1 and V2
# train
# evaluate and convert

Thank you for your reply!
How about evaluation: what order would it follow?

I'll also try comparing this method to iterative pruning ("To prune or not to prune", Zhu et al) and some dynamic sparse training techniques (RiGL, Evci et al. 2020).

Thank you for your reply!
How about evaluation: what order would it follow?

For accuracy and other metric evaluation you can make use of the GPU if you keep it in f_model.eval() mode. However, if you want to benchmark inference time, then you'd want to use f_model.deploy(). Presumably one would only need to actually go to reduced precision when deploying - if QAT can continue to emulate quantization during evaluation, I'd do that.

I'll also try comparing this method to iterative pruning ("To prune or not to prune", Zhu et al) and some dynamic sparse training techniques (RiGL, Evci et al. 2020).

Awesome. One of the cool things about FeatherMap is the ability to compound other compression methods. I'm very curious to see what kind of performance you might get compared to 'unstacked' compression methods.

For accuracy and other metric evaluation you can make use of the GPU if you keep it in f_model.eval() mode. However, if you want to benchmark inference time, then you'd want to use f_model.deploy(). Presumably one would only need to actually go to reduced precision when deploying - if QAT can continue to emulate quantization during evaluation, I'd do that.

I'm actually just interested in compressing the weights V_1, V_2. So I don't need to worry about eval? (model.state_dict should have V_1, V_2?).

Yes, the state_dict will save V1 and V2 as the weights, as well as the batchnorm layers


Great a PlantUML integration/exporter/viewer

Connect Umple to PlantUML:

Allow generation of PlantUML text, and hence viewing of UML diagrams, Latex and other PlantUML features:

See
http://modeling-languages.com/plantuml-textual-uml-online/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+ModelingLanguages-blog+%28Software+Modeling+Blog%29

Professor, maybe this tool can be used to build a link between UMPLE and Plantuml
UMPLE can generate java sources then this tool generates Plantuml scripts https://github.com/mirajp1/java2plantuml


Jobs migration: schema

Summary
Refactor jobs schema including decorators for swagger

Closing this one as Done. The final changes aiming for an MVP are included in #1265 , where we decided to store the full configuration in the Job schema, instead of only the configuration version.

Re-opening this issue as there are still some things to be fixed.
The current job.schema contains:

a property messageSent which is never used
properties datasetValidation and datasetsValidation. Only the second one is used, so we should remove the first one.


Support for Container.depends_on

It looks like this library doesn't yet support setting container dependencies:
https://cloud.google.com/run/docs/reference/rpc/google.cloud.run.v2#container shows a depends_on field but it's absent from https://cloud.google.com/python/docs/reference/run/latest/google.cloud.run_v2.types.Container. (Setting it via annotation is not an option because system annotations are rejected.)

The Cloud Run python client is automatically generated using the source protos located here. The requested feature is not yet released by the API team. See the source proto here. Please open an issue in the public issue tracker here request an ETA for when the feature will be released.

I'm going to transfer this issue to google-cloud-python as we're planning to move the code for google-cloud-run there in the next 1-2 weeks


Adding edge case tests for overlaps operators.

This adds a few edge case tests for the overlaps, overlaps before and overlaps after operators. These tests are related to the technical correction submitted to the specification here:
https://jira.hl7.org/browse/FHIR-46051
The author's guide is unclear about the intended interpretation of "overlaps before" and "overlaps after", whereas the reference and logical specification are both clear. These tests cover the edges of:

overlaps for equal intervals (true)
overlaps where the first interval starts at the end of the second (true)
overlaps before for equal intervals (false)
overlaps after for equal intervals (false)

These new tests have been run successfully on the Java engine (see Cooking with CQL session 83)


Transform to numpy arrays

Pytorch files are too platform specific. Instead, let the data be available as numpy arrays. This has the same amount of compression. In addition, we make a couple of name changes to more intuitive names.
Todo

[x] Convert files from pytorch to numpy
[x] Name changes: Call the displayed slates "slates" instead of "action", call the type of interaction (recs or search) "interaction_type" instead of "displayType". Remove ind2user and ind2item as they are scrambled anyways. Remove the popularity count in itemattr object.
[x] Update the dataloader
[ ] Update quickstart (wait until we also build the pip package?)

Pushed some of your changes @NegatioN . I now have two files: "datahelper.py" and "dataset_torch.py" which contain relevant code. The idea is that we can also add a "dataset_tensorflow.py" etc to reduce needed dependencies.
In addition to these I think it makes sense to add some evaluation functions, which I will write in pytorch-lightning as a first implementation. So maybe a dataset_lightning.py that can inherit the dataset from dataset_torch maybe. Is this reasonable or too complicated?

Pushed some of your changes @NegatioN . I now have two files: "datahelper.py" and "dataset_torch.py" which contain relevant code. The idea is that we can also add a "dataset_tensorflow.py" etc to reduce needed dependencies.

(I already wrote this in a comment, but:) I would aim for a numpy generator that people who don't use PyTorch could use for their own dataloaders (ex: TensorFlow?)

In addition to these I think it makes sense to add some evaluation functions, which I will write in pytorch-lightning as a first implementation.

Maybe writing the evaluation functions for the framework you're using first is the right call. But I think it's wise to put some effort into separating the eval functions into a self-contained format, so that they can be used in both Lightning/Pytorch/regular Python? I can provide more feedback on those things, or try to help out, if you sketch out how the function itself is supposed to look like :slightly_smiling_face:

So maybe a dataset_lightning.py that can inherit the dataset from dataset_torch maybe. Is this reasonable or too complicated?
The datasets look identical for Lightning and Pytorch, right? It's only the eval functions you're worried about?

Pushed some of your changes @NegatioN . I now have two files: "datahelper.py" and "dataset_torch.py" which contain relevant code. The idea is that we can also add a "dataset_tensorflow.py" etc to reduce needed dependencies.

(I already wrote this in a comment, but:) I would aim for a numpy generator that people who don't use PyTorch could use for their own dataloaders (ex: TensorFlow?)

In addition to these I think it makes sense to add some evaluation functions, which I will write in pytorch-lightning as a first implementation.

Maybe writing the evaluation functions for the framework you're using first is the right call. But I think it's wise to put some effort into separating the eval functions into a self-contained format, so that they can be used in both Lightning/Pytorch/regular Python? I can provide more feedback on those things, or try to help out, if you sketch out how the function itself is supposed to look like 🙂

So maybe a dataset_lightning.py that can inherit the dataset from dataset_torch maybe. Is this reasonable or too complicated?

The datasets look identical for Lightning and Pytorch, right? It's only the eval functions you're worried about?

I agree on all these points! But its out of scope for this PR I think. It also requires us to be a bit careful so that all functions and dataset splits are equivalent. So need to be worked on slightly separate. But we gone one step with moving data from torch to numpy. so next steps would be to also move code to be only np-dependent. :) numpy generators could a good way to "fix" this.


Remove Much of the HTTP API Surface Area

In accordance with https://github.com/kubernetes-incubator/service-catalog/issues/54, this patch attempts to only remove the API surface area. It won’t refactor any code.
Fixes https://github.com/kubernetes-incubator/service-catalog/issues/54

LGTM

LGTM (again).

LGTM


Added support for specifying tag name when downloading a list of pact…

… files.

Thanks for your work!

Sure, thanks for merging :-)


LiteDB goes Cross-Platform

There are probably many debug steps, which I'll remove later, just wanted to push my current version of it.
I removed the .NetCore Project (which was NetCore, UWP and .Net46), because the .Net45 Library can be also used in .Net46, UWP has its own Platform. And for NetCore, I tried to create a simple website and using any sample pcl project..... but after 5 hours of wasted time, I kinda decided to wait to support NetCore until its easier to understand what and why it really works. ( Need help :stuck_out_tongue_winking_eye:  )
Changes from master:

Crossplatform with all Tests!
Shell is an own Class / Library (in PCL)
FileDiskService and EncryptedDiskService merged together
Encryption fully working on each platform! (also compatible encryption settings)
LiteDatabaseFactory inorder to get an instance of ILiteDatabase
There is a LiteDBPlatform / TestPlatform for each Target (which handles the specific stuff)

Breaking changes:

need to have one call which sets the LiteDBPlatform.Instance
more use of the LiteDatabaseFactory (if this is too much of a break, I'll remove it again and let the Databases be created "the old way")

FYI:

#140 can be closed, cause Reflection.Emit is still be used on .Net35 / .Net45 (Desktop), Expression.Compile on all other PCL's
#195 still needs to be applied after merge
I readded some tests , in-order to see if all works as expected on that many platforms, so we maybe even need more tests
the ConcurrentTest still not working, I don't know if its due my changes, maybe I forgot something from the master
for the Tests on Mac I had to create a simple UI cause there isn't any test project available :sweat_smile:

Let the review begin!
Btw, we should also use Gitter / Slack to discuss more.

Hi @negue and @szurgot . I'm think how can we talk about this changes to cross plataform. This change are much more deep than @szurgot propose.
@negue, I do a quick check in your source and have some questions:

Are necessary LiteDatabaseFactory? Old way isn´t simple? I dont see devs using another implementation of ILiteDatabase than LiteDatabase or I'm not understand something...
You exposes some classes and create interfaces like IIndexService and DbEngine (class) that was internal only.
Did you remove shell from LiteDatabase class (Run method) but keeps LiteShell class inside LiteDB project?
I guess that you "copy" some files instead "link as reference", like in "LiteDB.Core\DbEngine\DbEngine"

I think that the easy way to create this cross plataform is have only 1 project with real files (using #if condition) and all others vs project are a linked sources.

@mbdavid A chat room could help this go faster (and probably not distract others). I set one up on my repo and sent the two of you an invite.
After working with my implementation, I have found there is room for improvement there... Let's discuss over in the gitter chat (I sent the invite to both of you)

Removed the LiteDatabaseFactory, it was before I merged those DiskServices into one
& 3. in LiteDB-Project the Shell is still in the same Project, in PCL its separated to a different one, cause not everbody needs to use the Shell. Also the Shell needs to have access to the DbEngine, thats why its public and not internal (due that the Shell is now in an own Project)
-> I could try to use "InternalsVisibleTo" if you want @mbdavid
Good catch, I wanted to have all files linked , fixed that in a commit


User types are broken.

The file appears to be loaded by binaryninja.

This should be fixed with #1193

I don't think this issue should have been closed.

This seems like it has since been fixed:


Define macros that expand to integers as constants

So in my header I have at least one hundred constants like:
#define X 5
#define Y 10
ETC. I would like it if Autopxd could translate them to Cython expressions.
I'm 99% sure this can be done like this:
cdef extern from "someheader.h":
cdef const int X, Y

@Keithcat1 I'm maintaining this project - merging pull requests and fixing bugs - but my focus is elsewhere. If you create a pull request, I will review it and can put out a new release with your changes.


message_events: Don't update data structures for unavailable msgs.

For messages which we don't have stored locally, we don't update
our data structures. This was actually our behaviour before
59e5f2d8fc48ce8b5cb27d97875c7db59a8032c9, which introduced this
bug.
Not doing this caused a major bug where we ran into errors
moving messages for topics for which we didn't have all
the messages available.

@ryanreh99 FYI, it would be great if you could have a chance to look at this before this gets merged.


SendGrid tool

Just "send an email"  just needs three args: to, subject, body
API Key auth, don't need oauth for this.

Moving to in progress


ci-health, plot: add support for --start-date flag

Adds a filter function looking at the result file path, and comparing
the date extracted to given start date. Also add parameter to signature
to have it.
Example: using newly introduced flag start-data as below
go run ./cmd/batch \
          --gh-token /home/dhiller/.tokens/github/kubevirt-bot/oauth \
          --path $(pwd)/output \
          --mode plot \
          --target-metric merged-prs \
          --log-level debug \
        --start-date $(date -d "- 3 months" -I)

would only plot the graph for last 3 months.

/remove-lifecycle stale

@brianmcarey I don't want to have this lying around at my branch - can we merge it, so that I don't have to always rebase?


ImportError: cannot import name '_shutdown_cluster'

Any idea what my problem might be?
thanks
andy
from caspanda.bear import CasPanda

cl = CasPanda()
session = cl.connect()
ImportError                               Traceback (most recent call last)
 in ()
----> 1 from caspanda.bear import CasPanda
2
3 cl = CasPanda()
4 session = cl.connect()
/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/caspanda-<IP_ADDRESS>-py3.4.egg/caspanda/bear.py in ()
9 from caspanda.metabear import ColumnMeta, KeyspaceMeta, TableMeta
10 from cassandra.query import dict_factory
---> 11 from cassandra.cluster import _shutdown_cluster
12
13 from caspanda.bamboo import CassandraFrame
ImportError: cannot import name '_shutdown_cluster'

Are you using Cassandra 3.0+?

@aaronbenz I have got the same issue here - am using Cassandra 3.7.0
many thanks in advance

I'm getting the same error message. Any help will be much appreciated!

Same error message, thanks in advance.

_shutdown_cluster is no longer a function in cassandra.cluster. This can be solved by removing the line which is importing it, which is unused anyway: https://github.com/aaronbenz/caspanda/blob/master/caspanda/bear.py#L11

Can anybody create a PR?

Resolved on form #18


Reminder: Re-try VRF update in PR 245

Link: https://github.com/novifinancial/akd/pull/245
CI was broken, let's retry this pr again later

Closed by https://github.com/novifinancial/akd/pull/255


add charter communications to wireline subcategory

may need to update to Spectrum, but leaving as is for now

Build failed because of:
Duplicate item: Dream Home Improvement LLC Charter Communications at path Wireless & Wireline / Wireline
Duplicate item: Dream Home Improvement LLC Charter Communications at path LF Edge Member Company / Premier


my.core.serialize: simplejson support, more types

I added a couple extra checks to the default function,
serializing datetime, dates and dataclasses (incase
orjson isn't installed)
(copied from below)
if orjson couldn't be imported, try simplejson
This is included for compatibility reasons because orjson
is rust-based and compiling on rarer architectures may not work
out of the box
as an example, I've been having issues getting it to install
on my phone (termux/android)
unlike the builtin JSON modue which serializes NamedTuples as lists
(even if you provide a default function), simplejson correctly
serializes namedtuples to dictionaries
this just gives another option to people, simplejson is pure python
so no one should have issues with that. (and often they already have
it installed since its used in older python projects for compatability)
orjson is still way faster, so still preferable if its easy and
theres a precompiled build for your architecture (which there typically is)
If you're ever running this with simplejson installed and not orjson,
its pretty easy to tell as the JSON styling is different; orjson has
no spaces between tokens, simplejson puts spaces between tokens. e.g.
simplejson: {"a": 5, "b": 10}
orjson: {"a":5,"b":10}

As dataclasses and datetime is what caused most serialization errors previously, this also just means that hpi query should work out of the box for most use cases (though NamedTuples are sorta broken as they're serialized to lists instead; the warning is sent telling you to install orjson/simepljson), even if the user didn't install the optional dependencies

that's awesome, thanks! Wondering, what are you trying to achieve on the phone, just potentially running same tools as on the computer?

Yeah, pretty much
I have a few scripts here that use hpi query to check how much water/food I've eaten in the last day
Other stuff like my albums, have a few aliases to let me know what music to listen to next, which use hpi query

As an update, orjson now successfully builds on termux (sometime after https://github.com/ijl/orjson/issues/360)


If "files to include/exclude" is hidden, then it shouldn't take effect

In the search panel, if "files to include/exclude" is hidden, then it shouldn't take effect.

I disagree, it's often useful to have a bunch of patterns in there, or toggle the button, but hide it to get it out of the way.


Embeds show only the first image from the posts with multiple pics

It would've been nice to see few pics in the attachment when a user is trying to embed a pixiv link that contains not only 1 image. Now it works in the way that we will see only the first pic from the pixiv post.
From the technical point I understand that sometimes there might be too many images in the post to embed, but what if we limit it to 10, for example? or showing them in a few different blocks in parts?

Discord does not currently have the ability to embed multiple images in one embed from a website and I feel that stitching together images will provide a worse user experience in general.
If you'd like to embed other images in a post you can add /<index> to the end of the url.


Modify the documentation for the installation of the developer version…

… to avoid authentification issue while cloning the GitHub repository

@ipa-bak thanks for the hint. Unfortunately, the GitCloneTask doesn't allow to set by default the "HTTP" protocol.
But the user can select it manually while installing, I updated the documentation and added an explanatory screenshot.

This PR updates only the documentation. It can be merged without further review.


api.user_follows returns "full_name" not "first_name" and/or "last_name"

The JSON returned for https://api.instagram.com/v1/users/3/follows?access_token=ACCESS_TOKEN is "full_name" but the example within http://instagram.com/developer/endpoints/relationships/#get_users_follows returns  "first_name" and/or "last_name"
The API documentation at http://instagram.com/developer/endpoints/relationships/#get_users_follows needs to be amended to reflect "full_name" and not "first_name" and/or "last_name"

Thank you for reporting this issue and appreciate your patience. We've  notified the core team for an update on this issue. We're looking for a response within the next 30 days or the issue may be closed.


🐞 Bug: the navbar is not responsive and not working properly

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug
the notification message is not responsive in mobile view also the messages are not deleting after viewinf the message
Steps To Reproduce

go to circuitverse
check the navbar

Expected Behavior
message-container should get deleted after view and should be responsive
Screenshots
No response
Device Information [optional]
- OS:
- Browser:
- version:

Additional context
No response
Are you working on this issue?
Yes

i am working on this issue

Please describe the issue properly. Add screenshots where the responsiveness breaks.
It's not expected behaviour that notification will be deleted after view, it will be marked as read only.

in this picture the notification is not visible so we can add vertical scroll bar so that both issues can be resolve
but we can remove the message after some time or we can remove old message that are viewed previously

in this picture the notification is not visible so we can add vertical scroll bar so that both issues can be resolve
but we can remove the message after some time or we can remove old message that are viewed previously

You can add a vertical scrollbar. Update issue according to that.
User should be able to check old notification anyway. If you check any platform (Youtube, Facebook, Github, etc..) which have notification, will show you all notifications, it's not going to be deleted. It will be better if we can update the UI to show read & unread messages seperately.

so do i have to update issue
yes thats a nice suggestion that we can make read and unread saparatley

so do i have to update issue
yes thats a nice suggestion that we can make read and unread saparatley

Update this issue
You can raise another issue for that. That will be required design prototype (in figma) to start working on

I would like to work on this issue @tanmoysrt

I would like to work on this issue @tanmoysrt

Hi @upsaurav12 you can start working on the issue

Since this issue is still open, can i work on it?


Update bpm limit to Infinity

Hi :wave:
I was fiddling around with Orca and wanted to play with creating some extratone music. I removed the upper bpm limit to be able to do so. This turned my Korg Volca Kick into an oscillator :cheese:
Thanks for the awesome program! Never made music before, and find this really intuitive and exciting as a scriptkid :smile:

I'm glad to hear you like Orca :) This is a nice hack, but I cannot merge it. I've capped it to limit possible errors for new users. But this is what I tell everyone who needs extra speed to do.

Thanks for the response, helps me understand :) looking forward to play more and hear more Orca-music :bird:


TLS options for LDAP

Context
In order to use this container with my LDAP server, I have to use TLS with SASLauth and Dovecot. Unfortunately, I don't see any envvar that would allow me to set the desired options.
Expected Behavior
- SASLAUTHD_LDAP_START_TLS=yes set ldap_start_tls: yes in /etc/saslauthd.conf
- SASLAUTHD_LDAP_TLS_CHECK_PEER=no set ldap_tls_check_peer: no in /etc/saslauthd.conf
- DOVECOT_TLS_REQUIRE_CERT=never set tls_require_cert = never in /etc/dovecot/dovecot-ldap.conf.ext
Generally, I would like to be able to use more ldap envvars without mounting a file in the container (I'm using Kubernetes and it's not very practical), like those :
https://blog.sys4.de/cyrus-sasl-saslauthdconf-man-page-en.html
http://ftp.ics.uci.edu/pub/centos0/ics-custom-build/BUILD/dovecot-1.1.7/doc/dovecot-ldap-example.conf
Possible Fix
Add more envvars to configure LDAP for dovecot and salsauth.

It is merged


update golang.org/x/text to v0.3.5

Issue: https://github.com/open-cluster-management/backlog/issues/9890
Go version v0.3.3 of the x/text package fixes a vulnerability in encoding/unicode that could lead to the UTF-16 decoder entering an infinite loop, causing the program to crash or run out of memory.
This sets all golang.org/x/text versions to v0.3.5, which includes the fix.

/approve


[Preview deployement] Update profile - Extra space above "Update your parent entitity..."

In the Preview deployment there is an extra 60px of space on one of the forms, however the code on main when run locally does not exhibit this behavior.
We will need to revisit this issue once the updates from the preview branch are all merged into main

Preview
Local

I think this one is resolved by #862 @shindigira?


[Feature] Add opportunity to connect border leafs to super spine in 5 clos design

Issue Type

[x] Role enhancement

Is your feature request related to a problem? Please describe.
In this time there aren't feature to connect border leaf switches to super spine in 5clos design. We could do it only through l3_edge & dci extension.
Describe the solution you'd like
I would like to have opportunity to connect border leaf switches right to super spine inside 5 clos design.
Describe alternatives you've considered
Now, I created some fake pod for border_leafs with empty spine session like:
# blank for edge_leafs
spine:
  bgp_as: 777
  nodes: []

and describe two switches with empty spines:
# In DC1 we define all variables on specific node / node_group
l3leaf:
  defaults:
    bgp_as: 65555
    virtual_router_mac_address: 00:1c:73:00:dc:01
  node_groups:
    # EDGE LEAF SWITCHES INSIDE DC
    EDGE_LEAFS:
      platform: vEOS-LAB
      spines: []
      uplink_to_spine_interfaces: []
      bgp_as: 65399
      mlag_interfaces: [ Ethernet5, Ethernet6 ]
      mlag_dual_primary_detection: true
      spanning_tree_mode: mstp
      spanning_tree_priority: 4096
      evpn_role: client
      evpn_route_servers: [ DC1-SUPER-SPINE1, DC1-SUPER-SPINE2 ]
      filter:
        tenants: [Tenant_A]
        tags: [zone4]
      nodes:
        DC1-ESW1:
          id: 1
          mgmt_ip: <IP_ADDRESS>/24
          mac_address: '0c:4d:cb:fb:6b:00'
          spine_interfaces: []
        DC1-ESW2:
          id: 2
          mgmt_ip: <IP_ADDRESS>/24
          mac_address: '0c:4d:cb:15:f3:00'
          spine_interfaces: []

After that I use section named l3_edge to connect border leafs to super spines:
l3_edge:
  p2p_links_ip_pools:
    #pool-leaf: <IP_ADDRESS>/24
    #pool-spine: <IP_ADDRESS>/24
    #pool-super-spine: <IP_ADDRESS>/24
    pool_dc1_esw1_dc1_sspine1: <IP_ADDRESS>/31
    pool_dc1_esw1_dc1_sspine2: <IP_ADDRESS>/31
    pool_dc1_esw2_dc1_sspine1: <IP_ADDRESS>/31
    pool_dc1_esw2_dc1_sspine2: <IP_ADDRESS>/31
  p2p_links_profiles:
    generic-profile:
      mtu: 1499
      bfd: false
      ptp_enable: true
  p2p_links:
    - nodes: [ DC1-ESW1, DC1-SUPER-SPINE1 ]
      id: 1
      interfaces: [ Ethernet9, Ethernet9 ]
      as: [ 65399, 65100 ]
      ip_pool: pool_dc1_esw1_dc1_sspine1
      profile: generic-profile
      include_in_underlay_protocol: true
      #macsec_profile: MACSEC_PROFILE
    - nodes: [ DC1-ESW1, DC1-SUPER-SPINE2 ]
      id: 1
      interfaces: [ Ethernet10, Ethernet10 ]
      as: [ 65399, 65100 ]
      ip_pool: pool_dc1_esw1_dc1_sspine2
      profile: generic-profile
      include_in_underlay_protocol: true
      #macsec_profile: MACSEC_PROFILE
    - nodes: [ DC1-ESW2, DC1-SUPER-SPINE1 ]
      id: 1
      interfaces: [ Ethernet10, Ethernet10 ]
      as: [ 65399, 65100 ]
      ip_pool: pool_dc1_esw2_dc1_sspine1
      profile: generic-profile
      include_in_underlay_protocol: true
      #macsec_profile: MACSEC_PROFILE
    - nodes: [ DC1-ESW2, DC1-SUPER-SPINE2 ]
      id: 1
      interfaces: [ Ethernet9, Ethernet9 ]
      as: [ 65399, 65100 ]
      ip_pool: pool_dc1_esw2_dc1_sspine2
      profile: generic-profile
      include_in_underlay_protocol: true
      #macsec_profile: MACSEC_PROFILE

Additional context
I think this feature is important in 5clos designs because now we haven't option to connect border leafs to superspine directly like similar l3_leafs.
in this design there aren't proper path for north to south. Every traffic from another DC or from internet will go through this pod (where border leafs are connected) and utilise links for this pods. This is not correct design in my opinion.

@inetman28 thanks you for submitting these issues. Indeed there are various topology options that are valid, and we have been investigating a way to provide flexibility for less strict topologies. As we develop a solution, we will tag this issue and would appreciate your feedback!

@inetman28
In PR #932 we are refactoring a lot of the logic, and I believe it should also address this ask. Feel free to try it out before we merge. If you write the hostnames of the super-spines in the spines variable on your border leafs it should just work. And you should no longer need an empty spine: configuration or l3_edge at all. Remember that you should keep your border leafs in a separate POD, since IP adressing is taken from the underlay pool for that POD.
Let me know how it goes. (Please tag me)

Tested #932 with this data. Required some adjustments to not fail on missing spine: key, but it is working now.
# L3LS Fabric Values - update these values with caution,
# SOME CHANGES COULD BE DISRUPTIVE.

pod_name: DC1_BORDER-POD

# Point to Point Network Summary range, assigned as /31 for each
# uplink interfaces
# Assign range larger then total [spines * total potential leafs * 2]
underlay_p2p_network_summary: <IP_ADDRESS>/24

# IP address range for evpn loopback for all switches in fabric,
# assigned as /32s
# Assign range larger then total spines + total leafs switches
overlay_loopback_network_summary: <IP_ADDRESS>/24

# VTEP VXLAN Tunnel source loopback IP for leaf switches, assigned in /32s
# Assign range larger then total leaf switches
vtep_loopback_network_summary: <IP_ADDRESS>/24
vtep_vvtep_ip: <IP_ADDRESS>/32

# Enable vlan aware bundles
vxlan_vlan_aware_bundles: false

# This is referring to super-spines in this case, since the l3leaf are uplinked to super-spines.
# Used for calculating IP adresses.
max_spines: 4

# Leaf switch groups
# A maximum of two nodes can form a leaf group
# When two nodes are in a leaf group this will automatically form mlag pair

l3leaf:
  defaults:
    # virtual router mac for VNIs assigned to Leaf switches
    # format: xx:xx:xx:xx:xx:xx
    virtual_router_mac_address: 00:1c:73:00:dc:01
    platform: vEOS-LAB
    spines: [DC1-SUPER-SPINE1, DC1-SUPER-SPINE2]
    uplink_to_spine_interfaces: [Ethernet2, Ethernet3]
    spanning_tree_priority: 4096
    spanning_tree_mode: mstp
    evpn_route_servers: ['DC1-RS1', 'DC1-RS2']
    mlag: false
  node_groups:
    DC1_BORDER_POD_LEAF1:
      bgp_as: 65301
      nodes:
        DC1-BORDER-POD-LEAF1A:
          id: 1
          mgmt_ip: <IP_ADDRESS>/24
          spine_interfaces: [Ethernet12, Ethernet12]
        DC1-BORDER-POD-LEAF1B:
          id: 2
          mgmt_ip: <IP_ADDRESS>/24
          spine_interfaces: [Ethernet13, Ethernet13]

@ClausHolbechArista Hi
is it work on version 2.1.1 or is it going only at 2.2.0?
I see that it was merged to devil branch but for me is not still working

Hello @inetman28
It is currently part of the devel branch and will be ship in version 2.2 as per milestone of this PR.
AVD v2.2 should land around mid-june if everything goes well.
Kind Regards

yes, it works on devil branch.
Thank you.


[Resource Explorer]: Properties selection is not clearing after clicking on Add button in the Unmodeled section

Properties selection is not clearing after clicking on the Add button in the Unmodeled s.ection
Steps to reproduce:
https://github.com/awslabs/iot-app-kit/assets/142866907/46d5a799-d9bd-4c00-b7af-b56656169f35

Add the Widget
Go to the Unmodeled tab in Resource Explorer
Select the properties and click the add button
After adding properties to the widget, the selection list is not clearing

Expected behavior
Clear the properties selection after the user clicks the "Add" button in the "Unmodeled" section in the Resource Explorer, similar to the "Modeled" section.

Added the reset selection list in the Unmodeled section and opened the PR
Updated this ticket to not include the Dynamic assets section. cause properties selection is working differently in the Dynamic assets section and looks like keeping the selection list is expected.


Support scroll to top when paging changes.

Original feature request: https://discord.com/channels/1219902517304098836/1248850650457182292/1254699706861355039
Support both boolean and paging settings for custom control.

Added a few comments.

As a follow up to this it would also be good to let users set some defaults in settings (similar to the default paging settings).

Okay, I will add a setting for this soon.

Done.


Failed to parse the private key file of Ed25519!

try  to parser the private key file of ed25519, and fail ,  how to fix it ?

Private key file content:
-----BEGIN PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,b619a06a16b7b7a6436579f06a14f45e

QmysBFzoMkgvVTM39kvHjkKhcBjK6PVMZ6a/taF44ZXeOl3t5DUp4EWxyfs8htng
tjsKIb0yKJigIZGrCeHROQ==
-----END PRIVATE KEY-----

exception info:
org.bouncycastle.openssl.PEMException: problem parsing PRIVATE KEY: java.lang.IllegalArgumentException: failed to construct sequence from byte[]: corrupted stream - out of bounds length found: 108 >= 64
at org.bouncycastle.openssl.PEMParser$PrivateKeyParser.parseObject(Unknown Source)
at org.bouncycastle.openssl.PEMParser.readObject(Unknown Source)
....

core code:
`        PrivateKeyInfo pki;
try (PEMParser pemParser = new PEMParser(new StringReader(encryptPrivateKey))) {
Object o = pemParser.readObject();
if (o instanceof PKCS8EncryptedPrivateKeyInfo) {
PKCS8EncryptedPrivateKeyInfo epki = (PKCS8EncryptedPrivateKeyInfo) o;
JcePKCSPBEInputDecryptorProviderBuilder builder
= new JcePKCSPBEInputDecryptorProviderBuilder().setProvider("BC");
InputDecryptorProvider idp = builder.build(password.toCharArray());
pki = epki.decryptPrivateKeyInfo(idp);
     } else if (o instanceof PEMEncryptedKeyPair) {
         PEMEncryptedKeyPair epki = (PEMEncryptedKeyPair) o;
         PEMKeyPair pkp = epki.decryptKeyPair(new BcPEMDecryptorProvider(password.toCharArray()));
         pki = pkp.getPrivateKeyInfo();
     } else {
         log.error("Invalid encrypted private key class:{}", o.getClass().getName());
         return null;
     }
     JcaPEMKeyConverter converter = new JcaPEMKeyConverter().setProvider("BC");
     return converter.getPrivateKey(pki);
 } catch (Exception e) {
     log.error("loadPrivateKey exception:{}", ExceptionUtil.getBriefExceptionStackTrace(e));
 }
 return null;`

Usually this would mean the password is wrong, it's possibly an issue with the PBE algorithm, but given how long it has been since we have had a report like this, I'd check the password.

Usually this would mean the password is wrong, it's possibly an issue with the PBE algorithm, but given how long it has been since we have had a report like this, I'd check the password.

I don't think it was the wrong password that caused the problem；The exception occurs when this line of code is executed：Object o = pemParser.readObject();   When I parse private key files generated by other algorithms (RSA/CE) in the same way, it's OK;  I  think the class (PEMParser ) does not support parsing  the ed25519  private key encrypted in the preceding format;

It should decrypt to a private key object, as in a PKCS#8 PrivateKeyInfo. If there was an issue with the fact it was ed25519 then the problem would show up trying to interpret the contents of the PrivateKeyInfo

@dghgit It's not an ENCRYPTED PRIVATE KEY, it's a PRIVATE KEY using PEM encryption. While KeyPairParser supports the PEM encryption (returning a PEMEncryptedKeyPair), PrivateKeyParser doesn't have a pathway to e.g. a "PEMEncryptedPrivateKey".

Oh yeah... think it must be a slow day today. Let me think about this further.

Oh yeah... think it must be a slow day today. Let me think about this further. @chen0411 would you provide a sample with a password.

Yes, password is: Vjvyhfngz0MCUs$kwOF0
and can use openssl command to change the private key to ENCRYPTED PRIVATE KEY.
for example:

I think this is an example of what I was describing in #1350 - this is a PKCS8 PrivateKeyInfo encrypted with the 'traditional' PEM method.
It looks like openssl can read such keys (although I've yet to find a way to get it to generate them), but BC just sees it as an unencrypted PKCS8 key, and the subsequent error messages are it trying to parse the random data as a PrivateKeyInfo.
@chen0411 - was the original encrypted key generated by BC or something else?


Define admin

[Enter feedback here]
Please state which admin role can act as an admin.  It just says Admins can do this & that.  I have Office Admin and Global Reader roles, but I do not have the Global Admin role.  My admin roles do not let me manage Office add-ins.
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e62dfa25-e420-38e4-ff47-05cca62d14ec
Version Independent ID: 5ea03ce0-38b9-e005-a430-e98442e2ec83
Content: Deploy add-ins in the admin center - Microsoft 365 admin
Content Source: microsoft-365/admin/manage/manage-deployment-of-add-ins.md
Service: o365-administration
GitHub Login: @Sirkkuw
Microsoft Alias: sirkkuw

@giterdn Thank you for your feedback. We have made changes to the document via PR#https://github.com/MicrosoftDocs/microsoft-365-docs/pull/3947
Thanks
Sri

@giterdn Thank you for your feedback. We have made changes to the document via PR#https://github.com/MicrosoftDocs/microsoft-365-docs/pull/3947
Thanks
Sri

@msbemba Thank you very much for the contribution and updating the documentation with PR. @giterdn Hope this update is helpful for you. Thanks for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


[WIP] Final branch for 4.1 upgrade

Nowhere near ready to be used. But when it will be, you should be able to require 4.1@dev in your composer. Nice and clean.
Features
MUST

[x] provide support for Laravel 7
[x] https://github.com/Laravel-Backpack/CRUD/pull/2308 "Fetch" operation - much easier to add AJAX selects
[x] https://github.com/Laravel-Backpack/CRUD/pull/2308 CRUDable models have identifiableAttribute, so that developers don't have to define a column to show to the user every time - in most cases it's "name", "title" or something like that, so we can guess it;
[x] https://github.com/Laravel-Backpack/CRUD/pull/2311 "relationship" field type and column type, with "Add new" button next to Selects, so users can add a new connected entry right then and there then select it (aka "on the fly" adding)
[x] #2448  ability to add links to column items
[x] https://github.com/Laravel-Backpack/CRUD/pull/2592 solution for adding custom CSS inside widgets
[ ] https://github.com/Laravel-Backpack/CRUD/pull/2266 "repeatable" field type

SHOULD

[x] drop support for PHP 7.1
[x] https://github.com/Laravel-Backpack/CRUD/pull/2356 ability to change button options at the end of Create and Update forms
[x] https://github.com/Laravel-Backpack/CRUD/pull/2408 configurable web middleware
[x] https://github.com/Laravel-Backpack/CRUD/pull/2479 setting to hide the search bar inside the List operation
[x] #2521 #2502 remove laravel/helpers dependency
[x] https://github.com/Laravel-Backpack/CRUD/issues/2430 remove venturecraft/revisionable dependency - add it as a step for using the Revisions operation;
[x] https://github.com/Laravel-Backpack/CRUD/issues/2538 remove intervention/image dependency - add it as a step for using the Image field type;
[x] #2587 remove barryvdh/laravel-elfinder from the installation process; create a separate package with the Backpack interface for it, and instruct people to require and install it if they want to use (a) the main elfinder view, (b) the browse field type or (c) the browse_multiple field type;
[x] https://github.com/Laravel-Backpack/CRUD/pull/2509 list operation "Reset" button to clear local storage;
[ ] https://github.com/Laravel-Backpack/CRUD/pull/2513 fluent syntax for working with Columns, Fields, Buttons, Filters
[x] https://github.com/Laravel-Backpack/CRUD/pull/2599 fluent syntax for working with Widgets
[x] https://github.com/Laravel-Backpack/CRUD/pull/2596 basic chart widgets - pie chart and line/bar chart;
[x] https://github.com/Laravel-Backpack/CRUD/pull/2601 rename fields wrapperAttributes to wrapper for consistency;
[x] https://github.com/Laravel-Backpack/CRUD/pull/2615 new column, relationship, to match the relationship field;
[x] https://github.com/Laravel-Backpack/CRUD/pull/2615 new columns, relationship_count, that shows how many related items there are (like array_count, but for relationships);
[x] https://github.com/Laravel-Backpack/CRUD/pull/2477 Delete operation can now respond with custom notification messages/colors;
[x] https://github.com/Laravel-Backpack/CRUD/pull/2356 SaveActions (buttons at the end of Create and Update forms) can now be easily customized or replaced;
[ ] make it possible to easily extend the List operation, to show a list of entries from another perspective (ex: different columns, some filters already applied, different number of entries paginated, etc) - would be useful for repeated exports (ex: previous months' invoices);

COULD

[x] drop support for Laravel 5.8
[x] https://github.com/Laravel-Backpack/CRUD/pull/2610 the Show operation now pushes a new SaveAction inside Create and Update operations - "Save and Preview";
[x] https://github.com/Laravel-Backpack/CRUD/pull/2554 update default icon font (Line Awesome) to latest version for more icons; change syntax so that it no longer conflicts with font awesome if they're both loaded on page;
[ ] https://github.com/Laravel-Backpack/CRUD/pull/1987 "layout" field type
[ ] https://github.com/Laravel-Backpack/CRUD/issues/2374 separate "image" and "cropper" fields
[ ] remove guzzlehttp/guzzle in favour of zttp, or Laravel's default HTTP Client (Laravel 7.0 only);
[ ] https://github.com/Laravel-Backpack/CRUD/issues/2120 Remove App\Models\BackpackUser, in favour of using the default user model App\User;

WON'T

drop support for Laravel 6

Todo:

[x] new folder for docs
[x] upgrade guide in the docs
[ ] tag release
[x] new features page
[ ] send newsletter

A new inspection was created.

A new inspection was created.

A new inspection was created.

A new inspection was created.

Merged https://github.com/Laravel-Backpack/CRUD/pull/2550 into this branch. CrudTrait is now split into multiple smaller traits.

4.1 now has a folder inside docs, with a first draft for the Release Notes and Upgrade Guide.
All changes to the docs should be made there, on the master branch, but using PRs.

A new inspection was created.

A new inspection was created.

A new inspection was created.

A new inspection was created.

Merged https://github.com/Laravel-Backpack/CRUD/pull/2553 into 4.1. This updates and upgrades NPM dependencies
Fixes:

cross-env https://github.com/Laravel-Backpack/CRUD/pull/2422
sass-loader https://github.com/Laravel-Backpack/CRUD/pull/2497
laravel-mix https://github.com/Laravel-Backpack/CRUD/pull/2334
tinymce https://github.com/Laravel-Backpack/CRUD/pull/2331
source sans pro https://github.com/Laravel-Backpack/CRUD/pull/2098
line awesome https://github.com/Laravel-Backpack/CRUD/pull/2273

Also:

font-awesome fix inside iconpicker field https://github.com/Laravel-Backpack/CRUD/pull/2363
we're using the la la-home not fa fa-home syntax for Line Awesome Icons now https://github.com/Laravel-Backpack/CRUD/pull/2554

A new inspection was created.

A new inspection was created.

A new inspection was created.

A new inspection was created.

A new inspection was created.

The inspection completed: 2943 Issues, 121 Patches

A new inspection was created.

The inspection completed: 442 Issues, 82 Patches

A new inspection was created.

The inspection completed: 445 Issues, 82 Patches

A new inspection was created.

The inspection completed: 445 Issues, 82 Patches

A new inspection was created.

The inspection completed: 445 Issues, 82 Patches

A new inspection was created.

The inspection completed: 445 Issues, 80 Patches

A new inspection was created.

The inspection completed: 442 Issues, 80 Patches

A new inspection was created.

A new inspection was created.

The inspection completed: 442 Issues, 80 Patches

The inspection completed: 442 Issues, 80 Patches

A new inspection was created.

The inspection completed: 434 Issues, 78 Patches

A new inspection was created.

A new inspection was created.

The inspection completed: 428 Issues, 78 Patches

The inspection completed: 429 Issues, 78 Patches

A new inspection was created.

The inspection completed: 427 Issues, 77 Patches

A new inspection was created.

The inspection completed: 428 Issues, 77 Patches

A new inspection was created.

A new inspection was created.

A new inspection was created.

The inspection completed: 427 Issues, 77 Patches

The inspection completed: 431 Issues, 77 Patches

A new inspection was created.

The inspection completed: 442 Issues, 78 Patches

A new inspection was created.

The inspection completed: 442 Issues, 78 Patches

A new inspection was created.

The inspection completed: 442 Issues, 78 Patches

A new inspection was created.

The inspection completed: 442 Issues, 78 Patches

A new inspection was created.

The inspection completed: 443 Issues, 78 Patches

A new inspection was created.

The inspection completed: 442 Issues, 78 Patches

A new inspection was created.

The inspection completed: 443 Issues, 78 Patches

A new inspection was created.

The inspection completed: 441 Issues, 78 Patches

A new inspection was created.

The inspection completed: 441 Issues, 78 Patches

A new inspection was created.

The inspection completed: 438 Issues, 76 Patches

A new inspection was created.

The inspection completed: 439 Issues, 76 Patches

A new inspection was created.

The inspection completed: 439 Issues, 76 Patches

A new inspection was created.

The inspection completed: 438 Issues, 76 Patches

A new inspection was created.

The inspection completed: 472 Issues, 80 Patches

A new inspection was created.

The inspection completed: 471 Issues, 79 Patches

A new inspection was created.

The inspection completed: 471 Issues, 79 Patches

A new inspection was created.

The inspection completed: 471 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

The inspection completed: 474 Issues, 79 Patches

A new inspection was created.

A new inspection was created.

The inspection completed: 487 Issues, 78 Patches

The inspection completed: 487 Issues, 78 Patches

A new inspection was created.

The inspection completed: 487 Issues, 78 Patches

A new inspection was created.

The inspection completed: 487 Issues, 78 Patches

A new inspection was created.

The inspection completed: 487 Issues, 78 Patches


Added working openapi specification

Fixes #336
Co-authored-by: Lukas Hennies<EMAIL_ADDRESS>Co-authored-by: felixmhofmann<EMAIL_ADDRESS>

Todo: test it

added files necessary for testing in pr below
https://github.com/united-manufacturing-hub/united-manufacturing-hub/pull/466
see recommended changes in  #466  (pr targets this branch)
tldr:

minor changes on api and tested generation
generation was succesfull (enough)
the heavy use of anyOf in the message structure will induce some problems for ceratain generators.

added linux generator scripts

maybe the index.htmls should be excluded from sonar cloud as they are not manually created, but i do not know / have the rights to do that


Condition for response added to skip, added test cases

Inserted the lines if existing_user_email and existing_user_nickname then return 400 to ensure proper handling. Adjusted the tests accordingly. Additionally, included the is_professional attribute in UserResponse.model_construct for both the get_user and update_user APIs.

Incorporated logic to check if skip is less than zero or if limit is not greater than zero, in which case it returns a 400 error. Made corresponding updates to the test cases to ensure coverage for this behavior.-DONE


Drop support for --once

Once merged, this removes the support for the legacy --once flag for actions/runner.
Upgrade your GHES to 3.3 or greater if you use our runner image tagged with latest, or fork our dockerfile and use a custom runner image if you absolutely need the --once flag! But note and beware that --once has never been officially supported even by actions/runner.
Ref #1196

I"ve locally tested this. Merging.


Faster canonicalization of affine expressions

CVXPY canonicalizes affine expression trees by constructing a sparse matrix for each linear operator, and recursively multiplying these sparse matrices together (see cvxpy/cvxcore/src/). This is fine for small problems, but it becomes a bottleneck (both time and memory) for problems with many constraints, and also for problems with large variables/data. For example, matrix stuffing in cvxcore is the bottleneck for cvxpy/tests/test_benchmarks.py:TestBenchmarks.test_diffcp_sdp_example.
Parallelizing the matrix stuffing (across expression trees) is one way to speed things up (#706); however, this can cause OOMs for large problems (e.g., using 12 processes to canonicalize the SDP in the benchmark mentioned above, with n=300, p=100, OOM'd my 2012 MBP).
Another way might be to do something similar to JuMP, which I believe maintains the coefficients for each variable along the way, using elementwise operations to update them as needed. @angeris brought JuMP's approach to my attention. One way to see if this is promising would be to implement the SDP example in JuMP and compare the model construction time to CVXPY.

Assigning @angeris, but contributions and ideas from others are still welcome.

I would be careful about taking an elementwise approach to tracking symbolic affine expressions. The main thing that can go wrong is the speed of adding large expressions together. If you have x and y as two scalar expressions, representing linear combinations of 1000 symbolic terms each, then computing the symbolic expression x + y can be expensive. If you then have large matrices containing such expressions, then performing these symbolic scalar additions for every element of the resulting matrix can take a long time.
I implemented such an approach for a separate project I work on, where a symbolic ScalarExpression is an operator-overloaded dictionary that maps a hashable ScalarVariable symbol to a coefficient. The performance was very good for small problems, but it didn't scale well. There were some tricks to cope with this, but I ended up using a different approach for large problems. I don't think my solution would be appropriate for cvxpy, but I'll post the idea nonetheless.
pre-compiled affine operators
Suppose I needed to constrain a symbolic expression M1 @ y1 + M2 @ y2 >= 0, where y1 and y2 are symbolic expressions over variables x. The normal computations here would be ...

Perform a symbolic matrix vector product M1 @ y1; store the result in z1.
Perform a symbolic matrix vector product M2 @ y2; store the result in z2.
Perform a symbolic vector-vector additon, z1 + z2; store the result in z3.
From the symbolic expression z3, generate A and b so that z3 >= 0 can be represented as A @ x + b >= 0.

A pre-compiled approach would be to define a single function that takes in arguments M1, y1, M2, y2 and returns the data A, b for the desired affine expression, without performing a single symbolic matrix-vector product.
Pre-compiled affine operators have the following upsides

They can be orders of magnitude faster than methods which require intervening symbolic computation.
They are simple to write (and often must be written when building models in languages like C++)
They don't need to be used when prototyping small models. You only need to bother with them when performance is critical.

Pre-compiled affine operations have the following downsides

Small changes to the form of a given affine operator may require a separate "pre-compiled" implementation.
The output of a pre-compiled affine operator cannot be used for subsequent symbolic computation.

Unrelated to my previous post:
When cvxcore multiplies the sparse matrices together, does it run the standard dynamic programming algorithm to find the optimal order of multiplication?
https://en.wikipedia.org/wiki/Matrix_chain_multiplication
That algorithm could also be modified to account for the sparsity factor of a given matrix (i.e. a 10,000-by-10,000 matrix with 100 entries is "smaller" than a dense 500-by-500 matrix).

Some quick notes on why this might be a good idea (and, potentially, a terrible one)
The initial problem came up when attempting to optimize a (relatively small)
problem with a large number of reshapes and concatenations. It seemed that JuMP
would heavily outperform CVXPY in a similar problem of this form, but it was
somewhat unclear why this would be the case.
When profiling this, @akshayka found a few things: (a) is_constant and many similar methods were being recomputed, even though most atoms are essentially immutable (fixed by 9b59d045ef8) and (b) that a huge chunk of time was being spent by cvxcore in generating the correct affine transformations. This is particularly surprising, since most of the affine expressions present in the program are mostly concerned with stacking and concatenating matrices and vectors (note that the size of the final program was moderate: 4000 variables with ~4000 very sparse SOC constraints, generated essentially by a tridiagonal matrix). For more information, take a peek at this profile dump of the code.
Another somewhat large chunk of time seems to be spent converting and checking the validity of sparse matrices (e.g., calls to coo.py:_check, from SciPy) and similar things. This, at least to me, appears to be that there are a large number of unnecessary conversions between formats or that too many operations are being performed on these matrices, even though most of them will have a very small number of nonzero entries.
So, this generates some ideas, which might be interesting to try and implement (not all of them independent of each other):

Full elementwise construction of build_matrix either during the construction of the parse tree or after.
Elementwise construction of all (and only) the sub-indexing or stacking expressions, etc. More specifically, the types of operations that generate submatrices of the identity or kronecker products thereof, and leaving the rest for cvxcanon.
Working on a DP-type approach as @rileyjmurray suggested in the second post. This part is fairly unclear to me, as (unlike in the sense case) it seems that different sparsity patterns will yield very different matrix-build times. I'm also not sure if this is being done in the dense case? @SteveDiamond ?

Number 1 is essentially JuMP's approach. This yields somewhat similar times in the current CVXPY implementation, for example, when naïvely translating test_benchmarks.py:TestBenchmarks.test_diffcp_sdp_example to Julia, as in this gist. Of course, the CVXPY tr function could be a bit smarter about computing the correct result, but that's another point. There are a few questions, though: (a) Python isn't Julia in terms of element-wise speed, of course, so we may not be able to get away with this, unlike JuMP. But the question remains: what is the performance impact for most problems? This is sort-of addressed by suggestion 2, which attempts a "best-of-both-worlds" approach, but then (b) what should the interface look like between cvxcore and cvxpy? Almost all of these matrix-building operations are currently being sent off to cvxcore, but now we have some transformations done within cvxpy and others within cvxcore. Unsure of what the implications in terms of structure are, but with the decreased communication cost and likely reduced number of operations, it's worth thinking about.
Suggestion 3 strikes me as somewhat sensible, since it's quite possible that a simple heuristic will do quite well in the sparse case, but it's not clear to me what such a heuristic would look like (the suggestion above may be good, but I really don't have much intuition about what the results might be). I also know that @SteveDiamond is currently working on cvxcore, so I'll leave this for him to think about :)
Anyways, I likely forgot a few things on this topic, so I may come back and edit this later comment, but this is, at the moment, where we are at.
Additionally, apologies for the partial response, @rileyjmurray : I haven't thought enough about your first suggestion to fully appreciate it, though it's likely that @akshayka and @SteveDiamond have some thoughts?

@rileyjmurray, we don't find an optimal order for matrix multiplications. This does sound like a good idea. Using the number of nonzeros in a sparse matrix as a heuristic might work? In any case it would be easy to experiment with this, and with various heuristics. This, plus modifying cvxcore to process constraints in parallel, should yield pretty good speed-ups for problems with many constraint. As @angeris mentioned, cvxcore is currently being rewritten to extract an affine map from parameters to problem data, so we should probably wait until that rewrite is complete before spending too much time on performance optimization.
Options (1) and (2) would be interesting to try out, but I think we'd need substantial evidence that elementwise construction is a good idea before investing the time needed to actually implement it robustly.

@akshayka I agree that we should wait on modifications to the matrix multiplications until cvxcore settles down. As a start I think implementing the basic matrix-chain-multiplication algorithm is likely to improve performance. The sparse case will be harder, but I actually think it would make an excellent research project. It's simple enough to state that an undergrad (or team of undergrads) could work on it, and I bet there are applications beyond cvxcore-type computations that would benefit from an understanding of this problem. Do you or @SteveDiamond know of any undergrads in Boyd's lab that might want to take on this project? If not I can ask around Caltech (the problem is that I'll be away from Caltech this summer, so I couldn't supervise someone here).

The sparse case will be harder, but I actually think it would make an excellent research project. It's simple enough to state that an undergrad (or team of undergrads) could work on it, and I bet there are applications beyond cvxcore-type computations that would benefit from an understanding of this problem.

That does sound like a fun project! We'll ask around and see if anyone's interested. If we can't find anyone, we'll let you know.

@akshayka did you end up finding someone to work on the sparse matrix-chain multiplication problem?

Nope, unfortunately.

I'm interested in what I think is the same problem in Convex.jl. I have been exploring rewriting some of the internals (ref https://github.com/jump-dev/Convex.jl/pull/393), and have come to the problem of composing many affine transformations together. I also came across the idea of solving the matrix chain problem, but the issue for me is that the transformations are generally not just a matrix-multiplication but also a vector addition (i.e. a sequence of x -> A*x +b). Multiplying those out gives a combinatorial explosion of terms, but left-to-right and right-to-left one can accumulate into a matrix and a vector result, i.e. something like
for (new_matrix, new_vector) in transformations
    vec = vec + matrix * new_vector
    matrix = matrix * new_matrix
end

I was wondering how CVXPY avoids having the vector addition to deal with in order to consider matrix chain multiplication; I tried looking at the source but unfortunately I am almost illiterate in C++ and python.
P.S. My solution for now is to always go left-to-right, because for the objective function, the result will be 1-dimensional, so you are essentially performing a sequence of matvecs instead of matmuls, which is probably optimal. For constraints, of course, the output could have a higher dimension, so this might not be the right choice in that case.

Hi @ericphanson cvxpy uses matrix multiplications of the form
[A b][x]
[0 1][1]

to represent affine transformations. Not sure if that was your question.

Yep it was! Thanks, I might try that for Convex.jl too then.

@sbarratt , since you expressed interest in this problem.

Compiling problems that are (very close to) standard forms should be fast.
https://github.com/cvxpy/cvxpy/issues/1521

I am eager to attempt an implementation of the matrix-chain-multiplication problem, as I occasionally experience the performance implications of this issue myself, and I am keen to expand my understanding of scientific computing in general.
There are a few aspects that remain unclear to me:

I noticed that @phschiele has shifted this issue from "Help Wanted" to "In Progress", which I presume is connected to cvxpy 1.3.0's new Scipy canonicalization backend. If that is the case, could you please explain the relationship between this new backend and the issue at hand?
Considering this new backend, on which areas should one concentrate to achieve the most significant performance improvements?
Would you be able to recommend any academic literature that might assist in addressing this issue? My familiarity with sparse matrices is limited, and I lack a strong intuition for the performance characteristics of handling sparse matrices.

I appreciate your assistance!

Hi @mvanaltvorst, the newer backends no longer rely (purely) on matrix multiplication chains, so this approach is probably not the first optimization we would try to speed up the canonicalization further. Closing this issue as we will likely open new ones discussing methods to improve the new backend implementations.


Use nhsbsa.nhs.uk domain for the playbook

What
Currently the playbook is hosted and accessed via GitHub pages i.e. https://nhsbsa.github.io/nhsbsa-digital-playbook/
It would be good to explore the possibility of using an nhsbsa.nhs.uk domain too e.g. https://ddat.nhsbsa.nhs.uk or https://ddat-playbook.nhsbsa.nhs.uk or https://digital-data-technology.nhsbsa.nhs.uk/
Why
Adding the playbook to an NHSBSA domain should help increase visibility of the playbook and be a lot easier to remember when people need to return when guidance is needed.

@yantantether do you know who could help with this please? Just not great when sharing the link round and still references just digital... even if we can 'just' update that part to DDaT for now perhaps? Don't know who has access to do the work sorry so feel free to tag others

Hi @torihutchinson, we should discuss this on internal channels as there's some security implications.


Updates to build system to check that DPDK is built successfully

Moved DPDK to top level because its location in tests was confusing

Looks good to me


Build fails on i386: error[E0412]: cannot find type u512 in this scope

Summary
error[E0412]: cannot find type `u512` in this scope
     --> /wrkdirs/usr/ports/lang/rustpython/work/target/release/build/unicode_names2-fc074b82fe4b3d20/out/generated.rs:13939:43
      |
13939 | pub static PHRASEBOOK_OFFSETS2: &'static [u512] = &[
      |                                           ^^^^ not found in this scope

unicode_names2's last release is 1.2.2, but some other tag  is used, which might be a cause of this problem.
log
rust-1.77.0
FreeBSD 13.2

Rust target used in the log is i686-unknown-freebsd.
I tried to build both unicode_names2 master and RustPython main with the target, RustPython main + unicdoe_names2 1.2.2 with the target, but not reproducible.
How do I reproduce the problem?


feat: Lemmas about (extChartAt I x).target being open if we're boundaryless

(extChartAt I x).source is always open, so there are nice lemmas like

isOpen_extChartAt_source
extChartAt_source_mem_nhds
extChartAt_source_mem_nhds'

(extChartAt I x).target fails to be open near manifold boundary points, but we recover the nice lemmas when I.Boundaryless.

bors r+


Update Remove-PnPEventReceiver.md

Type

[ ] Bug Fix
[ ] New Feature
[ ] Sample
[x] Documentation

Related Issues?
n/a
What is in this Pull Request ?

Removed WhatIf and Confirm parameter descriptions
Changed Identity parameter Required to False
Standardised the spelling of "eventReceivers"
Settled on "web" instead of "site" to avoid confusion

Thank you @PowershellScripts !


”Corunca” is listed in the drop-down menu

Description:
"Corunca" is listed in the drop-down menu alongside its corresponding county and commune, according to the law.
Precondition:
The website is up an running.

Step 1
Type "Corunca" in the search bar.

Expected results
The location is listed in the drop-down menu as "Comuna Corunca, MUREȘ".


Data set "in preparation" but task is not

See e.g. this task: https://www.openml.org/t/146542
The according data set currently is flagged as "in preparation"

The data set is still in preparation. Would be nice if it could be activated :)

done
On Thu, Aug 10, 2017 at 6:34 PM Heidi Seibold<EMAIL_ADDRESS>wrote:

The data set is still in preparation. Would be nice if it could be
activated :)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/openml/website/issues/149#issuecomment-321487981, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABpQV_0E6ZxPQilRauWWPpa2JCBgal29ks5sWsCMgaJpZM4NxdGy
.

--
Thank you,
Joaquin


ci: fix semrel

:tada: This PR is included in version 1.0.0-alpha.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Updated steps and other info in PC Modding

Adds "THIS IS THE RECOMMENDED DOWNGRADER." in BSManager's section in Installers. Also updates the Beta Downgrading steps to include the 1.34.2_legacy option. Also fixes 1 typo somewhere.

somehow this included my other PR's commits that was merged in today, whoopsie

closing this now, this PR has weird issues
and creating new PR


istio-proxy  error  HPE_INVALID_HEADER_TOKEN

(NOTE: This is used to report product bugs:
To report a security vulnerability, please visit https://istio.io/about/security-vulnerabilities
To ask questions about how to use Istio, please visit https://discuss.istio.io)
Bug description
[ ] Docs
[ ] Installation
[X] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
Expected behavior
proxy service
Steps to reproduce the bug
':authority', 'soper.bookinfo:8080'
':path', '/jsoper-building-prov/building/queryBuildingInfoByBuildingId'
':method', 'GET'
'user-agent', 'curl/7.29.0'
'accept', '/'
2021-02-08T15:10:38.983827Z     debug   envoy http      [C83580][S2746416191019570570] request end stream
2021-02-08T15:10:38.983899Z     trace   envoy http      [C83580][S2746416191019570570] decode headers called: filter=0x55b6ff36f8c0 status=0
2021-02-08T15:10:38.983925Z     debug   envoy filter    AuthenticationFilter::decodeHeaders with config
policy {
peers {
mtls {
mode: PERMISSIVE
}
}
}
skip_validate_trust_domain: true
2021-02-08T15:10:38.983932Z     debug   envoy filter    [C83580] validateX509 mode PERMISSIVE: ssl=false, has_user=false
2021-02-08T15:10:38.983934Z     debug   envoy filter    Payload has not peer authentication data
2021-02-08T15:10:38.983937Z     trace   envoy filter    Got request path /jsoper-building-prov/building/queryBuildingInfoByBuildingId
2021-02-08T15:10:38.983939Z     debug   envoy filter    Set principal from peer:
2021-02-08T15:10:38.983942Z     debug   envoy filter    Origin authenticator succeeded
2021-02-08T15:10:38.983947Z     debug   envoy filter    Saved Dynamic Metadata:
2021-02-08T15:10:38.983950Z     trace   envoy http      [C83580][S2746416191019570570] decode headers called: filter=0x55b6fea545a0 status=0
2021-02-08T15:10:38.983954Z     trace   envoy http      [C83580][S2746416191019570570] decode headers called: filter=0x55b6fea55da0 status=0
2021-02-08T15:10:38.983960Z     trace   envoy http      [C83580][S2746416191019570570] decode headers called: filter=0x55b6feed5a40 status=0
2021-02-08T15:10:38.983964Z     trace   envoy http      [C83580][S2746416191019570570] decode headers called: filter=0x55b6feed5ce0 status=0
2021-02-08T15:10:38.983969Z     debug   envoy router    [C83580][S2746416191019570570] cluster 'inbound|8080|http|soper.bookinfo.svc.cluster.local' match for URL '/jsoper-building-prov/building/queryBuildingInfoByBuildingId'
2021-02-08T15:10:38.984014Z     debug   envoy router    [C83580][S2746416191019570570] router decoding headers:
':authority', 'soper.bookinfo:8080'
':path', '/jsoper-building-prov/building/queryBuildingInfoByBuildingId'
':method', 'GET'
':scheme', 'http'
'user-agent', 'curl/7.29.0'
'accept', '/'
'x-forwarded-proto', 'http'
'x-request-id', 'bc205b8a-f57a-904f-8446-dc2881cc49ab'
'x-b3-traceid', '2c39f7fb80da227f72078418731bd866'
'x-b3-spanid', '72078418731bd866'
'x-b3-sampled', '1'
2021-02-08T15:10:38.984026Z     debug   envoy pool      queueing request due to no available connections
2021-02-08T15:10:38.984029Z     debug   envoy pool      creating a new connection
2021-02-08T15:10:38.984057Z     debug   envoy client    [C83581] connecting
2021-02-08T15:10:38.984062Z     debug   envoy connection        [C83581] connecting to <IP_ADDRESS>:8080
2021-02-08T15:10:38.984132Z     debug   envoy connection        [C83581] connection in progress
2021-02-08T15:10:38.984143Z     trace   envoy http      [C83580][S2746416191019570570] decode headers called: filter=0x55b6fecede00 status=1
2021-02-08T15:10:38.984147Z     trace   envoy http      [C83580] parsed 142 bytes
2021-02-08T15:10:38.984154Z     trace   envoy connection        [C83580] socket event: 2
2021-02-08T15:10:38.984159Z     trace   envoy connection        [C83580] write ready
2021-02-08T15:10:38.984161Z     trace   envoy connection        [C83581] socket event: 2
2021-02-08T15:10:38.984163Z     trace   envoy connection        [C83581] write ready
2021-02-08T15:10:38.984166Z     debug   envoy connection        [C83581] connected
2021-02-08T15:10:38.984168Z     debug   envoy client    [C83581] connected
2021-02-08T15:10:38.984172Z     debug   envoy pool      [C83581] attaching to next request
2021-02-08T15:10:38.984175Z     debug   envoy pool      [C83581] creating stream
2021-02-08T15:10:38.984181Z     debug   envoy router    [C83580][S2746416191019570570] pool ready
2021-02-08T15:10:38.984193Z     trace   envoy connection        [C83581] writing 334 bytes, end_stream false
2021-02-08T15:10:38.984200Z     trace   envoy connection        [C83581] write ready
2021-02-08T15:10:38.984219Z     trace   envoy connection        [C83581] write returns: 334
2021-02-08T15:10:38.984226Z     trace   envoy connection        [C83581] socket event: 2
2021-02-08T15:10:38.984228Z     trace   envoy connection        [C83581] write ready
2021-02-08T15:10:38.986256Z     trace   envoy connection        [C83581] socket event: 3
2021-02-08T15:10:38.986264Z     trace   envoy connection        [C83581] write ready
2021-02-08T15:10:38.986267Z     trace   envoy connection        [C83581] read ready. dispatch_buffered_data=false
2021-02-08T15:10:38.986274Z     trace   envoy connection        [C83581] read returns: 593
2021-02-08T15:10:38.986279Z     trace   envoy connection        [C83581] read error: Resource temporarily unavailable
2021-02-08T15:10:38.986283Z     trace   envoy http      [C83581] parsing 593 bytes
2021-02-08T15:10:38.986285Z     trace   envoy http      [C83581] message begin
2021-02-08T15:10:38.986292Z     trace   envoy http      [C83581] completed header: key=Access-Control-Allow-Origin value=*
2021-02-08T15:10:38.986301Z     trace   envoy http      [C83581] completed header: key=Access-Control-Allow-Credentials value=true
2021-02-08T15:10:38.986306Z     trace   envoy http      [C83581] completed header: key=Access-Control-Allow-Methods value=POST, GET, PUT, OPTIONS, DELETE, PATCH
2021-02-08T15:10:38.986309Z     trace   envoy http      [C83581] completed header: key=Access-Control-Expose-Headers value=*
2021-02-08T15:10:38.986314Z     trace   envoy http      [C83581] completed header: key=Access-Control-Allow-Headers value=Authentication,Origin, X-Requested-With, Content-Type, Accept, x-access-token,Access-Control-Allow-Origin
2021-02-08T15:10:38.986318Z     trace   envoy http      [C83581] completed header: key=Cache-Control value=no-cache
2021-02-08T15:10:38.986350Z     debug   envoy client    [C83581] protocol error: http/1.1 protocol error: HPE_INVALID_HEADER_TOKEN
2021-02-08T15:10:38.986354Z     debug   envoy connection        [C83581] closing data_to_write=0 type=1
2021-02-08T15:10:38.986356Z     debug   envoy connection        [C83581] closing socket: 1
2021-02-08T15:10:38.986377Z     debug   envoy client    [C83581] disconnect. resetting 1 pending requests
2021-02-08T15:10:38.986382Z     debug   envoy client    [C83581] request reset
2021-02-08T15:10:38.986384Z     trace   envoy main      item added to deferred deletion list (size=1)
2021-02-08T15:10:38.986389Z     debug   envoy router    [C83580][S2746416191019570570] upstream reset: reset reason connection termination
2021-02-08T15:10:38.986416Z     debug   envoy http      [C83580][S2746416191019570570] Sending local reply with details upstream_reset_before_response_started{connection termination}
2021-02-08T15:10:38.986429Z     trace   envoy http      [C83580][S2746416191019570570] encode headers called: filter=0x55b6ff337e50 status=0
2021-02-08T15:10:38.986434Z     trace   envoy http      [C83580][S2746416191019570570] encode headers called: filter=0x55b6ff4084b0 status=0
Version (include the output of istioctl version --remote and kubectl version --short and helm version --short if you used Helm)
1.7.6
How was Istio installed?
iop
Environment where the bug was observed (cloud vendor, OS, etc)
Additionally, please consider running istioctl bug-report and attach the generated cluster-state tarball to this issue.
Refer cluster state archive for more details.

Without proxy ，service is ok.  With proxy, return error 503.

Can you show the response with curl -v? Probably the service is returning invalid http response.,

I mean without envoy. So we can see what the service returns

2021-02-08T15:10:40.499954Z     trace   envoy router    [C0][S9714646464900352072] proxying 14655 bytes
2021-02-08T15:10:40.499967Z     trace   envoy http2     [C83556] dispatched 290 bytes
2021-02-08T15:10:40.499974Z     trace   envoy connection        [C83556] writing 14664 bytes, end_stream false
2021-02-08T15:10:40.499980Z     trace   envoy http2     [C83556] sent frame type=0
2021-02-08T15:10:40.499989Z     trace   envoy connection        [C83556] socket event: 2
2021-02-08T15:10:40.499992Z     trace   envoy connection        [C83556] write ready
2021-02-08T15:10:40.500168Z     trace   envoy connection        [C83556] ssl write returns: 14664
2021-02-08T15:10:40.500348Z     trace   envoy connection        [C83556] socket event: 3
2021-02-08T15:10:40.500353Z     trace   envoy connection        [C83556] write ready
2021-02-08T15:10:40.500356Z     trace   envoy connection        [C83556] read ready. dispatch_buffered_data=false
2021-02-08T15:10:40.500366Z     trace   envoy connection        [C83556] ssl read returns: 30
2021-02-08T15:10:40.500371Z     trace   envoy connection        [C83556] ssl read returns: -1
2021-02-08T15:10:40.500374Z     trace   envoy connection        [C83556] ssl read 30 bytes
2021-02-08T15:10:40.500378Z     trace   envoy http2     [C83556] dispatching 30 bytes
2021-02-08T15:10:40.500381Z     trace   envoy http2     [C83556] about to recv frame type=8, flags=0
2021-02-08T15:10:40.500384Z     trace   envoy http2     [C83556] recv frame type=8
2021-02-08T15:10:40.500386Z     trace   envoy http2     [C83556] about to recv frame type=6, flags=0
2021-02-08T15:10:40.500389Z     trace   envoy http2     [C83556] recv frame type=6
2021-02-08T15:10:40.500391Z     trace   envoy http2     [C83556] dispatched 30 bytes
2021-02-08T15:10:40.500394Z     trace   envoy http2     [C83556] about to send frame type=6, flags=1
2021-02-08T15:10:40.500396Z     trace   envoy http2     [C83556] send data: bytes=17
2021-02-08T15:10:40.500399Z     trace   envoy connection        [C83556] writing 17 bytes, end_stream false
2021-02-08T15:10:40.500403Z     trace   envoy http2     [C83556] sent frame type=6
2021-02-08T15:10:40.500408Z     trace   envoy connection        [C83556] socket event: 2
2021-02-08T15:10:40.500410Z     trace   envoy connection        [C83556] write ready
2021-02-08T15:10:40.500437Z     trace   envoy connection        [C83556] ssl write returns: 17
[2021-02-08T15:10:38.983Z] "GET /jsoper-building-prov/building/queryBuildingInfoByBuildingId HTTP/1.1" 503 UC "-" "-" 0 95 2 - "-" "curl/7.29.0" "bc205b8a-f57a-904f-8446-dc2881cc49ab" "soper.bookinfo:8080" "<IP_ADDRESS>:8080" inbound|8080|http|soper.bookinfo.svc.cluster.local <IP_ADDRESS>:36852 <IP_ADDRESS>:8080 <IP_ADDRESS>:36414 - default
2021-02-08T15:10:40.695309Z     debug   envoy filter    original_dst: New connection accepted
2021-02-08T15:10:40.695338Z     debug   envoy filter    tls inspector: new connection accepted
2021-02-08T15:10:40.695347Z     trace   envoy filter    tls inspector: recv: 142
2021-02-08T15:10:40.695356Z     debug   envoy filter    http inspector: new connection accepted
2021-02-08T15:10:40.695360Z     trace   envoy filter    http inspector: recv: 142
2021-02-08T15:10:40.695367Z     trace   envoy filter    http inspector: http_parser parsed 74 chars, error code: 0
2021-02-08T15:10:40.695372Z     trace   envoy filter    http inspector: done: true
2021-02-08T15:10:40.695434Z     debug   envoy conn_handler      [C83583] new connection
2021-02-08T15:10:40.695454Z     trace   envoy connection        [C83583] socket event: 3
2021-02-08T15:10:40.695460Z     trace   envoy connection        [C83583] write ready
2021-02-08T15:10:40.695464Z     trace   envoy connection        [C83583] read ready. dispatch_buffered_data=false
2021-02-08T15:10:40.695474Z     trace   envoy connection        [C83583] read returns: 142
2021-02-08T15:10:40.695486Z     trace   envoy connection        [C83583] read error: Resource temporarily unavailable
2021-02-08T15:10:40.695492Z     trace   envoy filter    Alpn Protocol Not Found. Expected istio-peer-exchange, Got
2021-02-08T15:10:40.695512Z     trace   envoy http      [C83583] parsing 142 bytes
2021-02-08T15:10:40.695519Z     trace   envoy http      [C83583] message begin
2021-02-08T15:10:40.695525Z     debug   envoy http      [C83583] new stream
2021-02-08T15:10:40.695545Z     trace   envoy http      [C83583] completed header: key=User-Agent value=curl/7.29.0
2021-02-08T15:10:40.695555Z     trace   envoy http      [C83583] completed header: key=Host value=soper.bookinfo:8080
2021-02-08T15:10:40.695561Z     trace   envoy http      [C83583] onHeadersCompleteBase
2021-02-08T15:10:40.695564Z     trace   envoy http      [C83583] completed header: key=Accept value=/
2021-02-08T15:10:40.695571Z     trace   envoy http      [C83583] Server: onHeadersComplete size=3
2021-02-08T15:10:40.695580Z     trace   envoy http      [C83583] message complete
2021-02-08T15:10:40.695585Z     trace   envoy connection        [C83583] readDisable: disable=true disable_count=0 state=0 buffer_length=142

Can you show the response with curl -v? Probably the service is returning invalid http response.,

I mean the output of curl, bypassing envoy. So we can see wehat headers the server is returning. Or what headers the client is sending. One of them is probably invalid

curl cmd
[root@busybox-7898ddd945-srlhn /]# curl soper.bookinfo:8080/jsoper-building-prov/building/queryBuildingInfoByBuildingId -v
About to connect() to soper.bookinfo port 8080 (#0)
Trying <IP_ADDRESS>...
Connected to soper.bookinfo (<IP_ADDRESS>) port 8080 (#0)
GET /jsoper-building-prov/building/queryBuildingInfoByBuildingId HTTP/1.1
User-Agent: curl/7.29.0
Host: soper.bookinfo:8080
Accept: /
< HTTP/1.1 503 Service Unavailable
< content-length: 95
< content-type: text/plain
< date: Tue, 09 Feb 2021 01:37:25 GMT
< server: istio-envoy
< x-envoy-decorator-operation: soper.bookinfo.svc.cluster.local:8080/*
<
Connection #0 to host soper.bookinfo left intact
upstream connect error or disconnect/reset before headers. reset reason: connection termination
curl <IP_ADDRESS>
[root@soper-79bb557566-hjm2v ~]# curl <IP_ADDRESS>:8080/jsoper-building-prov/building/queryBuildingInfoByBuildingId -v
About to connect() to <IP_ADDRESS> port 8080 (#0)
Trying <IP_ADDRESS>...
","success":false}
proxy log
Proxy log is :
2021-02-08T15:10:40.499954Z trace envoy router [C0][S9714646464900352072] proxying 14655 bytes
2021-02-08T15:10:40.499967Z trace envoy http2 [C83556] dispatched 290 bytes
2021-02-08T15:10:40.499974Z trace envoy connection [C83556] writing 14664 bytes, end_stream false
2021-02-08T15:10:40.499980Z trace envoy http2 [C83556] sent frame type=0
2021-02-08T15:10:40.499989Z trace envoy connection [C83556] socket event: 2
2021-02-08T15:10:40.499992Z trace envoy connection [C83556] write ready
2021-02-08T15:10:40.500168Z trace envoy connection [C83556] ssl write returns: 14664
2021-02-08T15:10:40.500348Z trace envoy connection [C83556] socket event: 3
2021-02-08T15:10:40.500353Z trace envoy connection [C83556] write ready
2021-02-08T15:10:40.500356Z trace envoy connection [C83556] read ready. dispatch_buffered_data=false
2021-02-08T15:10:40.500366Z trace envoy connection [C83556] ssl read returns: 30
2021-02-08T15:10:40.500371Z trace envoy connection [C83556] ssl read returns: -1
2021-02-08T15:10:40.500374Z trace envoy connection [C83556] ssl read 30 bytes
2021-02-08T15:10:40.500378Z trace envoy http2 [C83556] dispatching 30 bytes
2021-02-08T15:10:40.500381Z trace envoy http2 [C83556] about to recv frame type=8, flags=0
2021-02-08T15:10:40.500384Z trace envoy http2 [C83556] recv frame type=8
2021-02-08T15:10:40.500386Z trace envoy http2 [C83556] about to recv frame type=6, flags=0
2021-02-08T15:10:40.500389Z trace envoy http2 [C83556] recv frame type=6
2021-02-08T15:10:40.500391Z trace envoy http2 [C83556] dispatched 30 bytes
2021-02-08T15:10:40.500394Z trace envoy http2 [C83556] about to send frame type=6, flags=1
2021-02-08T15:10:40.500396Z trace envoy http2 [C83556] send data: bytes=17
2021-02-08T15:10:40.500399Z trace envoy connection [C83556] writing 17 bytes, end_stream false
2021-02-08T15:10:40.500403Z trace envoy http2 [C83556] sent frame type=6
2021-02-08T15:10:40.500408Z trace envoy connection [C83556] socket event: 2
2021-02-08T15:10:40.500410Z trace envoy connection [C83556] write ready
2021-02-08T15:10:40.500437Z trace envoy connection [C83556] ssl write returns: 17
[2021-02-08T15:10:38.983Z] "GET /jsoper-building-prov/building/queryBuildingInfoByBuildingId HTTP/1.1" 503 UC "-" "-" 0 95 2 - "-" "curl/7.29.0" "bc205b8a-f57a-904f-8446-dc2881cc49ab" "soper.bookinfo:8080" "<IP_ADDRESS>:8080" inbound|8080|http|soper.bookinfo.svc.cluster.local <IP_ADDRESS>:36852 <IP_ADDRESS>:8080 <IP_ADDRESS>:36414 - default
2021-02-08T15:10:40.695309Z debug envoy filter original_dst: New connection accepted
2021-02-08T15:10:40.695338Z debug envoy filter tls inspector: new connection accepted
2021-02-08T15:10:40.695347Z trace envoy filter tls inspector: recv: 142
2021-02-08T15:10:40.695356Z debug envoy filter http inspector: new connection accepted
2021-02-08T15:10:40.695360Z trace envoy filter http inspector: recv: 142
2021-02-08T15:10:40.695367Z trace envoy filter http inspector: http_parser parsed 74 chars, error code: 0
2021-02-08T15:10:40.695372Z trace envoy filter http inspector: done: true
2021-02-08T15:10:40.695434Z debug envoy conn_handler [C83583] new connection
2021-02-08T15:10:40.695454Z trace envoy connection [C83583] socket event: 3
2021-02-08T15:10:40.695460Z trace envoy connection [C83583] write ready
2021-02-08T15:10:40.695464Z trace envoy connection [C83583] read ready. dispatch_buffered_data=false
2021-02-08T15:10:40.695474Z trace envoy connection [C83583] read returns: 142
2021-02-08T15:10:40.695486Z trace envoy connection [C83583] read error: Resource temporarily unavailable
2021-02-08T15:10:40.695492Z trace envoy filter Alpn Protocol Not Found. Expected istio-peer-exchange, Got
2021-02-08T15:10:40.695512Z trace envoy http [C83583] parsing 142 bytes
2021-02-08T15:10:40.695519Z trace envoy http [C83583] message begin
2021-02-08T15:10:40.695525Z debug envoy http [C83583] new stream
2021-02-08T15:10:40.695545Z trace envoy http [C83583] completed header: key=User-Agent value=curl/7.29.0
2021-02-08T15:10:40.695555Z trace envoy http [C83583] completed header: key=Host value=soper.bookinfo:8080
2021-02-08T15:10:40.695561Z trace envoy http [C83583] onHeadersCompleteBase
2021-02-08T15:10:40.695564Z trace envoy http [C83583] completed header: key=Accept value=/
2021-02-08T15:10:40.695571Z trace envoy http [C83583] Server: onHeadersComplete size=3
2021-02-08T15:10:40.695580Z trace envoy http [C83583] message complete
2021-02-08T15:10:40.695585Z trace envoy connection [C83583] readDisable: disable=true disable_count=0 state=0 buffer_length=142

Expires : -1
This is not a valid header (space in key). Fix your app. if you cannot fix your application, you can downgrade the protocol to tcp and avoid envoy HTTP parsing (https://preliminary.istio.io/latest/docs/ops/configuration/traffic-management/protocol-selection/)

Expires : -1
This is not a valid header (space in key). Fix your app. if you cannot fix your application, you can downgrade the protocol to tcp and avoid envoy HTTP parsing (https://preliminary.istio.io/latest/docs/ops/configuration/traffic-management/protocol-selection/)

I edit svc name prefix tcp- ,this is ok, but this is not my target . I want to http

In pod ,curl <IP_ADDRESS> is ok ,but curl svc is error. You means Expires Cause problems.

Expires : -1
This is not a valid header (space in key). Fix your app. if you cannot fix your application, you can downgrade the protocol to tcp and avoid envoy HTTP parsing (https://preliminary.istio.io/latest/docs/ops/configuration/traffic-management/protocol-selection/)

Thk u very much. I fix app(delete space in key) , there is no error.


127 test for device root certificate validity

Description
Adds the ability to check if a firewall is affected by the following emergency notification affecting device root certificates:
https://live.paloaltonetworks.com/t5/next-generation-firewall/urgent-action-required-pan-os-certificate-expiration-advisory/td-p/565682#:~:text=Essentially%2C the root and default,etc.)%20is%20configured%2C%20potentially
Motivation and Context
Customers running certain combinations of features, software and content could be affected by this issue. This test allows us to do a readiness check to see if they are affected, essentially giving the users the ability to mark or report on devices that are impacted.
How Has This Been Tested?
Tested against devices running 10.2.6 (fixed software) and older, unfixed versions
Types of changes

New feature (non-breaking change which adds functionality)

Checklist

[X] I have updated the documentation accordingly.
[X] I have read the CONTRIBUTING document.
[X] I have added tests to cover my changes if appropriate.
[X] All new and existing tests passed.

@FoSix @alperenkose
This PR introduces the first "health check" test along with an additional new convienience method "run_health_checks" that runs in the same manner as readiness tests.
We can expand the health check library over time in response to newly published issues and as a way to get clients to test their environments and check if they need to upgrade.

:tada: This PR is included in version 0.3.1 :tada:
The release is available on PyPI and GitHub release

Posted by semantic-release bot


Some features

Hello! You project is awesome. I think you need to add some useful features:

Parameter names with low line and string as value Convolution(in_channels=n_dim);
Some "x N" symbol near module that define how many times it's repeated;
Displaying a complex module name as a parameter.

Hi @mlajtos
I totally agree with Dimonovez, it is an incredible tool for making quick yet complex sketches!
Could you please update the macos-app with the latest display-parameters-branch? I tried updating the files inside the app manually but failed since I'm not familiar with building such apps.
Thanks for your work!

These are good suggestions. I look into this as soon as I have some free time.
I would like to address these issues:

Parameter names with low line and string as value Convolution(in_channels=n_dim)
Displaying a complex module name as a parameter
Polish parameters
Add support for KaTeX expressions
Ditch native app and publish it as a web app


There is a warning of about Shadow issue

View #2607 of type RCTView has a shadow set but cannot calculate shadow efficiently. Consider setting a background color to fix this, or apply the shadow to a more specific component

same here

I don't know if the maintainer is still working on the project, I made some changes, fixed the warning and added some additional features


Add Diversifier QD Meta Algorithm - JAX backend

This PR adds a new JAX-based QD meta algorithm called Diversifier. It is a generalization of CMA-ME (see https://arxiv.org/pdf/1912.02400.pdf).
It uses a MAP-Elites archive not for solution candidate generation, but only to modify the fitness values told (via tell) to the wrapped algorithm. This modification changes the fitness ranking of the population to favor exploration over exploitation. Tested with CR-FM-NES and CMA-ES, but other wrapped algorithms may work as well. Based on https://github.com/dietmarwo/fast-cma-es/blob/master/fcmaes/diversifier.py (see https://github.com/dietmarwo/fast-cma-es/blob/master/tutorials/MapElites.adoc)
The generalization over CMA-ME is necessary in the EvoJAX context, because CMA-ES struggles with a very high number of decision variables. Therefore CR-FM-NES-ME is superior here - as possibly are other not yet tested alternatives.
https://doi.org/10.1145/2739480.2754664 proposes the QD score (sum of fitness values of all elites in the map) as metric for comparison.
For Brax-Ant CR-FM-NES-ME (Diversifier applied to CR-FM-NES), compared with MAP-Elites, reaches a higher QD-score for high iteration numbers (see details below). So MAP-Elites should only be preferred for a low evaluation budget or if you want to maximize the number of occupied niches.
With the configurations given in the Brax-Ant "examples/train_ant_..." on a NVIDIA 3090 + AMD 5950, Linux Mint we measured:

MAP-Elites has a slightly lower optimizer overhead (a higher evaluation/sec rate).
MAP-Elites has a higher number of occupied niches.

but

CR-FM-NES-ME has a much higher QD score and found a better global optimum for a high
evaluation budget (150 minutes for our test).
both QD score and global optimum continue to grow for CR-FM-NES-ME, where MAP-Elites stagnates after 150 minutes.

Detailed measurements for the Brax-Ant example:
After 60 minutes:
MAP-Elites:

iterations: 1064
fitness evals: 1090560
best fitness: 744.935
QD score: 2964073
occupied: 6264
mean of occupied: 473.19
std of occupied: 78.456

CR-FM-NES-ME, fitness_weight=0:

iterations: 1670
fitness evals: 855040
best fitness: 1038.41
QD score: 2952754
occupied: 5529
mean of occupied: 534.04
std of occupied: 150.45

CR-FM-NES-ME, fitness_weight=1:

iterations: 1705
fitness evals: 872960
best fitness: 1734.955
QD score: 3218120
occupied: 5303
mean of occupied: 606.84
std of occupied: 264.31

After 150 minutes:
MAP-Elites:

iterations: 2821
fitness evals: 2888704
best fitness: 805.787
QD score: 3448151
occupied: 6888
mean of occupied: 500.6
std of occupied: 81.735

CR-FM-NES-ME, fitness_weight=0:

iterations: 4490
fitness evals: 2298880
best fitness: 1866.516
QD score: 4846117
occupied: 5747
mean of occupied: 843.24
std of occupied: 395.4

CR-FM-NES-ME, fitness_weight=1:

iterations: 3918
fitness evals: 2006016 (after 133 minutes, no improvement after that)
best fitness: 3031.478
QD score: 3935811
occupied: 5370
mean of occupied: 732.92
std of occupied: 527.67

Note that 'fitness_weight' is a concept neither used in CMA-ME (see https://arxiv.org/pdf/1912.02400.pdf) nor in fcmaes https://github.com/dietmarwo/fast-cma-es/blob/master/fcmaes/diversifier.py. All these use implicitely fitness_weight=0. For fcmaes the reason is that there are other means to improve the elites of a given map, so the focus is on exploration here. We use as default fitness_weight=0, because for Brax Ant the final QD score is higher - but the final global optimum found is lower.
fcmaes even supports sequences of wrapped algorithms, something probably not relevant for EvoJAX.

Could tweak some parameters: For MAP-Elites line-sigma = 0.2 ssems better than the default line-sigma = 0.3.
And for CR-FM-NES-ME I changed the default to init-std = 0.16. Played also with other parameters, but they seem ok.
CR-FM-NES-ME requires a lower population size as  MAP-Elites which reduces the potential gain caused by paralellization.
But since the results are for specific wall times (60 and 15 minutes) this effect is already "considered" since CR-FM-NES-ME
computes less fitness evaluations in this time. Nevertheless, for 150 min wall time the final gap widens:
CR-FM-NES-ME with a clear 50% lead regarding QD score, although achieved with less occupied niches.
After 60 minutes:
MAP-Elites iso-sigma = 0.05, line-sigma = 0.2:

iterations: 1134
fitness evals: 1116160
best fitness: 778.636
QD score: 3160228
occupied: 6569
mean of occupied: 481.08
std of occupied: 81.18

CR-FM-NES-ME, fitness_weight=0, init-std = 0.16

iterations: 1673
fitness evals: 856576
best fitness: 968.033
QD score: 2956379
occupied: 5663
mean of occupied: 522.052
std of occupied: 130.444

After 150 minutes:
MAP-Elites iso-sigma = 0.05, line-sigma = 0.2:

iterations: 2857
fitness evals: 2925568
best fitness: 790.275
QD score: 3598626
occupied: 7039
mean of occupied: 511.241
std of occupied: 86.828

CR-FM-NES-ME, fitness_weight=0, init-std = 0.16

iterations: 4470
fitness evals: 2288640
best fitness: 1908.29
QD score: 5392502
occupied: 5944
mean of occupied: 907.218
std of occupied: 409.10


Replace URI.escape/unescape

These methods are removed from Ruby 3.
We are using the URI::RFC2396_Parser to maintain current behavior.

Looks like the new parser has a decent amount of logic / stuff going on in the initializer https://github.com/ruby/uri/blob/master/lib/uri/rfc2396_parser.rb#L338
are any of these code paths hotspots that we should memoize the parser for instead of reconstructing every time?

That's a good call out, thanks for finding that. I was wondering that, but was staying on the side of caution to prevent introducing any tricky bugs. I can definitely take another look and see what we can do.


Added ability to customise gradle command.

On windows, a new gradle install had the gradle command in a 'gradle.bat' file. This works fine using cmd, I can just type in 'gradle XXX'. However, with rust's 'Command' object (which was being used in cargo-apk), it returns file not found. Attempts to change the name of gradle.bat to gradle were unsuccessful (windows didn't recognise gradle as executable anymore maybe?)
I added a gradle_command field to the android metadata section of the cargo.toml. This lets me put 'gradle.bat' as my gradle command, and the build now passes the gradle check.

Thanks for the PR.
However for me the [metadata.android] section of the Cargo.toml shouldn't contain information about the machine you're compiling with, such as the path to gradle or the SDK and NDK.
Ideally the path to gradle could be loaded from a .cargo/config file, but for now the easiest way is probably to load it from an environment variable.

Hey!
I thought about this, but I wouldn't want to take up an env var like GRADLE_COMMAND or something that might clash with an official gradle env var in the future.
Any suggestions? Would I have to rip out / add a lot to the build script to support reading from .cargo/config?

I changed it to use the CARGO_APK_GRADLE_COMMAND environment variable. A little wordier maybe, but should be specific enough so nobody gets confused.

Thanks. R+ after nit.

Changes made, should be all good!