Calico is frequently down

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce (for bugs)

Context

Your Environment

Calico version
Orchestrator version (e.g. kubernetes, mesos, rkt):
Operating System and version:
Link to your project (optional):

I am installing 3 node k8s cluster, calico will be stable for 1 or 2 days, again after that it goes down. After describing POD it says BGP not established, but Internal IPs are pinging.(after rebooting calico will come up, but again it goes down, rebooting as rework is not possible since some of the suites are running in background).
After describing calico POD it shows--
[root@master ~]# kubectl describe po calico-node-nwvgh -n kube-system
Name:                 calico-node-nwvgh
Namespace:            kube-system
Priority:<PHONE_NUMBER>
Priority Class Name:  system-node-critical
Node:                 worker1/<IP_ADDRESS>
Start Time:           Tue, 10 Nov 2020 09:23:32 +0530
Labels:               controller-revision-hash=6bfdf56d8
k8s-app=calico-node
pod-template-generation=1
Annotations:
Status:               Running
IP:                   <IP_ADDRESS>
IPs:
IP:           <IP_ADDRESS>
Controlled By:  DaemonSet/calico-node
Init Containers:
upgrade-ipam:
Container ID:  docker://cc001dc5ed7f203677993f899e06a83ccd14a1aff59df93671335eb867bc1d30
Image:         calico/cni:v3.15.2
Image ID:      docker-pullable://docker.io/calico/cni@sha256:ab530cb190ef2cffba9e82260b69604705d7dc838b538a134761f59b15be7221
Port:
Host Port:
Command:
/opt/cni/bin/calico-ipam
-upgrade
State:          Terminated
Reason:       Completed
Exit Code:    0
Started:      Tue, 10 Nov 2020 09:23:47 +0530
Finished:     Tue, 10 Nov 2020 09:23:47 +0530
Ready:          True
Restart Count:  0
Environment:
KUBERNETES_NODE_NAME:        (v1:spec.nodeName)
CALICO_NETWORKING_BACKEND:  <set to the key 'calico_backend' of config map 'calico-config'>  Optional: false
Mounts:
/host/opt/cni/bin from cni-bin-dir (rw)
/var/lib/cni/networks from host-local-net-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-sxxcq (ro)
install-cni:
Container ID:  docker://994d33ffd3b35c33b4c4b3753bcbc77f53feb71c4fd60f5567a8ca6c06e37f69
Image:         calico/cni:v3.15.2
Image ID:      docker-pullable://docker.io/calico/cni@sha256:ab530cb190ef2cffba9e82260b69604705d7dc838b538a134761f59b15be7221
Port:
Host Port:
Command:
/install-cni.sh
State:          Terminated
Reason:       Completed
Exit Code:    0
Started:      Tue, 10 Nov 2020 09:23:48 +0530
Finished:     Tue, 10 Nov 2020 09:23:48 +0530
Ready:          True
Restart Count:  0
Environment:
CNI_CONF_NAME:         10-calico.conflist
CNI_NETWORK_CONFIG:    <set to the key 'cni_network_config' of config map 'calico-config'>  Optional: false
KUBERNETES_NODE_NAME:   (v1:spec.nodeName)
CNI_MTU:               <set to the key 'veth_mtu' of config map 'calico-config'>  Optional: false
SLEEP:                 false
Mounts:
/host/etc/cni/net.d from cni-net-dir (rw)
/host/opt/cni/bin from cni-bin-dir (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-sxxcq (ro)
flexvol-driver:
Container ID:   docker://af5bbe5ae14cce12956af331ec08ae42e801b16a29373a0a7c04d3013b9bc70f
Image:          calico/pod2daemon-flexvol:v3.15.2
Image ID:       docker-pullable://docker.io/calico/pod2daemon-flexvol@sha256:3ad57e6bf32098e61909cc3bfc7266500a610d9ea4309ec6f8e6275fc132102a
Port:
Host Port:
State:          Terminated
Reason:       Completed
Exit Code:    0
Started:      Tue, 10 Nov 2020 09:24:06 +0530
Finished:     Tue, 10 Nov 2020 09:24:07 +0530
Ready:          True
Restart Count:  0
Environment:
Mounts:
/host/driver from flexvol-driver-host (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-sxxcq (ro)
Containers:
calico-node:
Container ID:   docker://d5af93f4ea7d3270a77601d2b1654e174f80c998a482f263ee4ecc66123f1a24
Image:          calico/node:v3.15.2
Image ID:       docker-pullable://docker.io/calico/node@sha256:b03990be48e252c3b046b2f56bd7bbc2ed35c6026c0dcee3b50a1c834deb0bfb
Port:
Host Port:
State:          Running
Started:      Tue, 10 Nov 2020 09:24:26 +0530
Ready:          False
Restart Count:  0
Requests:
cpu:      250m
Liveness:   exec [/bin/calico-node -felix-live -bird-live] delay=10s timeout=1s period=10s #success=1 #failure=6
Readiness:  exec [/bin/calico-node -felix-ready -bird-ready] delay=0s timeout=1s period=10s #success=1 #failure=3
Environment:
DATASTORE_TYPE:                     kubernetes
WAIT_FOR_DATASTORE:                 true
NODENAME:                            (v1:spec.nodeName)
CALICO_NETWORKING_BACKEND:          <set to the key 'calico_backend' of config map 'calico-config'>  Optional: false
CLUSTER_TYPE:                       k8s,bgp
IP:                                 autodetect
CALICO_IPV4POOL_IPIP:               Always
CALICO_IPV4POOL_VXLAN:              Never
FELIX_IPINIPMTU:                    <set to the key 'veth_mtu' of config map 'calico-config'>  Optional: false
FELIX_VXLANMTU:                     <set to the key 'veth_mtu' of config map 'calico-config'>  Optional: false
FELIX_WIREGUARDMTU:                 <set to the key 'veth_mtu' of config map 'calico-config'>  Optional: false
CALICO_DISABLE_FILE_LOGGING:        true
FELIX_DEFAULTENDPOINTTOHOSTACTION:  ACCEPT
FELIX_IPV6SUPPORT:                  false
FELIX_LOGSEVERITYSCREEN:            info
FELIX_HEALTHENABLED:                true
Mounts:
/lib/modules from lib-modules (ro)
/run/xtables.lock from xtables-lock (rw)
/var/lib/calico from var-lib-calico (rw)
/var/run/calico from var-run-calico (rw)
/var/run/nodeagent from policysync (rw)
/var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-sxxcq (ro)
Conditions:
Type              Status
Initialized       True
Ready             False
ContainersReady   False
PodScheduled      True
Volumes:
lib-modules:
Type:          HostPath (bare host directory volume)
Path:          /lib/modules
HostPathType:
var-run-calico:
Type:          HostPath (bare host directory volume)
Path:          /var/run/calico
HostPathType:
var-lib-calico:
Type:          HostPath (bare host directory volume)
Path:          /var/lib/calico
HostPathType:
xtables-lock:
Type:          HostPath (bare host directory volume)
Path:          /run/xtables.lock
HostPathType:  FileOrCreate
cni-bin-dir:
Type:          HostPath (bare host directory volume)
Path:          /opt/cni/bin
HostPathType:
cni-net-dir:
Type:          HostPath (bare host directory volume)
Path:          /etc/cni/net.d
HostPathType:
host-local-net-dir:
Type:          HostPath (bare host directory volume)
Path:          /var/lib/cni/networks
HostPathType:
policysync:
Type:          HostPath (bare host directory volume)
Path:          /var/run/nodeagent
HostPathType:  DirectoryOrCreate
flexvol-driver-host:
Type:          HostPath (bare host directory volume)
Path:          /usr/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds
HostPathType:  DirectoryOrCreate
calico-node-token-sxxcq:
Type:        Secret (a volume populated by a Secret)
SecretName:  calico-node-token-sxxcq
Optional:    false
QoS Class:       Burstable
Node-Selectors:  kubernetes.io/os=linux
Tolerations:     :NoSchedule
:NoExecute
CriticalAddonsOnly
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/network-unavailable:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/pid-pressure:NoSchedule
node.kubernetes.io/unreachable:NoExecute
node.kubernetes.io/unschedulable:NoSchedule
Events:
Type     Reason            Age                            From              Message

Warning  DNSConfigForming  19m (x8095 over 7d1h)          kubelet, worker1  Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>
Warning  Unhealthy          (x34280 over 4d23h)  kubelet, worker1  (combined from similar events): Readiness probe failed: 2020-11-17 05:38:39.529 [INFO][1814] confd/health.go 180: Number of node(s) with BGP peering established = 0
calico/node is not ready: BIRD is not ready: BGP not established with <IP_ADDRESS>,<IP_ADDRESS>

how to resolve calico permanently?

  Warning  DNSConfigForming  19m (x8095 over 7d1h)          kubelet, worker1  Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>
  Warning  Unhealthy         <invalid> (x34280 over 4d23h)  kubelet, worker1  (combined from similar events): Readiness probe failed: 2020-11-17 05:38:39.529 [INFO][1814] confd/health.go 180: Number of node(s) with BGP peering established = 0
calico/node is not ready: BIRD is not ready: BGP not established with <IP_ADDRESS>,<IP_ADDRESS>

From your describe output there are two events, one of them from Calico, which explains why Calico is not ready.

Number of node(s) with BGP peering established = 0
calico/node is not ready: BIRD is not ready: BGP not established with <IP_ADDRESS>,<IP_ADDRESS>

This means the Calico instance is failing to establish BGP peering relationships with the node Calico instances. This could be due to incorrectly applied security groups / firewall rules, or something else.

but Internal IPs are pinging

ping uses ICMP, whereas BGP is TCP port 179. So, it is possible that the ping traffic is allowed but the BGP traffic is not. I'm skeptical of that because the other nodes seem to have established, though.


Focusing in componentDidMount

I try to make focus when component is loaded. This doesn't work.
componentDidMount() {
  this.refs.editor.focus();
}
And this also doesn't help:
componentDidMount() {
  setTimeout(() => this.refs.editor.focus(), 1);
}

@peasemaker - thanks for opening this issue. For issues like this, you can also post in the Draft.js Slack channel, as you're more likely to get a quicker response there (there are usually at least a handful of people hanging out there throughout the day).
What you're looking for should be represented in this jsfiddle: https://jsfiddle.net/n1qkw6kh/
I'm going to close, feel free to reopen if any issue (or post in the Slack channel :) )


Fix prisma deprecated package references

imports from "@prisma/client/runtime" are deprecated.
Use "@prisma/client/runtime/library",  "@prisma/client/runtime/data-proxy" or  "@prisma/client/runtime/binary"

Closed by #400


Adding vault service account and ibm key protect based KMS encryption support

Introducing new KMS features in 4.10:

Vault Kubernetes service account-based authentication:

The Kubernetes auth method can be used to authenticate with Vault using a Kubernetes Service Account Token.
Steps to enable Kubernetes to service account based auth is:

To configure vault:
https://github.com/rook/rook/blob/master/tests/scripts/deploy-validate-vault.sh#L173

ConfigMap: ocs-kms-connection-details
   data:
        KMS_PROVIDER: vault
        KMS_SERVICE_NAME: // name for this config
        VAULT_AUTH_KUBERNETES_ROLE: //required
        VAULT_AUTH_METHOD: kubernetes
        VAULT_CLIENT_CERT: //optional
        VAULT_NAMESPACE: //optional
        VAULT_CACERT: //optional
        VAULT_TLS_SERVER_NAME: //optional
        VAULT_CLIENT_KEY: //optional
        VAULT_BACKEND_PATH: //optional
        VAULT_ADDR: //required

IBM HPCS: https://www.ibm.com/cloud/blog/announcements/keep-your-private-keys-safe-on-red-hat-openshift-on-ibm-cloud-with-hyper-protect-crypto-services.

ConfigMap: ocs-kms-connection-details
   data:
       KMS_PROVIDER: "ibmkeyprotect",
       KMS_SERVICE_NAME: //name for this config
       IBM_SERVICE_INSTANCE_ID: //required
       IBM_KMS_KEY: //required (Secret Name)
       IBM_BASE_URL: //optional
       IBM_TOKEN_URL: //optional

Enable encryption

StorageCluster CR:
spec:
   encryption:
    clusterWide: true
    storageClassWide: true / false
    kms: {
        enabled: true
    }

PR is on top of: https://github.com/red-hat-storage/ocs-operator/pull/1412

Please provide more details on what the Clusterwide and StorageClass encryption features do. Do they use the same KMS if both are enabled? Is the StorageClass encryption applicable to all the storage classes created by OCS?

https://github.com/red-hat-storage/ocs-operator/pull/1412#issuecomment-984383500

/retest

@GowthamShanmugam ocs-ci failure looks related to your changes, can you please check once.

/hold
wait for #1412

can you merge your 1st and 3rd commit?

ack

/hold cancel
/test red-hat-storage-ocs-ci-e2e-aws
/test ocs-operator-bundle-e2e-aws


Fix one instance of push_str for a single char

Clippy has the lint
https://rust-lang.github.io/rust-clippy/master/index.html#single_char_add_str
There was one instance where push_str() was used to add a single SP
char. Other similar code used push. This change switches to push(),
which makes things consistent, and little Mr. Clippy happy.

👍 Thank you!


npm run lint:types  too many error

version
see my yarn.lock file
yarn.lock
step

do https://open-wc.org/linting/linting-types.html#manual

tsconfig https://github.com/open-wc/open-wc/commit/0f1d30d7a401be298d11c9e239e4af6231d49182

npm run lint:types

expect
no error
actual
long time wait...
output many error( An amount that the console cannot display )
error from

dist folder
node_modules/@open-wc folter

error.log

Sorry my bad english.
and
Thank you for the wonderful framework

Sorry for the late response on this issue. You will need to instruct your typescript config to not do any checking of dependencies in your node modules for this.


docs(guide-go): copy embed directories to container (Go-SDK)

Little docs following a question from Alvise on help channel:

explains how to recusively copy a directory to container

With warning, until we implement the corresponding API call
!! Rebased on top of git-login branch !!

Separated the snippets + rebased on top of main

@mircubed rebased this one too. Please merge #3638 first, I'll rebase afterwards


Remove latest_block_header

Fix #797:

Remove latest_block_header
Maintain (pre_state, block) -> (post_state, error) state transition function by caching the block root after checking the state root

def final_updates(state: BeaconState, block: BeaconBlock) -> None:
    # Check state root
    assert block.state_root == hash_tree_root(state)
    # Cache block root
    state.latest_block_roots[state.slot % SLOTS_PER_HISTORICAL_ROOT] = hash_tree_root(block)

This has the same problem as described in #797.
When "producing" a block, you don't set block.state_root until entirely out of the state transition function (and just avoid this state_root validity check) so at this point within the state transition function the state_root is not yet set in the block.
My intuition is telling me (pre_state, prev_block_root, block) -> post_state is the way to go.

I see, you're implying that this happens "between" two state transitions...
I think just being explicit about the oracle is cleaner.

I don't like this either. I remember implementing this exact fix years ago when doing an early Casper PoC, and it just led to a lot of confusion and client complexity around "signing post-state root" vs "real post-state root".


[Privacy] Gitea leaks hidden email addresses

Gitea version (or commit ref): 3e445cc
Git version: not relevant
Operating system: not relevant
Database (use [x]):

[ ] PostgreSQL
[ ] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[x] Yes (provide example URL)
[ ] No
[ ] Not relevant

Log gist:

Description
Gitea leaks users hidden email addresses when sending email notifications to multiple recipients.
Reproduction steps:

Create new repository from an account with an email address set as hidden. This is the victim account.
Create new user account ( this is attacker account ) , set email to private and watch repository from 1).
Create new user account and trigger notification in repository 1). For example open a new issue.

Result:
Both user 1 ( victim ) and user 2 ( attacker ) get each others emails even if set to hidden ( see the "to" form or email headers in the inbox website). This works when email notification is send to a list / multiple users, so works between users too, for example all watchers of a repository. All users in that list get each other emails.
Proposed fix:
Send notifications emails to a list of users with public email address and individual emails to users with hidden email address.
Screenshots

Just hit this bug due to the CSRF issue. Got a long list of Email addresses for every victim of that issue.
(Automatically followed OP, then got notifications of all events, leaking the Email addresses.)

For anyone looking to solve this issue, the line you can change is here: https://github.com/go-gitea/gitea/blob/95f2e2b57beedcdeb2b9623dc86e26f252fdd7bd/models/issue_mail.go#L102 it passes all emails into the function, but what could be done is to loop over emails and then individually pass them into the SendIssueMentionMail function (the last argument in that function is where the emails get passed in)

Not a complaint, just asking for clarification: This fix means that email notifications for issues will be sent out to each watcher independently starting with Gitea 1.5.1, correct?  This means that users will no longer be able to do 'Reply All' to continue an email-only thread on the topic, correct?

@HenrikBengtsson yes


Integrate all message types

Messages

[ ] Address
[x] Audio
[x] Contacts
[x] Document
[x] Image
[x] Interactive Call-To-Action URL
[ ] Flow
[x] Interactive List
[x] Interactive Reply Buttons
[x] Location
[x] Location Request
[x] Reaction
[x] Sticker
[ ] Template
[x] Text
[x] Video
[x] Read Receipts

Message types Address and Flow pending feature access.


Restest: On SetupScreen : Shutdown on Networkcable deplug

Got report that this is broken since v1.9.0 - should be fixed with next release

retested with v1.10.0rc2 and worked.closing.


Second database migration throws an error

Module: graphql-migrations
Version: 0.16

This occurs when the primary key field is NOT "id" - it fails on uuid: ID, _id: ID etc.

> yarn develop
yarn run v1.22.5
$ ts-node src/index.ts
🚀  Server ready at http://localhost:4000/graphql
Migrated database
^C
> yarn develop
yarn run v1.22.5
$ ts-node src/index.ts
🚀  Server ready at http://localhost:4000/graphql
(node:71304) UnhandledPromiseRejectionWarning: error: alter table "public"."note" alter column "_id" drop not null - column "_id" is in a primary key

Automatically generated comment to notify maintainers
/cc @machi1990, @wtrocki

I think we need an automated test or unit test that covers consecutive migrations like this.

Looks like the diff algorithm needs some love :-)

Patch available in https://github.com/aerogear/graphback/releases/tag/1.0.0-beta2


Color Picker closes prematurely after clicking anywhere in the picker.

Does this issue occur when all extensions are disabled?: Yes/No
Yes

VS Code Version: VS Code v1.84.1
OS Version: Windows 10 Pro x64 Version 22H2 (Build 19045.3448)

Steps to Reproduce:

Click or hover over a color square in a document, like settings.json
Color picker opens
Click on the slider on the right to change hue
Color picker closes before you can actually pick the color you want

Screen recording:
https://github.com/microsoft/vscode/assets/33441569/569e1581-80c2-441a-9666-dc30a503ac43

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.84.1. Please try upgrading to the latest version and checking whether this issue remains.
Happy Coding!

I was actually incorrect when I posted my version number initially. I am definitely running on 1.84.1. And the issue remains the same as my original post. I'm going to edit it to reflect the correct version.

I have the same issue, and I'm on 1.84.1 as well.

This issue seems to also affect me, i am currently using the insider build of vscode but i also tried the latest release and it also has this bug.

i has the same problem, i had to install an older version to fix it.

Same issue - v1.84.1

I have the same issue from nowhere. It started from the lateset vsc update.

I will work on this issue

I can reproduce this issue on Ubuntu 22.04.2 with VS Code 1.84 installed as a Snap, and also with VS Code Insiders 1.85 (f346494c77e1245505e53a2ddcab3bfb4ee52e08) installed as a Snap.
I wonder if this is related in any way to #136965?
Related on Stack Overflow: Why does the VS Code 1.84 color picker close after clicking any input in it?
Also related on Reddit: https://www.reddit.com/r/vscode/comments/17rc8h1/is_the_color_picker_supposed_to_close_once_you/

Same issue on v1.84.2

Are we going to fix it at all? The original issue has been posted since 2021. It's almost 2024 now.

@honguyenminh what do you mean? what issue ticket are you referring to? The one I mentioned above? Because I wasn't implying that they are the same issue, and if you compare the behaviour described, they are different. I was only asking (the maintainers) if they are related.

Hey guys, I took a look and it was actually a recent change that caused the issue. If anyone wants to give feedback on the PR I just opened it'd be greatly appreciated. Hopefully we can get this fixed soon!

same issue on
Version: 1.84.2 (user setup)

I have the same problem and because of this I decided to downgrade to an older version.

Hi @mk8270 the fix is in Insiders 1.85 or the next release of stable, which will come out in two weeks. You may download Insiders version of VS Code on https://code.visualstudio.com/insiders/
What version of VS Code are you using @Irohukl?

Thanks for the quick turnaround. Jeff…On Nov 15, 2023, at 2:33 PM, Aiday Marlen Kyzy @.***> wrote:
Hi @mk8270 the fix is in Insiders 1.85 or the next release of stable, which will come out in two weeks. You may download Insiders version of VS Code on https://code.visualstudio.com/insiders/
What version of VS Code are you using @Irohukl?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

I'm using version 1.83.1 while waiting for the next update to resolve the issue. @aiday-mar @jonesjl

Can we reopen this issue? The bug is still present in the latest version.

Hi, @marianhlavac. What version of VS Code are you using? I am on the following version:
1.85.0-insider (Universal)
Commit: e0b9ba5710060c8a8014f8a12980f011d81fb844
Date: 2023-11-20T05:35:23.138Z (2 hrs ago)

I checked the bug is resolved in the following CSS file:
https://github.com/microsoft/vscode/assets/61460952/488c1c4c-6f97-410f-bf2f-1c3eaa0efc87
Could you please include here the information you see under About Visual Studio Code or About Visual Studio Code - Insiders. Can you also paste here a snippet of the code which causes the error?

Working for me as well on insiders.vscode.dev with version 1.85.0-insider.
@aiday-mar do you think it would be beneficial to add a smoke test that checks on this?

@aiday-mar Sorry, I totally missed your previous comment about issue being fixed on the insider version. I'm on 1.84.2

Hi @oxcened, thanks for the update. That is an idea, we may come round to adding a test.

wait for new update cus in insider 1.85.0-insider is works.

This issue will be closed as the bug is fixed

This is still an issue for me, I'm using Visual Studio Code version 1.84.2 . Is there any fix I personally can do or do I have to wait for an update? Thanks :)

@DisenzZ please read https://github.com/microsoft/vscode/issues/197588#issuecomment-1813139342

Can confirm it’s solved in v1.85, which has now been released for the regular Visual Studio Code.


Store line numbers on Element

I added two new properties: Element.lineNumberStart and Element.lineNumberEnd. I'm building a tool that reports errors in XML files and I need to be able to report the line number so the user can find the error.
I also made some changes that make the tests run in a Swift Package Manager context. I believe they will continue to work as they were before; the only difference is I'm using a path-based lookup based on the unit test file instead of Bundle.

I have a small followup PR after this that adds CDATA support to fix #60.

Thanks!


Async and non-Async calls hang at POST to Cluster in WPF and WinForms [Bug]

Describe the bug
When trying to create a simple example:
            var accountInfo = rpcClient.GetAccountInfoAsync("pubkeyHERE");
            MessageBox.Show(accountInfo.Result.ToString());```
We will hang on GetAccountInfoAsync during its POST to the cluster. It never gives a timeout or finishes the call. Entire app hangs.

**To Reproduce**
Steps to reproduce the behavior:
Create a .Net 7 WPF or WinForms.
Add the above code

**Expected behavior**
Return the response from Cluster or Timeout

**Desktop (please complete the following information):**
 - OS: Win 10/11 tested
 - .Net 7

Obviously this is related to #430 but I've tested both ASYNC and non-async functions, however i've tested both winforms and wpf. Not sure if the comment from that ticket means we cant use it with any gui framework of windows or not

It does appear to work in console apps though


AssertionError: isolated node! hidden-tensor

Describe the bug
AssertionError: isolated node! hidden-tensor

On a particular model below
To Reproduce
pip install -e<EMAIL_ADDRESS>from hear21passt.base import get_basic_model,get_model_passt
import torch
# get the PaSST model wrapper, includes Melspectrogram and the default pre-trained transformer
model = get_basic_model(mode="logits")
print(model.mel) # Extracts mel spectrogram from raw waveforms.

# optional replace the transformer with one that has the required number of classes i.e. 50
model.net = get_model_passt(arch="passt_s_swa_p16_128_ap476",  n_classes=50)
print(model.net) # the transformer network.

# now model contains mel + the transformer pre-trained model ready to be fine tuned.
# It's still expecting input of the shape [batch, seconds*32000] sampling rate is 32k

model.train()

model_graph = draw_graph(
                model,
                input_size=(
                    4,
                    32000 * 10
                ),
                # # Graph left-to-right: https://github.com/mert-kurttutan/torchview/issues/56
                graph_dir="LR",
                depth=3,
                roll=True,
                expand_nested=True,
                # save_graph=True,
                # directory=tempdir,
            )

    model_graph = draw_graph(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/torchview/torchview.py", line 225, in draw_graph
    model_graph.fill_visual_graph()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/torchview/computation_graph.py", line 123, in fill_visual_graph
    self.render_nodes()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/torchview/computation_graph.py", line 132, in render_nodes
    self.traverse_graph(self.collect_graph, **kwargs)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/torchview/computation_graph.py", line 186, in traverse_graph
    self.traverse_graph(action_fn, **new_kwargs)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/torchview/computation_graph.py", line 186, in traverse_graph
    self.traverse_graph(action_fn, **new_kwargs)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/torchview/computation_graph.py", line 186, in traverse_graph
    self.traverse_graph(action_fn, **new_kwargs)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/torchview/computation_graph.py", line 155, in traverse_graph
    action_fn(**kwargs)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/torchview/computation_graph.py", line 198, in collect_graph
    self.check_node(cur_node)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/torchview/computation_graph.py", line 431, in check_node
    assert not node.is_leaf() or not node.is_root(), (
AssertionError: isolated node! hidden-tensor

Expected behavior
A graph should be produced.
Note that this model works with depth=1

Hi,
The error is essentially because of the line here where they use .item() to get the value inside the tensor. This then leads to the value of tensor being used instead of tensor itself. This means that this tensor is not attached to the rest of network as torch.Tensors would do in a computational graph.
There are two options I see:

Ignore these tensors
Keep these tensors

I will also drop the assertion since this is a legitimate use in a pytorch code.

Just for documentation purposes, the reason it worked for depth=1 is because when depth these tensors are not even reached since their depth is 2.

@mert-kurttutan I would be happy with either. If ignore is easier, then perhaps just do ignore and log a warning to the user?

Actually, these cases are really rare and correspond to very small part of network. For this to happen, there has to be no operation on the tensor whose item method being used. I think ignoring them seems reasonable without any message.
For instance,
fmin = self.fmin + torch.randint(self.fmin_aug_range, (1,)).item()

Tensor is created and right after this, item is applied. So only, this tensor creation part will be ignored. Since this type of part does not contain any operation and is crucial, I will ignore and not send any warning

Sounds great to me.

Can you try now using the main branch of the repo? After this, I can release the new version.

@mert-kurttutan I'm traveling right now so I cannot check immediately.

On SHA 00bd35b9bbb6e21df2613fcc89ce3f4833edf767
it works!

Thank you!


Fix at #13 seems to fail

See comment on https://github.com/CABLE-LSM/benchcab/issues/13 which would appear to be the source of a current error in benchcab.

This was solved. The problem was the latest changes in the example work directory needed to be updated locally before running benchcab.


symbolic link left in repository

I don't think this symbolic link is necessary or effective:
https://github.com/amazon-science/patchcore-inspection/blob/main/src/patchcore/anomaly_detection

Thanks for pointing this out. Removed it.


Macro command to call Ilastik Object Classification

Running the object classification workflow on FIJI surprisingly doesn't output a command in the recorder window. Any way to script the process for batch mode through FIJI? A command (with variables) and a way to identify/designate the the output paths for the image & table is I'd need. Its fine even if its called through exec() or whatever.

I think it only outputs the command after a successful run.


Update renovatebot/github-action action to v34.54.0

This PR contains the following updates:

Package
Type
Update
Change

renovatebot/github-action
action
minor
v34.52.0 -> v34.54.0

Release Notes

renovatebot/github-action
v34.54.0
Compare Source
See the the changelog for changes in all releases.
34.54.0 (2022-12-09)
Bug Fixes

deps: update renovate/renovate docker tag to v34.54.0 (dadf914)

v34.53.0
Compare Source
See the the changelog for changes in all releases.
34.53.0 (2022-12-09)
Bug Fixes

deps: update renovate/renovate docker tag to v34.53.0 (7ab2d37)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Enabled.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

Edited/Blocked Notification
Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.
You can manually request rebase by checking the rebase/retry box above.
⚠ Warning: custom changes will be lost.

Edited/Blocked Notification
Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.
You can manually request rebase by checking the rebase/retry box above.
⚠ Warning: custom changes will be lost.

Edited/Blocked Notification
Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.
You can manually request rebase by checking the rebase/retry box above.
⚠ Warning: custom changes will be lost.


Adds paramater name to JwtClientAssertion validation error Fixes gh-1449

The validation could be re-written to validate all parameters and add the name of all invalid parameters, however, this change is more minimal as it only requires a small refactor of the existing code.

Closing as per comment


Thank you for this @AskMeAboutLoom.
Ping @sinfonietta, this might be for your list?


Text threshold vs detection threshold

I think you might have mixed up the detection_threshold and text_threshold variables in  detection.py since they differ from the original CRAFT implementation and sine there is an incongruity between a comment and the variable name in one instance.
Here you have:
_, text_score = cv2.threshold(textmap,
    thresh=text_threshold,
    maxval=1,
    type=cv2.THRESH_BINARY)

The original implementation uses the low_text parameter here (what you're calling detection_threshold). So I believe text_threshold should be replaced with detection_threshold.
And then a few lines down you have:
# If the maximum value within this connected component is less than
# text threshold, we skip it.
if np.max(textmap[labels == component_id]) < detection_threshold:
     continue

Should detection_threshold be text_threshold as the comment says and as it is in the original implementation?
If so, the fix seems easy. I could open the PR if you'd like. Or you as the maintainer can take it.
Again, thanks for your awesome work on this 👏

Thanks for taking a close look under the hood! You're right that the variable names don't match. But I believe the values used in the function (and therefore the net effect) do match.
parser.add_argument('--text_threshold', default=0.7, type=float, help='text confidence threshold')
parser.add_argument('--low_text', default=0.4, type=float, help='text low-bound score')
parser.add_argument('--link_threshold', default=0.4, type=float, help='link confidence threshold')

[source]
The variable names are hard for me to keep in my head so I made a table to summarize the differences.

purpose
keras-ocr variable name
keras-ocr variable value
CRAFT-pytorch variable name
CRAFT-pytorch

threshold the text map
text_threshold
0.4
low_text
0.4

threshold the link map
link_threshold
0.4
link_threshold
0.4

filter out detections
detection_threshold
0.7
text_threshold
0.7

That still leaves us with the question of whether we should change the variable names to match the original implementation. In my humble opinion, the new names are more semantically descriptive. text_threshold and link_threshold are used in similar ways, so I think it makes sense for their variable names to have similar structure. This is in contrast with the low_text / link_threshold naming which, to me, implies that these values are used in different ways. Could you share your thoughts on that?
I'd also appreciate you checking to see if I've made a mistake above -- this can all be a little confusing and I may have misread something.

You're right, your values do match the original implementation, so there's no impact on post-processing. It was, however, a little confusing when I adjusted the values and wasn't seeing the effects I expected.
I do agree with you about not changing the variable names to match the original ones. Yours are more descriptive, particularly with detection_threshold, which is used with the connected-component labeling that actually does the "detecting". I also like how you added the size_threshold, which was originally just hardcoded.
As such, I think a little more documentation that highlights the meaning and usage of these params is all you need. A docstring in the source would probably suffice, as most users likely won't want/need/know to alter these for their use case. This discussion in the original repo does a pretty good job at describing each's purpose in plain English.

I think updating the docstring is a fine idea. Unfortunately, the description in the linked comment defines things in terms of their perceived effect as to how they are actually used. Below are what I believe are more accurate definitions that discuss how the values are used in addition to their effect. I'm interested in your feedback on the definitions. If you agree, I'll add them to the docstring for detector.detect. Or, since it was your idea, you're welcome to file a PR. I'd very much like you to get credit!

text_threshold: When the text map is processed, it is converted from confidence (float from zero to one) values to classification (0 for not text, 1 for text) using binary thresholding. The text_threshold value determines the breakpoint at which a value is converted to a 1 or a 0. For example, if text_threshold is 0.4 and a value for a particular point on the text map is 0.5, that value gets converted to a 1. The higher this value is, the less likely it is that characters will be merged together into a single word. The lower this value is, the more likely it is that non-text will be detected. Therein lies the balance.
link_threshold: This is the same as text_threshold, but is applied to the link map instead of the text map.
detection_threshold: We want to avoid including boxes that may have represented large regions of low confidence text predictions. To do this, we do a final check for each word box to make sure the maximum confidence value exceeds some detection threshold. This is the threshold used for this check.

I've added this documentation in 717bfcde4e0447c6630487f78787ff62bdabe73e. Thanks for raising these questions!


bug: test/e2e/run-test.sh --test-type ginkgo is flaky

Describe the bug
The CI testing for #2199 produced inconsistent results for the ginkgo E2E test. See https://github.com/kubestellar/kubestellar/actions/runs/9476682004?pr=2199 , and note that of the three attempts, two succeeded and one failed. I got the same results when testing on my laptop: the first try failed, and then two more succeeded.
There is also a temporal correlation: the failures on GitHub and my laptop were earlier in the day, and the successes on GitHub and my laptop were late in the evening.
Steps To Reproduce
See above.
Expected Behavior
Consistent success.
Additional Context
No response

The failure in CI involved a timeout somewhere in setup-kubestellar.sh, but the logging is insufficient to identify where (this is #2200).
The failure on my laptop produced the following log from the container named ocm-ocm in the trying to do clusteradm init for its1.
Preflight check: HubApiServer check Failed with 0 warnings and 1 errors
Preflight check: cluster-info check Failed with 0 warnings and 1 errors
Error: [preflight] Some fatal errors occurred:
	[ERROR HubApiServer check]: failed to find the given Current Context in Contexts of the kubeconfig
	[ERROR cluster-info check]: configmaps "cluster-info" is forbidden: User "system:serviceaccount:its1-system:default" cannot get resource "configmaps" in API group "" in the namespace "kube-public"

That was from the older of the two pods from the ocm Job (sorry, should have captured the log from the younger; there are commonly two, with the older failing and the younger succeeding). Here is the pod listing from the hosting cluster:
mspreitz@mjs13 kubestellar % kubectl --context kind-kubeflex get pods -A
NAMESPACE            NAME                                                READY   STATUS             RESTARTS        AGE
ingress-nginx        ingress-nginx-admission-create-s8sks                0/1     Completed          0               25m
ingress-nginx        ingress-nginx-admission-patch-97j9b                 0/1     Completed          2               25m
ingress-nginx        ingress-nginx-controller-779f699d74-c9xww           1/1     Running            0               25m
its1-system          coredns-68559449b6-xqbrs-x-kube-system-x-vcluster   1/1     Running            0               22m
its1-system          ocm-6bqj2                                           0/2     Error              0               22m
its1-system          ocm-dxgp2                                           0/2     Error              0               21m
its1-system          update-cluster-info-8hdvt                           0/1     Completed          0               22m
its1-system          update-cluster-info-dr8pf                           0/1     Error              0               23m
its1-system          vcluster-0                                          2/2     Running            0               23m
kube-system          coredns-76f75df574-t87cs                            1/1     Running            0               25m
kube-system          coredns-76f75df574-zdghj                            1/1     Running            0               25m
kube-system          etcd-kubeflex-control-plane                         1/1     Running            0               25m
kube-system          kindnet-hvrxt                                       1/1     Running            0               25m
kube-system          kube-apiserver-kubeflex-control-plane               1/1     Running            0               25m
kube-system          kube-controller-manager-kubeflex-control-plane      1/1     Running            0               25m
kube-system          kube-proxy-mnzr2                                    1/1     Running            0               25m
kube-system          kube-scheduler-kubeflex-control-plane               1/1     Running            0               25m
kubeflex-system      kubeflex-controller-manager-649f5484cc-k7bg8        2/2     Running            0               24m
kubeflex-system      kubeflex-operator-fxt6x                             0/1     Completed          0               24m
kubeflex-system      postgres-postgresql-0                               1/1     Running            0               25m
local-path-storage   local-path-provisioner-7577fdbbfb-885nz             1/1     Running            0               25m
wds1-system          kube-apiserver-76bb46975b-xxfr7                     2/2     Running            0               22m
wds1-system          kube-controller-manager-868b9f8bdb-hppgb            1/1     Running            0               22m
wds1-system          kubestellar-controller-manager-6fc5c45d7-bx6v8      1/2     CrashLoopBackOff   8 (4m34s ago)   20m
wds1-system          transport-controller-5cd8457bbb-2h92t               0/1     CrashLoopBackOff   8 (4m20s ago)   20m

The failure in CI was different. The listing of pods in the hosting cluster did not even have any in the its1-system namespace! Here is the listing:
Run date
  date
  kubectl --context kind-kubeflex get pods -A
  kubectl --context kind-kubeflex get pods -A | grep -vw Running | grep -vw Completed | grep -v NAME | while read ns name rest; do echo; kubectl --context kind-kubeflex describe pod -n $ns $name; echo; kubectl --context kind-kubeflex logs -n $ns $name || true; done
  shell: /usr/bin/bash -e {0}
  env:
    KO_DOCKER_REPO: ghcr.io/kubestellar/kubestellar
Wed Jun 12 05:23:48 UTC 2024
NAMESPACE            NAME                                             READY   STATUS      RESTARTS   AGE
ingress-nginx        ingress-nginx-admission-create-bcxvh             0/1     Completed   0          58m
ingress-nginx        ingress-nginx-admission-patch-77wnq              0/1     Completed   2          58m
ingress-nginx        ingress-nginx-controller-876969cd6-bmtmv         1/1     Running     0          58m
kube-system          coredns-7db6d8ff4d-6d9xz                         1/1     Running     0          58m
kube-system          coredns-7db6d8ff4d-bcxzg                         1/1     Running     0          58m
kube-system          etcd-kubeflex-control-plane                      1/1     Running     0          58m
kube-system          kindnet-tvdk6                                    1/1     Running     0          58m
kube-system          kube-apiserver-kubeflex-control-plane            1/1     Running     0          58m
kube-system          kube-controller-manager-kubeflex-control-plane   1/1     Running     0          58m
kube-system          kube-proxy-9zhtf                                 1/1     Running     0          58m
kube-system          kube-scheduler-kubeflex-control-plane            1/1     Running     0          58m
kubeflex-system      kubeflex-controller-manager-6d957b7b8d-fcbpm     2/2     Running     0          58m
kubeflex-system      kubeflex-operator-9zzzz                          0/1     Completed   0          58m
kubeflex-system      postgres-postgresql-0                            1/1     Running     0          58m
local-path-storage   local-path-provisioner-988d74bc-dx565            1/1     Running     0          58m

The only thing that PR 2199 changes that is relevant to E2E testing without --released is the PostCreateHook named kubestellar --- which is invoked after whatever went wrong in both cases above.

Testing with cd test/e2e/ginkgo; KFLEX_DISABLE_CHATTY=true ginkgo -vv --trace --no-color on my laptop is also succeeding.


support splitting the screen

should be possible to show 2 logfiles in parallel (vertical split)

not missed up till now...dropping it


Can't transfer and record audio while playing on phone.

I can’t record and transfer audio during the game “Mafia Game Online with friends”, the phone works during the game as if it were a call, I tried different commands through scrcpy and even without it, but nothing works for me, I can do everything except transfer the audio players' speeches to the computer for recording

there is a direct connection with the developer, but he simply does not know what needs to be changed in the application itself to be able to make this recording

#4084 #4087?

#4084  #4087 ?

I read and tried, but I have another problem, the speeches of the players are not transferred and that’s it, the microphone can work, but the speeches of the players themselves are under no circumstances transferred or recorded on the PC


Help with graph structure, any advice is appreciated!

Hi all!
I start by saying sorry for this long thread, Im not good at getting to the point as you can see... I'm knee-deep into this and have so many questions, and so little people I can ask (nobody really lol), I really want to make this happen since I honestly believe in the potential and value it can bring.
Intro & Context
I'm working on a hackathon project for work: A Slack bot that answers questions about our product, so far created in Google Colab as a POC. I'm a Product Operations person and have no real programming background (I know SQL and thought myself qbasic when I was a kid, but thats it), I really have a hard time to understand the "reference" part of the docs and to translate the info into actual functions. So bare with me, I'm really proud that I got so far (thanks GPT4 and for all the example notebooks!)
I did read the WHOLE documentation, multiple times, believe me that it's not so clear for someone like me (not an engineer, but a technically inclined person, I do have a proxmox server and play around with docker but thats it lol). Meaning I'm good with finding answers online and to integrate answers to my cases.
Our product documentation is dispersed, we find information across 4 different platforms (3 KBs & Slack channels) and have many knowledge gaps. Info can be on spot and relevant, but some other articles can be outdated, wrong, and other topics can just be missing.  Slack in our org is super-active and many questions get only answered in specific slack channels, but it's hard to navigate and to find those. I general I'd say that any employee searching for answers has a hard time to find them, and not all employees have access to all knowledge bases...
My idea was to have centralized knowledge access via Slack, no matter the source.
By logging (an analyzing) all questions, answers & emoji reactions to snowflake, we would be able to identify areas that need updated or new content (no answer or negative reaction to answer). Thus replacing a knowledge related project I really didn't want to do (or believed in) with a smart and AI based tool, bringing demand driven knowledge improvement.
I thought of "simply" indexing each source and building a graph of those indices that can be queried.
I do have a working POC, but

getting an answer is very slow (~20 sec) &
I'm sure it could be done much better (probably the reason for the slow response)

Any hint or advice is appreciated !
Let me share my questions first followed by code and some info:
Questions:

High level structure
I created an Index for each source, added a broad explanation of each as summary and combined those into a graph. I wonder if this is the right approach. A graph needs summaries and it makes not much sense to summarize each source as a whole (Summarize Confluence....?). Wouldn't it make more sense to first index each article separately, then creating one graph per source, and then creating one graph thats made out of those 4 graphs? The docs talk about being able to indefinitely stack indices over indices.

Vector VS Tree VS ...
I created SimpleVector indices for each source and combined them as a TreeIndex-Graph. The example notebooks here I saw used for the graph either List or Knowledge-graph indeces. I thought list would force the query to go over all the content for every query which i wanted to avoid & Knowledge-graph wouldn't be a good fit since my data is not structured like a knowledge-graph should be. Is my gut feeling wrong?
Or... if I want to make sure to get combined answers from multiple sources , do I then I need to use ListIndex for the Graph??
Also, if I stack multiple indices over each other (as suggested in Question 1), what index type would make sense to use in such a case? Tree over Tree over Vector? Tree over Keyword over Vector?
I saw that page that talks about Routing VS Synthesis over Heterogenous Data, and I get the impression it would make sense to index each article with either vectorindex or keywordindex, have each article summarized and create a TreeGraph each for all 3 knowledgebases and then 1 listIndex for the final graph. Slack is different since its one big txt file that contains question, answer, summary. Mabye this could be used differently im not sure...
I have a hard time understanding the differences between indices since from my understanding all indices will turn text into a vector full of numbers, not only the vector one, so its a bit confusing...

query_configs
I did not define num_children, child_branch_factor, and am not sure if I defined index_struct_type, query_mode, similarity_top_k or response_mode correctly. Did I miss a parameter, did I chose some parameters wrongly? Does something in my code scream "this will make the query slow!"?

Text Chunking
I did not specify chunking, I assume it defaults to some chunk size? Articles on all the platforms can vary from very short to very long, there is no clear pattern.

Playground
I think I could answer some of my questions would I understand how to use the playground with such a complex structure, would be nice if someone could explain how to test not only different indices for the docs, but also testing different models, different indices for the graph, different query_configs etc...

Summaries
Like already mentioned, I understand those summaries I did make no sense and the LLM won't know from those summaries based on a query to what node to reach... Are those summaries really necessary? Can't I just take the whole content of all 4 sources, index the everything? Im sure I can but would it be more efficient?

Making it future-proof
The short-term goal of this bot will be to identify areas of improvement to give feedback to product owners and have content created. This means articles can be updated, removed, and new ones can be created. I of course want the most updated version of the content in my index. What needs to change to pull/load/index everything again but to skip what didn't change, only indexing the delta and removing whatever was removed?

Info & Code
What I did:
I'm dealing with 4 sources: 3 Knowledge Bases and Slack.
KBs: client facing helpcenter, Confluence, bloomfire
I could not make the Confluence connector nor the official "Confluence by atlassian-python-api" work (i need to basic auth with a base64 encoded "username:apitoken" combo, but both other solutions mentioned ask for username and pwd separately, couldnt figure out a solution), so I grabbed the articles with a simple get_data loop from the relevant spaces with a script and pulling all relevant articles. Each article is one CSV file containing columns: title, content, URL.
All CSVs are in the same folder.
The other 2 KBs I got 1 JSON file per KB containing all articles.
I'm not copying the whole thing, only the most relevant parts I have questions about or a lack of confidence in:
Loading content & Indexing
reader = JSONReader()

# LLMPredictor (gpt-4)
llm_predictor_gpt4 = LLMPredictor(llm=ChatOpenAI(temperature=0, model_name="gpt-4"))

service_context_gpt4 = ServiceContext.from_defaults(llm_predictor=llm_predictor_gpt4)

# Contentful
docs1 = reader.load_data("contentful_articles.json")
index1 = GPTSimpleVectorIndex.from_documents(docs2, service_context=service_context_gpt4)
index1.save_to_disk('contentful_index.json')
index1 = GPTSimpleVectorIndex.load_from_disk('contentful_index.json')

# Confluence
docs2 = SimpleDirectoryReader('docs').load_data()
index2 = GPTSimpleVectorIndex.from_documents(docs2, service_context=service_context_gpt4)
index2.save_to_disk('confluence_index.json')
index2 = GPTSimpleVectorIndex.load_from_disk('confluence_index.json')

Contentful, Bloomfire were handled the same way, loading with JSONReader and indexing with GPTSimpleVectorIndex.
Slack channels were different, due to PII issues I did not get yet access to production slack, so I created a new slack workspace, invited 2 colleagues and we reproduced some discussions from the real slack in where questions were answered. I then created a script that with openAIs help only takes those threads where the LLM decided a question was answered:
def process_messages(messages):
    chatgpt_prompt = (
        "If the following thread contains a question that you deem answered by the responses, "
        "please summarize it as Question: question | Answer: answer | Summary: summary. "
        "If the question is not answered, skip it completely. "
        "The Summary should be <=20 words. I will take your response and create an Index from it.")

Like that I created a .txt file with only those answered questions from the channels of interest and a summary. Loaded this one file with SimpleDirectoryReader and used the same GPTSimpleVectorIndex.
Graph
Summaries
index1_summary = "A client-facing knowledgebase "
index2_summary = "Those are articles from our Confluence Knowledgebase for teams A, B, C, D, E, F and G. Here you can learn about the teams, the product and internal information"
index3_summary = This is a collection of articles with very detailed information about the technical side of X. Very relevant for {employee type Y}"
index4_summary = "A collection of answered questions from multiple company slack chat channels"

(replaced some too specific info with generic stuff)
Building Graph
all_indices = [index1, index2, index3, index4]
index_summaries=[index1_summary, index2_summary, index3_summary, index4_summary]

graph = ComposableGraph.from_indices(GPTTreeIndex, all_indices, index_summaries=index_summaries)
graph.save_to_disk("graph.json")

** Querying **
# set query config
query_configs = [
    {
        "index_struct_type": "simple_dict",
        "query_mode": "default",
        "query_kwargs": {
            "similarity_top_k": 3,
            "response_mode": "tree_summarize"
        }
    },
]

response = index.query(text, query_configs=query_configs)

print(f"Your questions was {text}: \nThe answer is: {str(response)}")

And this is sent to the user to Slack

I am using graph for 2 indices. But querying on graph takes 20 seconds and for individual index takes 5-6 seconds. How to make graph queries faster?

You shouldn't have separate files for each articles, all articles can be in one Json file or a dictionary, with the title, links etc as keys.

For 2 of the KBs I do have 1 JSON per source, each JSON containing all articles. But for confluence I can pull per request up to 50 articles with the content, but we have over 500 articles to pull. I could pull all articles an join then into one JSON, but I feel that there is no difference between that and using the SimpleDirectoryReader.

You shouldn't continue indexing if you've indexed once already. Just load from disk. Only index when it's really needed/data source has been updated with something vital.

I just showed how I indexed because it may be part of the problem, when I run the bot the indexing is commented out and I only load from disk.

let us know if you manage to get this less slow

We are having the same issue with the graph taking forever (around the same 20 seconds).  Has anyone had any luck with their graph working quickly?  If so, which version of llama-index are you on and would you mind sharing your code?
Thanks!

We are still having the same issue with the slow graph response times.  Even with just a couple indexes, it still takes 30 - 40 seconds to get a response from a graph query.  Any hope that this is a priority to make the graph query responses quicker?

Yes, @logan-markewich - any help would be much appreciated!

Hey @logan-markewich,
Thanks for your help with this.  I've passed the link to this thread to our lead AI developer Yuvan Sharma and asked him to respond with the technical details so that you know our setup.
He should be responding here shortly.

@logan-markewich We've been creating indices for a few documents, and creating a simple keyword graph over even 2-3 of these indices leads to a long response time.

For instance, I just made a graph over just 2 indices for two different phones. For a simple comparison query, it took around 630 seconds for a response.

@yuvansharma that seems pretty extreme. Would be nice to have a reproducible case.
Tbh though, the composable graph is pretty much deprecated/unmaintained. I would point to other features instead (sub question engine, router engine, retriever router, agents)

Hi @logan-markewich - Thx for your thorughts.  We do have a reproducable case if you'd like to jump on a quick meeting sometime.  That one is longer than what we've seen in the past, but they've all been a couple minutes at least...
That's a bummer to hear that the composable graph is unmaintained.  We'd definitely have a use for it if it worked efficiently and quickly.  We'll take a look a the sub question engine, retriever router and agents, though.
If you do start to work on the graph, please let @yuvansharma and myself know.
Thanks!


fix: extend validators to validate component registration on editor mode [SPA-1899]

Purpose

Extend validations to make sure component definitionId doesn't not contain invalid characters and has max limit of 32
Validate components during component registration - only on editor mode

Approach

Define a simple validator function for component registrations
[In Editor Mode] Run the validation when a new component is registered
throw an error if the validation fails
Validation doesn't ran on preview/delivery mode to avoid introduction exception in the customer's website.

Can we add tests to this?


Library always pointing to the main net

After setting all variables as shown:
fcl.config() .put(key: .wallet, value: "https://flow-wallet-testnet.blocto.app/api") .put(key: .accessNode, value: "https://access-testnet.onflow.org") .put(key: .authn, value: "https://flow-wallet-testnet.blocto.app/api/flow/authn") .put(key: .scope, value: "email") .put(key: "0xNON_FUNGIBLE_TOKEN_ADDRESS", value: "0x631e88ae7f1d7c20") .put(key: "0xTFC_ITEMS_ADDRESS", value: "0x91a6217c3b70cae8") .put(key: "0xFungibleToken", value: "0x9a0766d93b6608b7") .put(key: "0xFUSD", value: "0xe223d8a629e49c68") .put(key: .env, value: "testnet")
I am getting error like this:
address c19e6dcb15630c30 is invalid for chain flow-mainnet

hi @Soksijada
This repo has been deprecated, please refer to our new one:
https://github.com/Outblock/fcl-swift


Define the related handlers to handler the requests of chart server/repo

add chartserver directory to put chart server code
add controller to coordinate the flow
add base/repo/manipulation handlers to handle requests
add operator to parse more details of chart version
call dep ensure

Pull Request Test Coverage Report for Build 7516

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.006%) to 64.62%

Totals

Change from base Build 7501:
0.006%

Covered Lines:
11085

Relevant Lines:
17154

💛  - Coveralls


added some GET commands

GET api/request/:id added to get requests by user id and GET api/auth added for testing for emails

didnt see that some of this was already in it lol


Can't get it to compile on Pi3 debian raspbian retropie

Am stuck on this :
pi@retropie ~/local/EmulationStation $ cmake .
make
Looking for bcm_host.h
bcm_host.h found
Looking for libMali.so
libMali.so not found
-- Looking for FreeImage...
-- checking for module 'freeimage'
--   package 'freeimage' not found
-- Found FreeImage: optimized;/usr/lib/libfreeimage.so;debug;/usr/lib/libfreeimage.so
-- Boost version: 1.55.0
-- Found the following Boost libraries:
--   system
--   filesystem
--   date_time
--   locale
-- VLC library found
-- Configuring done
-- Generating done
-- Build files have been written to: /home/pi/local/EmulationStation
pi@retropie ~/local/EmulationStation $ make
[  1%] Built target nanosvg
[  2%] Built target pugixml
[ 73%] Built target es-core
Linking CXX executable ../emulationstation
CMakeFiles/emulationstation.dir/src/MetaData.cpp.o: file not recognized: File truncated
collect2: error: ld returned 1 exit status
es-app/CMakeFiles/emulationstation.dir/build.make:920: recipe for target 'emulationstation' failed
make[2]: *** [emulationstation] Error 1
CMakeFiles/Makefile2:246: recipe for target 'es-app/CMakeFiles/emulationstation.dir/all' failed
make[1]: *** [es-app/CMakeFiles/emulationstation.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
pi@retropie ~/local/EmulationStation $

I'll try this out later but it looks like the MetaData object file was corrupted during the build. Does it consistently give this error? Have you tried deleting CMakeFiles/emulationstation.dir/src/MetaData.cpp.o and rebuilding?

Sorry for the noise, it was indeed a corrupt file, I  remove all the .o files in CMakeFiles/emulationstation.dir/src/, relaunched make and it worked.


Fix lint errors

This change is

Coverage remained the same at 44.073% when pulling 4103537ea1c5a56d610f61e85b2aaed17fa2ef29 on aledbf:go-lint into ea7f943160c47a40ceac143176c015bd21860062 on kubernetes:master.


Non breaking space UTF8 character is wiped in QueryParser

Issue summary
As the title mentions, I ran into a specific UTF8 character (�, "%A0" when URL encoded), that when it's passed through the QueryParser component, it (and the entire string it's part of) gets wiped out completely.  I tested with other UTF8 characters, and those seem to be handled fine.  This particular character had given us some trouble in the past when passed through htmlspecialchars (php > 5.4) where it would exhibit the same behavior (the entire string containing the character would be wiped), which is why we have some tests against it.
I ran the same character (same array) through http_build_query and it gets urlencoded and turned into a query string just fine.
System information

Information
Description

League\Uri version
4.2.1 (latest)

PHP/HHVM version
PHP 5.6.28 & PHP 7.0.14 (same output)

OS Platform
Linux

Both versions of PHP have the mbstring and intl extensions installed.
Standalone code, or other way to reproduce the problem
<?php

require_once __DIR__ . '/vendor/autoload.php';

use League\Uri\QueryParser;

$params = [
    'goodutf8' => '神宮館abcdefg',
    // url decoding here because of past editor issues with this character
    'badutf8' => urldecode('%A0TM1TMS061114IP1'),
    'test' => 'true',
];

var_dump($params);

$queryParser = new QueryParser();
$queryString = $queryParser->build($params);

var_dump($queryString);

$queryString = http_build_query($params);

var_dump($queryString);

Expected result
I'd expect both the $queryParser->build() command and the http_build_query() command to have the same output, but in the case of the badutf key they don't.  Here's the expected:
array(3) {
  'goodutf8' =>
  string(16) "神宮館abcdefg"
  'badutf8' =>
  string(16) "�TM1TMS061114IP1"
  'test' =>
  string(4) "true"
}
string(80) "goodutf8=%E7%A5%9E%E5%AE%AE%E9%A4%A8abcdefg&badutf8=%A0TM1TMS061114IP1&test=true"
string(80) "goodutf8=%E7%A5%9E%E5%AE%AE%E9%A4%A8abcdefg&badutf8=%A0TM1TMS061114IP1&test=true"

Actual result
Here's the actual result, note that the badutf key is wiped entirely in the first dump (from QueryParser):
array(3) {
  'goodutf8' =>
  string(16) "神宮館abcdefg"
  'badutf8' =>
  string(16) "�TM1TMS061114IP1"
  'test' =>
  string(4) "true"
}
string(62) "goodutf8=%E7%A5%9E%E5%AE%AE%E9%A4%A8abcdefg&badutf8=&test=true"
string(80) "goodutf8=%E7%A5%9E%E5%AE%AE%E9%A4%A8abcdefg&badutf8=%A0TM1TMS061114IP1&test=true"

@jaydiablo thanks for submitting this bug report. The fix was quite easy to make. Could you test it please so I can merge this to the master. Thanks in advance. Depending on #91 validation a patch version may be released next week

👍Works perfectly (php 5.6 and 7.0), and I ran that branch against a larger suite of URL processing that we have and all tests pass.
Thanks!

version 4.2.2 is out with the fix


🛠 Tooling: Enable @typescript-eslint/naming-convention

Bug Report Checklist

[X] I have tried restarting my IDE and the issue persists.
[X] I have pulled the latest main branch of the repository.
[X] I have searched for related issues and found none that matched my issue.

Overview
https://typescript-eslint.io/rules/naming-convention is a good rule but requires some configuration. I'll admit, I haven't figured out my preferred config for it yet.
Additional Info
Related: #18

I think I'm going to not do this for now. It's real opinionated and I don't yet want to get to that level of stylistic recommendations for consuming repos.

From my experience yeah enforcing a naming convention is pretty low value overall. It's one of those "sure it's nice" things, but there are a lot of edge cases that you end up needing to work around which is a pain.
As an anecdote - the Meta codebase uses no naming convention rules, yet scales to several thousand developers across close to a million JS files and yet the majority of the codebase is written using camelCase/PascalCase/UPPER_CASE.
Really the only time the convention is broken is when someone used to conventions from another language writes one-off code - but nobody really cares about the names cos the code works and the names are still readable - so it's not worth the effort to RC.

...I wonder if we should eject rules like naming-convention into their own package, and tell the community someone needs to set up & maintain it 😄

there is so much call from the community for rules that enforce naming conventions. We used to have a handful of individual rules that did similar but not quite the same things and sometimes conflicted.
I got sick of people asking for changes to things and naming-convention was my solution to that problem - merge them into one mega rule that does everything and shares the same features for any name.
A lot of users do use it because they believe you should enforce the naming convention in a codebase. TBH at one point in my career I fully agreed with them. At this point in time however I've learned that it doesn't matter at all.
Same with member-ordering tbh - at one point in my career I believed it was super important to consistently group and sort things. Now I've found it's best to just sort alphabetically and be done with it.
My reasoning: complex rules brings cognitive overhead. Which means people have to think about things which don't really matter. Which means they get distracted from solving real problems.
The most scalable stylistic linting method IMO:

Naming convention doesn't matter for internal code.
Everything should be sorted alphabetically with autofix on save.
Format the code silently and automatically on save.

Get out of the user's way with stylistic linting so they can focus on problems.


Update request: k3s → ?

ryan-bot has proposed an upgrade PR for 1.29.1: https://github.com/NixOS/nixpkgs/pull/286072
But:
Currently, at top-level, k3s points to k3s_1_27. Considering K3s/K8s needs the upgrade bump to be incremental.

When is k3s going to be bumped?
To which version 1.28 or 1.29? [I suppose 1.28?]
When older packages are going to be removed?
Who is going to remove it?

I think, would be good to document the criteria of bumping k3s.
I'd prefer to keep k3s tag in sync with latest version. (Every time.)
And I think, who ever is running K3s in production should pin a version (instead of k3s rolling) and manually bump version [after doing back-ups, etc].
Thoughts?
ping @euank @mic92 @yajo @wrmilling
Reference: https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/cluster/k3s/README.md

https://github.com/NixOS/nixpkgs/pull/286072

I think we just missed the window to make k3s = k3s_1_28 for NixOS 23.11, I don't think there is any reason we could not go with k3s = k3s_1_29 for current unstable, but I know that has implications for anyone running a cluster currently as they will suddenly jump two major versions with potential breaking changes.

I think we just missed the window to make k3s = k3s_1_28 for NixOS 23.11, I don't think there is any reason we could not go with k3s = k3s_1_29 for current unstable, but I know that has implications for anyone running a cluster currently as they will suddenly jump two major versions with potential breaking changes.

I think we could bump to k3s_1_28 first, then, a week or two later, bump to k3s_1_29. Such as who ever is using nixos-unstable will have a smooth upgrade. I personally only use latest directly. And I have no use for 'stable'. I'm only doing this for others.
Opening an issue for the bump, if you are okay about this, just approve it and let's get it merged. Unless somebody else complains.

Fixed by #287782


Manim issue

When i do the manim side view thingy its says
Manim Extension XTerm
Serves as a terminal for logging purpose.
Extension Version 0.2.16
MSV c:\Users\LENOVO\Desktop\Code>"c:\Users\LENOVO\miniforge3\envs\my-manim-environment\Scripts\manim" "c:\Users\LENOVO\Desktop\Code\main.py" FibonacciSequence
Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "c:\Users\LENOVO\miniforge3\envs\my-manim-environment\Scripts\manim.exe_main.py", line 4, in
from manim.main import main
File "C:\Users\LENOVO\miniforge3\envs\my-manim-environment\Lib\site-packages\manim_init.py", line 13, in
from .config import *
File "C:\Users\LENOVO\miniforge3\envs\my-manim-environment\Lib\site-packages\manim_config_init.py", line 7, in
from .logger_utils import make_logger
File "C:\Users\LENOVO\miniforge3\envs\my-manim-environment\Lib\site-packages\manim_config\logger_utils.py", line 21, in
from rich import color, errors
ModuleNotFoundError: No module named 'rich'
[24304] Execution returned code=1 in 0.912 seconds returned signal null
please help

I would say that this is a clear error message, you are missing the rich Python library, which is imported as
from rich import color, errors
Normally such dependencies should be automatically installed when you install Manim, but since I don't know more about your system or how you installed Manim I can not even guess what went wrong.
You seem to be using a miniconda environment - apart from having heard about it, I don't know how its managed, but you should somehow be able to add the missing dependency using pip install rich
https://pypi.org/project/rich/

I installed manim using conda because i cant use choco or scoop

Its mostly about the side manim view thingy when I do it from the terminal to export it its fine

Its mostly about the side manim view extension for vs code thingy when I do it from the terminal to export it its fine

I suspect you're running different environments. Make sure that all the required packages are installed in my-manim-environment.


Add assignment upload section

The teacher can upload the assignment  and the student can complete the assignment and upload it.

can share some sample ??

Hello maam,
This is a sample.
On Thu, 1 Jun 2023 at 23:30, Sahitya Roy @.***> wrote:

can share some sample ??
—
Reply to this email directly, view it on GitHub
https://github.com/Eduhub-Community/Eduhub-Community.github.io/issues/269#issuecomment-1572538907,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AZHSQUD4UQG3XB7FXQ5EFNDXJDKCVANCNFSM6AAAAAAYXFU3SI
.
You are receiving this because you authored the thread.Message ID:
@.***
com>


ActionDispatch::TestProcess needs to access :file_fixture_path on Rails 6.1

I had a similar problem to @koenpunt in #2349 where fixture_file_upload stopped working.
     NoMethodError:
       undefined method `file_fixture_path' for RSpec::Rails::FixtureFileUploadSupport::RailsFixtureFileWrapper:Class
       Did you mean?  fixture_path
     # /actionpack/lib/action_dispatch/testing/test_process.rb:25:in `fixture_file_upload'
     # /rspec-rails-dd093928f021/lib/rspec/rails/fixture_file_upload_support.rb:5:in `fixture_file_upload'

In my case, it's because I'm on Rails 6.1 alpha. As of these recent changes, ActionController::TestCase now needs access to file_fixture_path within fixture_file_upload.
There's not currently any CI for Rails 6.1 (it's not out yet) but I thought I would get a head start. I added the reader on a separate line, as the fixture_path reader is turning into an accessor on #2370.

Wondering if it makes sense to extend the matrix to run against Rails master. It would be nice to have a failing test along with the fix.

Wondering if it makes sense to extend the matrix to run against Rails master.

I think it will add lot's of work. What about follow alpha,beta and first releases when they are published?

It would be nice to have a failing test along with the fix.
+1

I wasn't sure how rspec-rails has historically coordinated Rails releases.

extend the matrix to run against Rails master

Having rspec-rails generally track rails master would be a recipe for constant breakage on main, but this would work well for a 6.1 branch. When 6.1 is released, the test target could be switched from master to 6.1. But again, I'm not sure how rspec-rails usually does things.

It would be nice to have a failing test along with the fix.

There will be failing tests aplenty when testing against Rails 6.1. I added a failing test specific to the file_fixture_path accessor, but I think its usefulness is debatable.
I'm happy with this remaining open or merged into 6.1 branch is supported. My primary intent was to communicate the problem/fix to others!

Can you rebase this?

✅ Rebased
Though I'm seeing some sort of Thor failure in the suite, looks like the issue has been on main recently:
      Thor::UndefinedCommandError:
11180        Could not find command "mountable_engine?".

It's flaky, I've restarted the job, should most probably pass.

Thanks for researching this and bringing it to our attention, there was more work needed to ensure it worked without deprecation warning but this is closed in #2398


Add NetworkUpdateCompat

NetworkUpdateCompat is a wrapper over NetworkUpdate which swaps Command
and Section when needed.
This function must be used instead of NetworkUpdate to be sure that the
NetworkUpdate call works both with older and newer libvirtd connections.
libvirt on-wire protocol had a bug for a long time where Command and Section
were reversed. It's been fixed in newer libvirt versions, and backported to
some older versions. This helper detects what argument order libvirtd expects
and makes the correct NetworkUpdate call.
Ideally 'NetworkUpdate' in internal/constants/remote_protocol.gen.go
could be renamed to 'networkUpdate' or such, and the new helper would be
'NetworkUpdate', but I don't know if it's possible to do this with the
code generator.
See https://listman.redhat.com/archives/libvir-list/2021-March/msg00054.html
https://gitlab.com/libvirt/libvirt/-/commit/94741bc53e9b9a405fae8f2e96b780c2891b2ff0
https://gitlab.com/libvirt/libvirt/-/commit/b0f78d626a18bcecae3a4d165540ab88bfbfc9ee
This fixes https://github.com/digitalocean/go-libvirt/issues/87

Hey, @cfergeau - Thanks a ton for this sweet PR! 😄
Would you please shoot me an email when you get a chance?
mcowley at digitalocean dot com 🎉


Syntax highlighting color changes after pasting codes into the editor and doesn't match the theme color

When Pylance extension is enabled, syntax highlighting color changes right after the codes are pasted into the editor, as shown in the following video (some codes changed from blue #0550AE to brown color #953800:
https://user-images.githubusercontent.com/10591665/171670432-fe7aa2fd-c7cc-44f0-85de-b0f83c06f9fd.mp4
This is the syntax highlighting color when Pylance was disabled:

How to reproduce:

Install Pylance
Paste codes into the editor

Operating System: macOS Monterey (Version 12.4)
VS Code: Version: 1.67.2 (Universal) Commit: c3511e6c69bb39013c4a4b7b9566ec1ca73fc4d5
Theme used: GitHub Light Default

This is actually expected. The change is due to semantic highlighting from Pylance. You can configure it by following https://github.com/microsoft/pylance-release#semantic-highlighting .


Provide command line tool for interacting with the network.

Provide a command line tool aztec-cli, a wrapper around aztec.js that has higher level apis for better cli experience.
For the first milestone, developers can use it to deploy a contract:
aztec deploy_contract path/to/abi.json [portalContractAddress] [contractAddressSalt] [...constructorArgs]

deploy_contract does the following under the hood:
const abi = JSON.parse(fs.readFileSync('path/to/abi.json'));
const client = new HTTPAztecRPCClient(aztecRpcServerUrl);
const deployer = new ContractDeployer(abi, client);
const txHash = await deployer.deploy(…constructorArgs).send(txOptions).getTxHash();

Terminal will print out contract address and txHash.
Developers can get tx receipt from the txHash:
aztec getTxReceipt txHash

Configuration
Server url and txOptions will be fetched from options or from environment variables:
export AZTEC_RPC_SERVER_URL=localhost:1234
aztec-cli deploy_contract path/to/abi.json 0xabc...132  --aztecRpcServerUrl localhost:45730

It will check options first, then env vars. So from running the above snippet, the url will be localhost:45730.
txOptions has one optional value for now:
interface TxOptions {
  from?: AztecAddress;
}

If from is not provided, it will be the first account in the key store.
Reference
Check out Commander for building this command line tool.

+1 for using Commander for this. I used it for a couple of projects in my last job. Very clean.

@spypsy tagging you since I believe you're working on a cli. This is an old issue from the beginning of the project. If you're working against another issue, please can you close this one?

ah see you closed it, thx 👍


Plugin doesn't create "uploads" folder

doesn't see any files if I create the "uploads" folder manually

what could be the issue? I'm running a Apple M2 chip MacBook, and I think Docker is a bit weird about it...

Hi @11x11alex,
I've added a simple upload form that can be accessed by:

navigating to  http://localhost:3333/upload, or
telling ChatGPT you want to upload a file, after which it will embed a hyperlink it the response taking you to the same /upload url:

This allows you to directly upload files to the running Docker container on your machine (Apple silicon/M2 or otherwise).
You can download the latest image here or run
docker pull ghcr.io/iamgreggarcia/codesherpa:latest

from your command line.
Let me know if that works!

Awesome. I'll check it out soon and I'll confirm if it works. I've sent you an email, did you get it? Would love to get in touch with you


Native scrollframe

The custom scrollingframes are cool, but it's a bad move. Trust me, I've been down this road.
Back before ScrollingFrames were smooth scrolling, I made a custom one for that. It was a pain to maintain, so instead I reworked it into SmoothScroll which added smooth functionality to default ScrollingFrames.
The main problems with using custom are:

Maintaining
Input methods (touchpad, gamepad, touchscreen)
OS level input settings we cant read
Input sinking

If you have nested scrollframes our custom method does not handle those correctly.
I've rewritten the scrollframe component to essentially just be a stylized bar overlay on top of an normal scrollingframe so all that stuff is correctly handled by the engine- all we really care about here is the QT styled scrollbar anyway, we aren't adding custom functionality (beyond the up/down buttons, but I've included those in this version too).

I've created a draft for you to give early feedback. I know I still need to add the color themes and give instances clear names.

Some things that'll need to be done before this is merged with the main branch:

ScrollFrame's main frame should have a background color that matches the current theme and should not be transparent. The theme background color should only be applied if a BackgroundColor3 isn't specified.
Decide whether or not the passing BackgroundColor3 to ScrollFrame will override the colors for the handles and bar as well
ScrollFrame is a base object that can do both horizontal and vertical scrolling with scroll bars
LayoutScrollFrame is a component that uses ScrollFrame but handles the CanvasSize for both UIPadding and a UILayout


Feat(eos_cli_config_gen): Add custom comments at the top of the rendered configuration.

Change Summary
Add custom comments at the top of the rendered configuration
Related Issue(s)
Fixes #2271
Component(s) name
arista.avd.eos_cli_config_gen
Checklist
Repository Checklist

[x] My code has been rebased from devel before I start
[x] I have read the CONTRIBUTING document.
[x] My change requires a change to the documentation and documentation have been updated accordingly.
[x] I have updated molecule CI testing accordingly. (check the box if not applicable)

Confused about this line. Are we expecting the comments to be displayed here? @Vibhu-gslab Otherwise, LGTM

Could you please specify the line number.


Redis VSS with 'LIMIT 0 '

I have a use case - need to find all records that have VSS score higher than specific threshold, there are no option to include this into filter (???) so decided to query for all results and then filter on Java side.
Based on docs, you can add LIMIT 0 <K> to query to get all results. See https://redis.io/docs/interact/search-and-query/search/vectors/#algorithm-specific-attributes. Is this supported?

Can you post your code?

I used the code from VSS demo. I've tried something like (from ProductController):
      List<Product> products = stream
          .filter(Product$.SENTENCE_EMBEDDING.knn(0, product.getSentenceEmbedding()))
          .sorted(Product$._SENTENCE_EMBEDDING_SCORE)
          .limit(K)
          .collect(Collectors.toList());

but it does not work. (not sure that even intended to work in this way)

I used the code from VSS demo. I've tried (from ProductController):
      List<Product> products = stream
          .filter(Product$.SENTENCE_EMBEDDING.knn(0, product.getSentenceEmbedding()))
          .sorted(Product$._SENTENCE_EMBEDDING_SCORE)
          .limit(K)
          .collect(Collectors.toList());

but it does not work. (not sure that even intended to work in this way)

Sorry @qUser13 I don't quite get what the issue is, could you create a reproducer app that shows the problem. That'll be the easiest way for me to fix it. Cheers.


MAINT: use f-string method

Updated to the f-string method for cleaner and better understanding of the code.

@cyai Can you please sign CLA. Thank you!

@cyai Can you please sign CLA. Thank you!

Signed! Could you proceed? Thank You

Please fix the lint error.

Thanks for the PR.

This branch has conflicts that must be resolved

Please fix.

Fixed!

Seems that the merge conflict is not fixed.

Sorry, merge conflict is not resolved. If you're facing git issues, you can always just close the PR and recreate it from master.


Add detection of private packages and don't publish them to NPM

Closes #28
Again, tested with --dry-run and does indeed not output the npm publish line when private is set to true in package.json.

:deciduous_tree: Thank you, Aldwin!


Fix illegal reflective access warning from protostuff

Signed-off-by: Kaituo Li<EMAIL_ADDRESS>Description
When using prostuff 1.7.4 under java9+ we get an Illegal reflective access warning due to protostuff 1.7.4's incompatibility with java9+.  The issue is solved in https://github.com/protostuff/protostuff/commit/b6ad96f229218a835ec3f7232fa8ee129a6d3806  and will be available in 1.8.0. Since  protostuff 1.8.0 is not released, this PR built 1.8.0 packages and made AD depend on the local packages directly. With the change, the illegal reflective access is gone.
Testing done:

Tested both single-stream and HCAD real-time detectors.
Tested backward compatibility: start a 1.2.0 cluster with protostuff 1.7.4, start detectors and wait for model checkpoints to be generated.  Reinstall AD plugin with locally packaged protostuff 1.8.0. Verified tha old model can come up and detectors can produce results.

Issues Resolved
https://github.com/opensearch-project/anomaly-detection/issues/263
https://github.com/opensearch-project/anomaly-detection/issues/319
Check List

[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Just realized this package was compiled under JRE 11. Meaning code can no longer be run after compiling AD with anything below (JRE 8). Is this an issue?


Integration Test - Release - 2019/05/16 23:09:09

Auto create for integration test. AnotherAuthorUpdatedArticleFlow

@PRMergerTest2 : Thanks for your contribution! The author(s) have been notified to review your proposed change.


enable CDL schematic conversion

At the moment gplugins support YAML schematic representation but recently some users asked for CDL support to use gdsfactory to generate cadence compatible schematics
How about suppor converting from YAML to CDL?

https://gdsfactory.github.io/gplugins/notebooks/11_get_netlist_spice.html


E2507 IAM Resource Policy statement should have Principal or NotPrincipal

When cfn-lint is ran against the template aws/FSS-Scanner-Stack.template this error occurs:
E2507 IAM Resource Policy statement should have Principal or NotPrincipal
This is the resource with the issue, the principal is set to NoValue.  Everything appears to work ok, but I think the policy still needs to be adjusted somehow to pass this check:
ScannerQueuePolicy:
Type: AWS::SQS::QueuePolicy
Properties:
PolicyDocument:
Version: 2012-10-17
Statement:
- Sid: send_message_from_accounts
Effect: Allow
Principal: !Ref AWS::NoValue
Action:
- sqs:SendMessage
Resource: !GetAtt ScannerQueue.Arn
Queues:
- !Ref ScannerQueue

Actually the policy is not required, but it cannot be deleted because when updating stack stuff which our backend's done to the policy will be deleted as well. I'm not in favor of adding extra principals just for passing this check.

@richardsonky any thoughts on that?


httping fails to build

error: builder for '/nix/store/29zy295s47w7nlx0bd8d82kiyp883spk-httping-2.9.drv' failed with exit code 2;
       last 10 log lines:
       > void update_stats(stats_t *resolve, stats_t *connect, stats_t *request, stats_t *total, stats_t *ssl_setup, int n_ok, int n_fail, const char *last_connect_str, const char *fp, char use_tfo, char dg, stats_t *st_to, stats_t *tcp_rtt_stats, int re_tx, int pmtu, int tos, stats_t *close_st, stats_t *t_write, int n_cookies, char abbreviate, stats_t *stats_header_size)
       >                                                                                ^
       > nc.c:738:244: warning: unused parameter 're_tx' [-Wunused-parameter]
       > void update_stats(stats_t *resolve, stats_t *connect, stats_t *request, stats_t *total, stats_t *ssl_setup, int n_ok, int n_fail, const char *last_connect_str, const char *fp, char use_tfo, char dg, stats_t *st_to, stats_t *tcp_rtt_stats, int re_tx, int pmtu, int tos, stats_t *close_st, stats_t *t_write, int n_cookies, char abbreviate, stats_t *stats_header_size)
       >                                                                                ^
       > nc.c:738:255: warning: unused parameter 'pmtu' [-Wunused-parameter]
       > void update_stats(stats_t *resolve, stats_t *connect, stats_t *request, stats_t *total, stats_t *ssl_setup, int n_ok, int n_fail, const char *last_connect_str, const char *fp, char use_tfo, char dg, stats_t *st_to, stats_t *tcp_rtt_stats, int re_tx, int pmtu, int tos, stats_t *close_st, stats_t *t_write, int n_cookies, char abbreviate, stats_t *stats_header_size)
       >                                                                                ^
       > 4 warnings and 1 error generated.
       > make: *** [<builtin>: nc.o] Error 1
       For full logs, run 'nix log /nix/store/29zy295s47w7nlx0bd8d82kiyp883spk-httping-2.9.drv'.
error: 1 dependencies of derivation '/nix/store/l44iyn6hij0bk9y3hzdcxmkl4j0lqg8b-system-applications.drv' failed to build
error: 1 dependencies of derivation '/nix/store/cphasgk20rb6284d9sd80vh92cx0si2h-darwin-system-23.05.20221125.32802ab+darwin4.6349b99.drv' failed to build

Your error log is clopped and does not show the actual error. Can you post a few more lines where actual error hides?
Also, master and staging-next seems to build httping just fine for me. Which branch do you see failing?

error: 1 dependencies of derivation '/nix/store/cphasgk20rb6284d9sd80vh92cx0si2h-darwin-system-23.05.20221125.32802ab+darwin4.6349b99.drv' failed to build

I guess you are on darwin? I think it is failing on hydra then, log: https://hydra.nixos.org/build/199361431/nixlog/2

Aha, perhaps it's just a matter of missing #nclude <unistd.h>.
clang -Wall -W -Wextra -pedantic -D_FORTIFY_SOURCE=2  -DVERSION=\"2.9\" -DLOCALEDIR=\"/usr/share/locale\" -DNC -DFW -D_DEBUG -ggdb   -c -o nc.o nc.c
In file included from nc.c:22:
./utils.h:19:15: error: unknown type name 'useconds_t'
void myusleep(useconds_t v);
              ^

This seems to be enough to fix the upstream build https://github.com/folkertvanheusden/HTTPing/pull/8. We can pull it downstream once it gets accepted (or after some reasonable time).

Proposed nixpkgs fix as https://github.com/NixOS/nixpkgs/pull/203030


Policies: add Helm Chart for deployment

Description
This PR introduces the Helm chart to deploy the OPA policies.
The policies are configured to be deployed in dryrun mode, hence not yet effectively enforcing the constraints.
This choice originates from initial tests performed on a KinD cluster, which uncovered some unexpected behaviors of the CrownLabsInstanceTemplateReference policy (I have not yet tested the other, but it is much more complex and has security implications).
Such behaviors include:

The policy returns a message that a namespace does not exists even though it is present (but it contains no templates)
The policy does not prevent the creation of certain invalid instances. The problem appears to be related to the access of non-existing fields in a map, which prevents the violation from being issued. An example of a failing test (no violation is triggered even though the instance is invalid) can be obtained with the following patch:--- a/policies/policies/instance-template-reference_test.rego
+++ b/policies/policies/instance-template-reference_test.rego
@@ -117,7 +117,7 @@ data_inventory(ns1, ns2, ns3) = output {
                              "namespace": ns3,
                      },
              }}}},
-               ns2: {"crownlabs.polito.it/v1alpha2": {"Template": {}}},
+               ns2: {},
              "default": {"crownlabs.polito.it/v1alpha2": {"Template": {"temp1": {
                      "apiVersion": "crownlabs.polito.it/v1alpha2",
                      "kind": "Template",

From my point of view, it is necessary to better analyze the different corner cases before deploying those policies in production
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Please also note any relevant details for your test configuration.

[x] Deploying the Helm chart in a local KinD cluster and performing some manual tests

/rebase

/rebase

/merge


Getting Error While doing mvn clean install -P release

Failed to execute goal org.apache.maven.plugins:maven-gpg-plugin:1.6:sign (sign-artifacts) on project browsermob-proxy: Unable to execute gpg command: Error while executing process.
I am getting this error.

I downloaded the source code
Ran mvn install -P release.
I am getting this error in Parent project

Please help me with this.

Try the below command inside the project directory
mvn clean install -Dhttps.protocols=TLSv1.2


Update sbt-buildinfo to 0.9.0

Updates com.eed3si9n:sbt-buildinfo from 0.7.0 to 0.9.0.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention @scala-steward in the comments below.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [{ groupId = "com.eed3si9n", artifactId = "sbt-buildinfo" }]

labels: semver-minor

Coverage remained the same at 88.312% when pulling 7d17b3f089858672dbdceb40e3d4b96858dc668c on scala-steward:update/sbt-buildinfo-0.9.0 into cc3fea59e1fb99a3fb652edac4bbdbe7f14ea57f on iheartradio:master.


fix(@ngtools/webpack): fix no real error information.

When node want to resolve module rxjs but he was not installed this try catch hide real error and throw not very useful error message.

Can you show an example of how the message is improved? I think the current one is quite clear.

@rayros Please provide an example of the message, and please fix the linting errors. Thanks!


Feature Request: Notify in VSCode

When I use jupyter-notify in VSCode Jupyter Notebook, it gives such error:

Is any plan to support VSCode Jupyter Notebook?

I am getting the same error, any chance this is going to be looked at in the near future?


How to understand TemplateDelegate template with PageEvent?

renderer#rednerDocument call
        // Theme must be set as this is only called in render, and render ensures theme is set.
        page.template = page.template || this.theme!.resources.templates.getResource(page.templateName)!.getTemplate();
        page.contents = page.template(page, { allowProtoMethodsByDefault: true, allowProtoPropertiesByDefault: true });

page.template defination in event.ts
    /**
     * The template that should be used to render this page.
     */
    template?: TemplateDelegate;

handlebars show Handlebars.template with simple json data
var source = "
Hello, my name is {{name}}. I am from {{hometown}}. I have " +
             "{{kids.length}} kids:
" +
             "
{{#kids}}
{{name}} is {{age}}
{{/kids}}
";
var template = Handlebars.compile(source):
var data = { "name": "Alan", "hometown": "Somewhere, TX",
             "kids": [{"name": "Jimmy", "age": "12"}, {"name": "Sally", "age": "4"}]};
var result = template(data);

How to understand page.template(page, ...), param page is PageEvent, so complex!

As discussed somewhat in gitter, this is an unanswerable question. I agree that the current theming system is horrible, but don't really have any good way to propose how to understand it. Every time I look at it I learn something new about it.


MM² Icon

Create MM² icon
Allow to change default electron icon to this icon

Out of curiosity: When and where is the icon displayed?

@rejas: wait, I will change "MM" color because it's so darkness... we see nothing lol

@KristjanESPERANTO
something like this ?

and task bar (when minified or when multi-screen using)

@veeck, it's ok (i think)

so this is the new favicon when running under electron (which is only visible if not running fullscreen), but for server only we have another favicon in index.html. So should we use the same icon for both?

that index.html favicon is this base64 encoded small string?

I think so, displayed is

you can replace this line e.g. with
<link rel="icon" type="image/png" href="path/file.png">

to test other icons (the path must served by mm)

will not works mm2.png have no entry on express

that was meant with "the path must served by mm", for testing is must be copied to e.g. config or modules or ...

add an  app.get to serve mm2.png in server.js ;)
I'm in lazy mode :D

unifying them would make sense. and maybe also ask in the forum if we want to make a "design the mm2 logo" contest (no offence against your design, bugsounet, but we are developers, no designers :-)

sure it's an first approach... I have planned to do that in next version of MM²