pymethods: support buffer protocol

For #1884
Implements the buffer protocol via two new methods __getbuffer__ and __releasebuffer__.
They are unsafe because they operate on ffi::Py_buffer directly. I considered making a "safe" API, however also at this point in time I think it's better if we just get the migration away from #[pyproto] complete. We can revisit a safe design later if needed.
Cython implements these methods exactly the same way. https://cython.readthedocs.io/en/latest/src/userguide/buffer.html

I'm wondering if the method needs to be unsafe. Unsafety already needs to be used internally to fill the Py_buffer* fields.

I'm wondering if the method needs to be unsafe. Unsafety already needs to be used internally to fill the Py_buffer* fields.

They need to be unsafe if these functions are callable from Rust.

They need to be unsafe if these functions are callable from Rust.

Exactly.
For the sake of putting it out there, I looked at a couple of alternatives:

These functions taking &mut ffi::Py_buffer instead of *mut. I don't think that's actually sound; in the general case running any Python APIs inside these functions could theoretically cause something else to modify the buffer structure (even if in practice it probably won't happen).
__getbuffer__ could be made safe if it returned ffi::Py_buffer instead. Unfortunately this same idea can't be applied to __releasebuffer__, as it's not producing a new ffi::Py_buffer.
__releasebuffer__ could probably be made safe by having some kind of safe PyBufferInfo wrapper which doesn't hold references for indeterminate length of time. It seemed like a lot of work so I felt it was easier to just keep to the ffi structures for now.

I'll give this a few more days in case anyone has further concerns and / or ideas, otherwise I'll merge this as-is in order to make progress towards 0.16.


[FEATURE]: Event cards

Detailed description

The image of the card and the content is getting mixed.
The cards that have the image with contents on it are getting mixed with the contents on the card and making it a bit hard to read the content on the card properly.

Context
I would like to add some stylings to the event cards to make the content visible clearly and look simple yet attractive.

Possible implementation
No response
Additional information
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

@anamika7153 so how are you gonna solve this issue? I mean how are you gonna implement the changes and can you explain a little bit?

@anamika7153 so how are you gonna solve this issue? I mean how are you gonna implement the changes and can you explain a little bit?

@Deepu178  I will add a dark overlay between the background image and the content.

@anamika7153 I am assigning this issue and looking forward to see the changes that you have suggested. Are you contributing under GSSoC?

@anamika7153 I am assigning this issue and looking forward to see the changes that you have suggested. Are you contributing under GSSoC?

Yes, I'm GSSoC'23 contributor


[release-1.9] [kiali] update to kiali 1.29

This is an automated cherry-pick of #30438

@istio-testing: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio_release-1.9
b69ac876bf11316425b469184a65122180b4d901
link
/test unit-tests_istio_release-1.9

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@istio-testing: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio_release-1.9
b69ac876bf11316425b469184a65122180b4d901
link
/test unit-tests_istio_release-1.9

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@googlebot I consent

@googlebot I consent

@googlebot I consent

@googlebot I consent

@googlebot I consent

@googlebot I consent

I don't know who else needs to consent - all authors (@jmazzitelli @shamsher31) have consented - not sure who it is waiting for.

I don't know who else needs to consent - all authors (@jmazzitelli @shamsher31) have consented - not sure who it is waiting for.

@stevenctl PTAL.

@stevenctl PTAL.

Someone needs to just take out the "cla: no" and add the "cla: yes" label. I cannot do that apparently.

Someone needs to just take out the "cla: no" and add the "cla: yes" label. I cannot do that apparently.

@howardjohn can you please take care of the CLA?

@howardjohn can you please take care of the CLA?


Jakarta Whiteboard WebSocket Specification

I now have poked around the spec, checked the API and some reference implementaion for Jakarta Websockets. For the Sake of an OSGi Whiteboard Service it seems feasable to support only Annotated Endpoints as described in the spec as "annotated endpoint"
To support self-publishing (referenced in the spec as programmatic endpoint) as well, then I can add methods to the Runtime service to obtain necessary interface, I just wondering if this is a case we want to support at all.
One problem is that the specification mandates, that after the initial configuration phase no more server endpoints have to be registered anymore and that there is only one container per VM ... both things can obviously not promised in OSGi.
So given that, one can specify that OSGi Jakarta Websockets is reusing the spec and can handle endpoints annotated according to the spec, but such an implementation can not be compliant to the spec in its current form.
Another pitfall is, that one can support clients in a Whiteboard fashion, but it seems a bit odd to handle practically one has to register one service instance per client connection and somehow tie the connection state to that registration.
Next quirk is, that only one listener per type is supported on an endpoint, so having the whiteboard collecting listeners and there are possibly different variants of them so installing proxies seem not feasible at all as there is a high risk of clashes.

I have investigated a bit further and it seems there is a standard compliant way to process an UPGRADE request see here in the servlet spec.
So an implementation of the Jakarta Websockets can implement a Filter on a context path using servlet whiteboard (what then becomes the root of the Webservice path) and listen for upgrade request, initiate the described procedure and then take over the stream, scheduling the request for processing by the webservice streams.
So this can work here like for Jakara Webservice specification, where we recommended to integrate with the Servlet Whiteboard but do not mandate its usage. This of course does not prevent an implementation to implement both in one service if it is supported native by the underlying container.

Some code samples used in the spec can be found here as executable maven project:

https://github.com/laeubi/osgi-websockets

Here is the current state as a PDF for review if anyone is interested in the current state:
osgi.cmpn-8.1.0.pdf


Add getName()  method to DynamicStringSetProperty

Add a getName() method to DynamicStringSetProperty that returns the name of the property that was passed to the constructor.

Closing this for now since there has been no traction on this.


TRUS-2701 : Validating dates

…ssages.

Date of birth:

min date: 01/01/1500
max date: date of death or now if user had answered no to do you know date of death

Date of death:

min date: date of birth or 01/01/1500 if user had answered no to do you know date of birth
max date: trust start date


License?

Hi @philhawksworth ! Would you mind adding a license file to this project? 🙏 ty!

EleventyOne is MIT licensed.
package.json lists it on the line 6:
https://github.com/philhawksworth/eleventyone/blob/67f0be3f17b6cbc1f2306983e3ce460967dbe58c/package.json#L6
Separate text file for the license would certainly be useful addition, though,
as it would display the license directly on the repository level.

Good catch. Added a License file to make it clear that this is MIT

thank you, @philhawksworth !


Delay in opening modals/shortcuts for application hosted in aws lambda + api gateway

I have an application hosted on aws using lambda + api gateway and I'm having a delay problem when opening modals, shortcuts etc
This happens when I open some functionality right after uploading to the cloud. After I open it for the first time, all app features start to open quickly. It looks like some kind of cache and then it crashes again like in the gif presented.

I tried to debug in several ways but I still couldn't understand the source of the problem. I don't know if it's in the gateway api (I even tried to enable the api cache but it didn't work), or in the application or in some other type of slowness along the way that I couldn't identify.
I put several prints with the execution time to try to identify if it was in some part of my code, but as you can see, when I click to open a sortcut for example, the load time is less than 2 seconds until the modules are full loaded.
When I open it for the second time, only the print of "start" appears, suggesting that it opened directly from some cache.

I've already tested it in two different AWS accounts and the same problem occurs.

Hi @r4ibOm, thanks for asking the question!
With the limited information given here, I cannot guess what the exact reason of your issue. However, probably your issue may be related to common "cold-start" issue (=overhead for spinning up a new Lambda runtime and loading resources). Refer to my response to a past question here: https://github.com/slackapi/bolt-js/issues/816#issuecomment-786360926
Also, searching past issues under slackapi org may be helpful too: https://github.com/search?q=org%3Aslackapi+aws+lambda&type=issues
I hope this was helpful to you and you will figure the cause out soon!

Hi @seratch ,
Thank you for the tips. I'm new to this cloud world and I didn't know about this cold start.
I did some research and analysis on my project.
In my requirements, there is only slack bold and requests libraries  As you can see my project is small and has only 2.3 MB:

Being able to reduce even more by removing some libraries that already come native to aws such as boto 3 and botocore, even so the problem persists. I'm still researching to try to understand what might help.


Add a page for raffling things

We should have a page that organizers can visit to raffle things off. The Meetup API can be leveraged to make the event from that time period available, and to list attendees and provide images to identify winners.

I want to try this.

We haven't had a giveaway in a while, but when we do I do the following:

Get the attendees from meetup (as a CSV)
Remove the organizers
Randomly order them and take the top ~5-7 attendees. I take more than 1 because some people RSVP and then don't attend.

If this process could be automated, even with a management command (since it doesn't need to be public per se), that would save some time and make it repeatable. I haven't done much with the meetup API beyond what's already in the repository. However, if there's a way to take a meetup URL/event ID as input and output a randomized list of attendees, that seems useful.
Overall, I don't think this is high priority but it would be useful for when we have a giveaway.


Update to the allowed base image

Doesn't matter, but we should still avoid this if possible.

👍

:+1:


Fallback to existing SQS/SNS entity if creation failed

MassTransit always try to create SQS queue or SNS topic when starting up application. This is fine as AWS API effectively returns "get information" data if "create entity" request contains same name as existing resource and same tags/attributes. The latter part is tricky as it makes it extremely difficult to update those on live system.
Additionally it seems if we're trying to send (not publish) something onto queue that is does not belong to our project MT fails as QueueCache.GetByName() does not even attempt to read data from AWS API.

Merged.

Merged.


feat:Add cmd to create file graphql at Application

PR python3 manage.py app_name -h option command.
Execute python3 manage.py app_name -n myapp
Rename command to maker
Execute python3 manage.py maker -n myapp

default value default=[] Not required.
python3 manage.py maker -n myapp -e id
or
python manage.py maker -n myapp


leading zeros are missing for the generated verification code

Steps to reproduce
a2fa generate xxxxxxxxx

Expected behaviour
003269
Actual behaviour
3269

@blazeoffire0
This bug has been fixed
fix the error of missing the leading zero in the code; update version


Add board options to README

It would be helpful to include the correct non-default board options in the README.
Board: Adafruit Feather RP2040 USB Host
CPU Speed: 120 MHz
USB Stack: Adafruit TinyUSB

After installing this library and its dependencies using the Arduino IDE 1.8.19 library manager, the example code builds without problems. So far an IK has been confirmed to work on a Mac M1 running Ventura 13.4 and an iPad M1 running iPadOS 16.5.

thank you for the issue, I have update readme to include instruction for compiling https://github.com/adafruit/Adafruit_IntelliKeys#compiling-from-source


Publish 1.8.1

Ready for update? Wait until I manged everything.

General Wrex fixed the "undefined" bug in the UI. Thanks to Tresmos for the information!
Lasse updated his Store Guild Info Mod and added the option to check if the server has Embed Links enabled.
And the biggest update is the new Send JSON to WebAPI! It allows you to edit the data located on the Internet.
And EliteArtz fixed the Link in the Splash Screen action.
That's all for today. Have a nice week and merry Christmas everyone!


fix(date-picker): Update flatpickr

closes #483

Any notable improvements with this?

It fixes issue #483. At first, the date was not being saved when you tab out of the first input in the range selection.


Pending return verbage

Good day all!!
The question I have is regarding after a Chromebook is returned. When you use the GnG app on the device to return the Chromebook, the Console for GnG shows the device as 'Pending Return'. However, there isn't a way to confirm the return. Is this expected results? Is this the way the system labels a device that has been taken, used, and returned?

Thank you for your thoughts!
Josh Casper

Pending return means that the user is no longer on the hook and won't be nagged about returning the device. We found this important in-case there was any period of time where a device sat on a shelf without a new user grabbing it and starting a new loan.
To truly return a device, there are two ways to accomplish this:

A user other than the one who previously had the device borrows the device and logs in.
A tech audits the device to a given shelf in the Grab and Go web app.

Hopefully this helps. Let us know if you have any other questions!

Thank you very much for that explanation! While I feel that the phrase 'pending returned' is misleading and a better status would be 'Pending new Check-out', I do understand the logic that goes into the phrase 'Pending Return'. Would it be possible for your explanation to be added to the documentation and a more thorough description of that status?
Thank you again for taking the time to help and for the excellent resolution to my issue.
Sincerely,
Josh Casper


The documentation and cli both confirm the --encrypt-format-all option but he running extension reports it is not supported, see below.

https://docs.microsoft.com/en-us/cli/azure/vm/encryption?view=azure-cli-latest
[AzureDiskEncryption] 9716: [Info] Executing: yum install -y cryptsetup lsscsi psmisc cryptsetup-reencrypt lvm2 uid at patch procps-ng util-linux gcc libffi-devel openssl-devel python-devel
AAzureDiskEncrytion] 9716: [Info] Executing: pip show adal
AzureDiskEncrAzureDiskEncryption] 9716: [Info] Command pip show adal failed with return code 1
stdout:
stderr:
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[AzureDiskEncryption] 9716: [Info] Executing: pip install --upgrade six
[AzureDiskEncryption] 9716: [Ino] Executing: pip install adal
[AzureDiskEncryption] 9716: [Info] Data volume /my_data is mounted from /dev/sd1
[AzureDiskEncryption] 9716: [Info] Data volume /my_data2 is mounted from /dev/sdd1
2017/11/20 11:34:03.083512 INFO Event: name=Microsoft.Azure.Security.AzureDiskEncryptionForLinux, op=Install, message=Launch command succeeded: main/handle.py -install, duration=0
2017/11/20 11:34:03.090569 INFO [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-<IP_ADDRESS>9306] Enable extension [main/handle.py -enable]
[AzureDiskEncrytion] 10117: [Ifo] Enabling exension
[AzureDiskEncryption] 10117: [Ifo] Public settings:
{
    'AADClientCertThumbprint': null,
    'AADClientID': ,
    'EncryptionOperation': 'EnableEncryptionFormatAll',
    'KeyEncryptionAlgorithm': 'RSA-OAEP',
    'KeyEncryptionKeyURL': 'disks.vault.azure.net/keys/testl01/a0f',
    'KeyVaultURL': '*',
    'SequenceVersion': '656a0648-d89f-48d7-872b-7403d768ef25',
    'VolumeType': 'ALL'
}
[AzureDiskEncryption] 10117: [Info] Encryption operation EnableEncryptionFormatAll is not supported
[AzureDiskEncryption] 10117: [Info] Executing: lvs --noheadings --nameprefixes --unquoted -o lv_name,vg_name,lv_kernel_major,lv_kernel_minor
AzureDiskEncryption] 10117: [Info] Data volume /my_data is mounted from /dev/sdc1
[AzureDiskEncryption] 10117: [Info] Data volume /my_data2 is mounted from /dev/sdd1

@digeler  - Please file a bug with github.com/Azure/azure-cli, the inline documentation is maintained by the CLI engineering team and this sounds like a bug or misfeature.


dcgm-exporter falied so start on GKE cluster (v1.16.11-gke.5)

I've GKE cluster with GPU node pull.
The GPU nodes has valid labels, the nvidia device plugin pods are running on each GPU node and the nvida driver daemon set was deployed as well.
The K8S detects 1 allocatable GPU.
However, when I'm deploying the kubectl create -f https://raw.githubusercontent.com/NVIDIA/gpu-monitoring-tools/2.0.0-rc.12/dcgm-exporter.yaml  the dcgm-exporter pod is in CrashLoopBackOff with the following error:
time="2020-07-21T18:49:33Z" level=info msg="Starting dcgm-exporter"
Error: Failed to initialize NVML
time="2020-07-21T18:49:33Z" level=fatal msg="Error starting nv-hostengine: DCGM initialization error"

The exact same setup works on AKS and EKS clusters without any issue.
Is there any limitation to use the dcgm-exporter on GKE?

I have exactly the same issue right now.

Same issue in EKS cluster now.

Same with on-premise cluster

Same with on-premise OKD 4.4 cluster

Well, I think I solved the problems

@tanrobotix can you share your solution?

In my case
The /etc/docker/daemon.json is
{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}

The default-runtime is not set. It should be set with the absolute path
{
    "default-runtime": "nvidia",
    "runtimes": {
        "nvidia": {
            "path": "/usr/bin/nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}

Then reload daemon and restart docker service
systemctl daemon-reload
systemctl restart docker

I am having the same issue with an on-premise cluster (1 VM-based master node, 2 DGX Station GPU nodes) setup via Ansible and DeepOps.
$:~/deepops# kubectl get nodes
NAME     STATUS                        ROLES    AGE   VERSION
gpu01    NotReady,SchedulingDisabled   <none>   2d    v1.18.9
gpu02    Ready                         <none>   23h   v1.18.9
mgmt01   Ready                         master   2d    v1.18.9

$~/deepops# kubectl get pods
NAME                                                              READY   STATUS             RESTARTS   AGE
dcgm-exporter-1608298867-jstgm                                    0/1     CrashLoopBackOff   242        19h
dcgm-exporter-1608298867-n52g2                                    1/1     Running            0          19h
gpu-operator-774ff7994c-gdpdl                                     1/1     Running            29         29h
gpu-test                                                          0/1     Terminating        0          23h
ingress-nginx-controller-6b4fdfdcf7-sb5hs                         1/1     Running            0          28h
nvidia-gpu-operator-node-feature-discovery-master-7d88b984j9grb   1/1     Running            3          29h
nvidia-gpu-operator-node-feature-discovery-worker-jpc24           1/1     Running            49         29h
nvidia-gpu-operator-node-feature-discovery-worker-lzn58           1/1     Running            26         29h
nvidia-gpu-operator-node-feature-discovery-worker-wfqgn           0/1     CrashLoopBackOff   195        19h

$~/deepops# kubectl logs pod/dcgm-exporter-1608298867-jstgm
time="2020-12-19T09:21:20Z" level=info msg="Starting dcgm-exporter"
Error: Failed to initialize NVML
time="2020-12-19T09:21:20Z" level=fatal msg="Error starting nv-hostengine: DCGM initialization error"

/etc/docker/daemon.json however looks fine on both GPU nodes. Not sure if the persistent NotReady status of one of the nodes is related to this dcgm-exporter issue or not.
{
    "default-runtime": "nvidia",
    "default-shm-size": "1G",
    "default-ulimits": {
        "memlock": {
            "hard": -1,
            "name": "memlock",
            "soft": -1
        },
        "stack": {
            "hard": 67108864,
            "name": "stack",
            "soft": 67108864
        }
    },
    "runtimes": {
        "nvidia": {
            "path": "/usr/bin/nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}

Many thanks in advance for any advice #57

Hi. Based in this stack overflow question, we solved using:
apiVersion: v1
kind: Namespace
metadata:
  name: monitoring
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: dcgm-exporter
  namespace: monitoring
  labels:
    app.kubernetes.io/name: dcgm-exporter
    app.kubernetes.io/instance: dcgm-exporter
    app.kubernetes.io/version: "2.1.1"
    app.kubernetes.io/component: "dcgm-exporter"
---
apiVersion: v1
kind: Service
metadata:
  name: dcgm-exporter
  namespace: monitoring
  labels:
    app.kubernetes.io/name: dcgm-exporter
    app.kubernetes.io/instance: dcgm-exporter
    app.kubernetes.io/version: "2.1.1"
    app.kubernetes.io/component: "dcgm-exporter"
spec:
  type: ClusterIP
  ports:
  - name: "metrics"
    port: 9400
    targetPort: 9400
    protocol: TCP
  selector:
    app.kubernetes.io/name: dcgm-exporter
    app.kubernetes.io/instance: dcgm-exporter
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: dcgm-exporter
  namespace: monitoring
  labels:
    app.kubernetes.io/name: dcgm-exporter
    app.kubernetes.io/instance: dcgm-exporter
    app.kubernetes.io/version: "2.1.1"
    app.kubernetes.io/component: "dcgm-exporter"
spec:
  updateStrategy:
    type: RollingUpdate
  selector:
    matchLabels:
      app.kubernetes.io/name: dcgm-exporter
      app.kubernetes.io/instance: dcgm-exporter
      app.kubernetes.io/component: "dcgm-exporter"
  template:
    metadata:
      labels:
        app.kubernetes.io/name: dcgm-exporter
        app.kubernetes.io/instance: dcgm-exporter
        app.kubernetes.io/component: "dcgm-exporter"
    spec:
      serviceAccountName: dcgm-exporter
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: cloud.google.com/gke-accelerator
                operator: Exists
      containers:
      - env:
        - name: DCGM_EXPORTER_LISTEN
          value: :9400
        - name: DCGM_EXPORTER_KUBERNETES
          value: "true"
        image: nvidia/dcgm-exporter:2.0.13-2.1.1-ubuntu18.04
        imagePullPolicy: IfNotPresent
        name: dcgm-exporter
        ports:
        - containerPort: 9400
          name: metrics
          protocol: TCP
        resources: {}
        securityContext:
          privileged: true
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/lib/kubelet/pod-resources
          name: pod-gpu-resources
          readOnly: true
        - mountPath: /usr/local/nvidia
          name: nvidia-install-dir-host
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
      tolerations:
      - effect: NoSchedule
        operator: Exists
      - key: nvidia.com/gpu
        operator: Exists
        effect: NoSchedule
      volumes:
      - hostPath:
          path: /var/lib/kubelet/pod-resources
          type: ""
        name: pod-gpu-resources
      - hostPath:
          path: /home/kubernetes/bin/nvidia
          type: ""
        name: nvidia-install-dir-host
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: dcgm-exporter
  namespace: monitoring
  labels:
    app.kubernetes.io/name: dcgm-exporter
    app.kubernetes.io/instance: dcgm-exporter
    app.kubernetes.io/version: "2.1.1"
    app.kubernetes.io/component: "dcgm-exporter"
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: dcgm-exporter
      app.kubernetes.io/instance: dcgm-exporter
      app.kubernetes.io/component: "dcgm-exporter"
  endpoints:
  - port: "metrics"
    path: "/metrics"
    interval: "15s"

Trying the suggested solution here of:

Adding securityContext.privileged=true
Adding nvidia-install-dir-host hostPath volume + volumeMount
We've seen that this resolved the issue for GKE using nvcr.io/nvidia/k8s/dcgm-exporter:2.0.13-2.1.1-ubuntu18.04, but with the more recent nvcr.io/nvidia/k8s/dcgm-exporter:2.1.4-2.3.1-ubuntu18.04 it's still broken.
We've opted to downgrade for now, of course

Tried downgrading dcgm-exporter from 2.2.9-2.4.1-ubuntu20.04 to 2.0.13-2.1.1-ubuntu18.04, and setting securityContext.privileged=true - keeps failing.
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  48s                default-scheduler  Successfully assigned monitoring-system/dcgm-exporter-8z8n5 to gke-np-epo-sentalign-euwe4a-gke--gpus-0fd942e9-zom5
  Normal   Pulling    48s                kubelet            Pulling image "nvcr.io/nvidia/k8s/dcgm-exporter:2.0.13-2.1.1-ubuntu18.04"
  Normal   Pulled     37s                kubelet            Successfully pulled image "nvcr.io/nvidia/k8s/dcgm-exporter:2.0.13-2.1.1-ubuntu18.04" in 10.688555255s
  Normal   Created    16s (x3 over 34s)  kubelet            Created container exporter
  Normal   Started    16s (x3 over 34s)  kubelet            Started container exporter
  Normal   Pulled     16s (x2 over 33s)  kubelet            Container image "nvcr.io/nvidia/k8s/dcgm-exporter:2.0.13-2.1.1-ubuntu18.04" already present on machine
  Warning  BackOff    12s (x5 over 32s)  kubelet            Back-off restarting failed container

❯ kubectl -n monitoring-system logs -p dcgm-exporter-8z8n5
time="2021-09-14T07:03:10Z" level=info msg="Starting dcgm-exporter"
Error: Failed to initialize NVML
time="2021-09-14T07:03:10Z" level=fatal msg="Error starting nv-hostengine: DCGM initialization error"

Also tried adding all volumes and volumeMounts from the nvidia-gpu-device-plugin DaemonSet, which is added by GKE - didn't fix the problem


Disable 'bounce' or 'elastic stretch' effect?

Hi guys
Is there a way to disable the bounce effect - ?
Cheers
Chris

adding return false will disable slide animation for all the cases, for just disabling the elastic effect at the ends check a condition for start and end and then do return false accordingly at line 322 where delta.x is defined write the following code before it
if(!index && delta.x > 0 || (index == slides.length - 1) && delta.x < 0){
    		return false;
}


Add Sharding Support

What type of pull request is this?

[ ] Non-breaking code change
[x] Breaking code change
[ ] Documentation change/addition

Description

This PR adds sharding support to the client.
In doing so, it moves gateway connection logic out of Snake and creates a new object called ConnectionState which houses data and methods related to the connection.
Changes

Create ConnectionState
Move gateway logic from Snake to ConnectionState
Refactor Snake properties and methods to point to ConnectionState
Create new properties in Snake that reference ConnectionState
Add attributes and logic to Gateway to allow for sharding
Add total_shards attribute to Snake
Add total_shards and shard_id arguments to Snake init

Checklist

[x] I've formatted my code with Black
[x] I've ensured my code works on Python 3.9.x
[x] I've tested my code

Well, that was fast.

I got home from work and started vibing to electro swing 🎶

Unrelated to the PR (only relevant because of sharding) but please don't add automatic sharding after this is in. Every attempt so far with getting that to work has been like walking on broken shards (no pun intended) of glass. There are numerous libraries that suffer from a proper implementation of shard automation and not even d.py's approach is all that good. 🙏🏼 keep it manual, most large bot devs will likely not automate it anyhow.

I've had 4 people tell me the same thing, I think I'll avoid auto-sharding


Unable to get it working in Electron. (This version of Node.js requires NODE_MODULE_VERSION 57.)

SerialPort Version: Latest

NodeJS Version: v9.4.0

Operating System and Hardware Platform: Raspbian GNU/Linux 9 (stretch) - Raspberry Pi B2

Electron Version: 1.8.2

Have you checked the right version of the api docs?: Yes.

Are you having trouble installing and you checked the Installation Special Cases docs? Yes.

Are you using Electron and have you checked the Electron Docs?: Yes.

Summary of Problem
I'm trying to use Serialport within a Electron application. Serialport describes the process in the documentation, but no matter what I try, Serialport keeps giving me the following error.

Error: The module '/home/pi/MagicMirror/modules/node_modules/serialport/build/Release/serialport.node'
was compiled against a different Node.js version using
NODE_MODULE_VERSION 48. This version of Node.js requires
NODE_MODULE_VERSION 57. Please try re-compiling or re-installing
the module (for instance, using npm rebuild or npm install).

When running electron-rebuild, I get ✔ Rebuild Complete. to be sure, I removed the build folder before running electron-rebuild, so it seems electron-rebuild is really doing it's work.
I've reinstalled all the node modules by removing the node_modules folder en rerunning npm install.
The last 12 hours have been nothing but googling, but so far I'm not able to solve the issue.
Any help is extremely appreciated!
Steps and Code to Reproduce the Issue

npm install
npm install electron-rebuild
npm install serialport
rm -Rf node_modules/serialport/build
electron-rebuild
npm run start

I finally managed to get it to work!
What I did was:

Remove node_modules folder from my project.
Remove any global electron packages (electron & electron-rebuild)
Remove any global Serialport packages
Reboot (CRITICAL STEP!)
Reinstall using the documentation of serial port.

So I think the reboot helped me.


Support fragments and includes

I am wondering if there is any way to support reusable elements in PlantUML diagrams as described at https://mrhaki.blogspot.com/2016/12/plantuml-pleasantness-include-partial.html?

Generating the image from the diagram is delegated to the server (see https://github.com/gmunguia/markdown-it-plantuml#advanced-usage). Customising how the image is generated is out of scope of this project.


Import of sccoral.tl

Report
sccoral.tl.XXX  returns error AttributeError: module 'sccoral' has no attribute 'tl'
sccoral.tl does not seem to be exported correctly to the namespace.
import sccoral.tl fixes the issue
Version information
No response

Solved with #20


(chore): update dependency: mocha

Fixes #<issue_number_goes_here> (it's a good idea to open an issue first for discussion)

[x] Tests and linter pass
[x] Code coverage does not decrease (if any source code was changed)
[x] Appropriate docs were updated (if necessary)

Codecov Report

Merging #3 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master     #3   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           1      1
  Lines           4      4
=====================================
  Hits            4      4

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9c4d823...5ba26e4. Read the comment docs.


Update my card (nouman-rahman.json)

Thank you for adding me to the content creator list. I appreciate your work and I am very glad to be on this list.
#WeAreDonet Creator Submission Guidelines

[x] Check all points before submitting the PR.

[x] By submitting this pull request to add my creator card, I verify that I adhere to the #WeAreDotnet creator card submission guidelines.

[x] Make sure to follow our Code of Conduct

[x] We do not accept submissions with the sole intention of promoting a product or company.

[x]  We add you as a creator, not your company.

[x]  The moderator team decides whether we accept or reject a submission.

When Do We Accept Someone's Creator Card Submission?

[x] You must have published at least three content pieces (long form content) since 2022 on any of your channels.

.NET itself

.NET libraries and projects

.NET languages C#, F#, Visual Basic

Tools needed during development (for .NET)

CI/CD

Only in combination with .NET: SPA Frontend technologies like Angular, Vue.js, React etc.

Cloud services related to .NET applications (AWS, Azure...)

[x] There must be three content pieces in English.

[x] There needs to be more than tweets.

[x] Longer-form article on LinkedIn is good, but a self-hosted blog or one on a well-known platform like Hashnode, medium etc., is better.

[x] Alternatively, you contributed with another form on a platform like YouTube, Twitch, Udemy or Pluralsight. If you have some recorded talks of conferences or anywhere else, that counts too! So in the form of a recorded video or a book.

RSS or Atom Feed

[x] The feed was validated using https://validator.w3.org/feed/ and does not contain errors

[x] The content with feed.type = dotnet must be .NET related:

.NET itself

.NET libraries and projects

.NET languages C#, F#, Visual Basic

Tools needed during development (for .NET)

CI/CD

Only in combination with .NET: SPA Frontend technologies like Angular, Vue.js, React etc.

Cloud services related to .NET applications (AWS, Azure...)

[x] the content under feed.type = personal may contain off-topic posts.

Project management

leadership

Other

[x] Submit an Atom or RSS URL inside your creator card that belongs to your blog

[x] If you do not submit an RSS or Atom URL. We cannot consider your feed for the aggregation feature.

Content Guideline

[x] If you have a self-hosted blog, please ensure you have a valid HTTPS certificate.

[x] We host NO malicious or offensive content on the blog (including photos, swearing, etc.)

[x] You must add the full URL to your GitHub profile to your card, so we can assign it to the member account that used the same GitHub account for sign-up.

Thanks for your submission. Could you use the new card format? https://github.com/WeAreDotnet/dotnet-creator-cards/blob/main/creator-cards/_creatorTemplate.json
Thanks!

Sorry I was late. but this is the one with new format: Update (nouman-rahman.json)

Could you go to the new format? It is still the old one.
Please use this JSON: https://github.com/WeAreDotnet/dotnet-creator-cards/blob/main/creator-cards/_creatorTemplate.json


Start/stop measurements before and after every type of job

The current execution flow is divided on the following stages:
start measurements -> job -> stop measurements -> calculate elapsedTime -> index jobSummary ->  evaluate alerts -> indexMetrics
meaning that elapsedTime was taking into account the "stop measurements" stage
This change proposed to stop Measurements after elapsedTime is calculated, so we won't take into account that time when indexing the job summary document
Fixes: https://github.com/cloud-bulldozer/kube-burner/issues/312

Maybe it's an error that was masked previously, but there is a measurement error now being logged due to being over the threshold. If I increase the threshold, the error goes away as expected.
master
time="2023-05-24 16:19:32" level=info msg="Stopping measurement: podLatency" file="factory.go:109"
time="2023-05-24 16:19:32" level=info msg="Evaluating latency thresholds" file="metrics.go:57"
time="2023-05-24 16:19:32" level=error msg="❗ P99 Ready latency (11.63s) higher than configured threshold: 5s" file="metrics.go:67"
time="2023-05-24 16:19:32" level=info msg="node-density: ContainersReady 50th: 9737 99th: 11632 max: 11632 avg: 9507" file="pod_latency.go:170"
time="2023-05-24 16:19:32" level=info msg="node-density: Ready 50th: 9737 99th: 11632 max: 11632 avg: 9507" file="pod_latency.go:170"
time="2023-05-24 16:19:32" level=info msg="Job node-density took 13s" file="job.go:163"

pr/319
time="2023-05-24 16:27:13" level=info msg="Stopping measurement: podLatency" file="factory.go:111"
time="2023-05-24 16:27:13" level=info msg="Evaluating latency thresholds" file="metrics.go:57"
time="2023-05-24 16:27:13" level=error msg="❗ P99 Ready latency (11.13s) higher than configured threshold: 5s" file="metrics.go:67"
time="2023-05-24 16:27:13" level=info msg="node-density: Ready 50th: 9876 99th: 11126 max: 11126 avg: 9574" file="pod_latency.go:170"
time="2023-05-24 16:27:13" level=error msg="Failed measurements: %v[podLatency]" file="job.go:165"

Maybe it's an error that was masked previously, but there is a measurement error now being logged due to being over the threshold. If I increase the threshold, the error goes away as expected.
master
time="2023-05-24 16:19:32" level=info msg="Stopping measurement: podLatency" file="factory.go:109"
time="2023-05-24 16:19:32" level=info msg="Evaluating latency thresholds" file="metrics.go:57"
time="2023-05-24 16:19:32" level=error msg="❗ P99 Ready latency (11.63s) higher than configured threshold: 5s" file="metrics.go:67"
time="2023-05-24 16:19:32" level=info msg="node-density: ContainersReady 50th: 9737 99th: 11632 max: 11632 avg: 9507" file="pod_latency.go:170"
time="2023-05-24 16:19:32" level=info msg="node-density: Ready 50th: 9737 99th: 11632 max: 11632 avg: 9507" file="pod_latency.go:170"
time="2023-05-24 16:19:32" level=info msg="Job node-density took 13s" file="job.go:163"

pr/319
time="2023-05-24 16:27:13" level=info msg="Stopping measurement: podLatency" file="factory.go:111"
time="2023-05-24 16:27:13" level=info msg="Evaluating latency thresholds" file="metrics.go:57"
time="2023-05-24 16:27:13" level=error msg="❗ P99 Ready latency (11.13s) higher than configured threshold: 5s" file="metrics.go:67"
time="2023-05-24 16:27:13" level=info msg="node-density: Ready 50th: 9876 99th: 11126 max: 11126 avg: 9574" file="pod_latency.go:170"
time="2023-05-24 16:27:13" level=error msg="Failed measurements: %v[podLatency]" file="job.go:165"

this is correct, this behavior is similar as it was before, btw, I fixed the typo by replacing Error by Errorf

looks good to me too.


ValueError: invalid literal for int() with base 10: '2.73\xa0GiB'

Code:
from tpblite import TPB

pirate_bay = TPB("https://thepiratebay.party/")

torrents = pirate_bay.search("matrix")

Error:
Traceback (most recent call last):
  File "C:\Users\mahas\PycharmProjects\DownloadArr\main.py", line 5, in <module>
    torrents = pirate_bay.search("matrix")
  File "C:\Users\mahas\PycharmProjects\DownloadArr\venv\lib\site-packages\tpblite\tpblite.py", line 43, in search
    return Torrents(q.html_source)
  File "C:\Users\mahas\PycharmProjects\DownloadArr\venv\lib\site-packages\tpblite\models\torrents.py", line 104, in __init__
    self.list = self._createTorrentList()
  File "C:\Users\mahas\PycharmProjects\DownloadArr\venv\lib\site-packages\tpblite\models\torrents.py", line 128, in _createTorrentList
    torrents.append(Torrent(row))
  File "C:\Users\mahas\PycharmProjects\DownloadArr\venv\lib\site-packages\tpblite\models\torrents.py", line 34, in __init__
    self.seeds, self.leeches = self._getPeers()
  File "C:\Users\mahas\PycharmProjects\DownloadArr\venv\lib\site-packages\tpblite\models\torrents.py", line 60, in _getPeers
    return int(taglist[0]), int(taglist[1])
ValueError: invalid literal for int() with base 10: '2.73\xa0GiB'

Lxml is already installed.
PS C:\Users\mahas\PycharmProjects\DownloadArr> pip install lxml
Requirement already satisfied: lxml in c:\users\mahas\pycharmprojects\downloadarr\venv\lib\site-packages (4.7.1)
PS C:\Users\mahas\PycharmProjects\DownloadArr>

PS: The link works in my country.

This is because the domain https://thepiratebay.party has a different html structure than the domain that this tool was built for. For example if you go on https://tpb.party and search for something, you will see that it is visually a little different. Since this tool parses the underlying html, it won't work for sites that differ. I would suggest just to use the default domain, i.e. https://tpb.party.


Update PPP

Codecov Report

Merging #263 (cc41279) into master (3346b83) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head cc41279 differs from pull request most recent head 8906373. Consider uploading reports for the commit 8906373 to get more accurate results

@@            Coverage Diff            @@
##             master     #263   +/-   ##
=========================================
  Coverage     75.78%   75.78%
  Complexity      823      823
=========================================
  Files           121      121
  Lines          2288     2288
  Branches        253      253
=========================================
  Hits           1734     1734
  Misses          502      502
  Partials         52       52

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3346b83...8906373. Read the comment docs.


Vet clinic database: add "join table" for visits

[ ] Prepare complex queries that answer analytical questions.
[ ] Use primary key & foreign key mechanisms for joining tables.

Who was the last animal seen by William Tatcher?

How many different animals did Stephanie Mendez see?

List all vets and their specialties, including vets with no specialties.

List all animals that visited Stephanie Mendez between April 1st and August 30th, 2020.

What animal has the most visits to vets?

Who was Maisy Smith's first visit?

Details for most recent visit: animal information, vet information, and date of visit.

How many visits were with a vet that did not specialize in that animal's species?

What specialty should Maisy Smith consider getting? Look for the species she gets the most.

What specialty should Maisy Smith consider getting? Look for the species she gets the most.


Generalize handling of GotoDefinitionResponse and add lsp-type-definition

There were a bunch of places that all handled GotoDefinitionResponse in their own way, and this meant, for example, :lsp-definition couldn't go to multiple impl blocks for Rust. This PR merges them into one implementation, and also adds lsp-type-definition.

It worth updating README as well to reflect the fact that buffer is called *goto* now for all those commands.


Does not work on latest RHEL AMIs. No easy way to specify release channel for binary

Currently running this on any up-to-date RHEL AMI will fail. This is due to this upstream issue, which I hope will be resolved soon:
https://github.com/rancher/rke2-selinux/issues/36
While I am aware that the latest patch that can fix this is in the testing channel, it's a little inconvenient that this terraform module has no official way to specify the release channel into the cloud-init userdata (without manually modding the script). Please correct me if I'm wrong and there's some hack that can be done using the extra_cloud_config_config variable.
As a feature request, it would be nice to be able to pass in the environment variables, such as "INSTALL_RKE2_CHANNEL" and have them be read into the relevant parts of the userdata scripts.
If you have any guidance for alternate workarounds, such as copying the binary file directly to the node, please let me know.

Fixed by #102


Make the info template on the chapters overview page more flexible

The template currently expects exactly three paragraphs. If those are not defined, the translation keys are rendered. As follows:

How can we make this template dynamic? Chapter 1 currently only needs 1 paragraph, other chapters also may need different amounts based (that copy is still TBD).

Ben does that mean we should close this issue?


fix(autocmd): add SafeState to the list of autocommand

The ci is failing because of this on master.

SafeStateAgain is only in vim should it still be in the autocmd list?

SafeStateAgain is only in vim should it still be in the autocmd list?

Can't hurt. People might want to work on Vim patches.


Please invite me to the GitHub Community Organization

Name
Arjun Hegde
Discord Username (if applicable)
No response
Additional Context
No response

Welcome to ChryzHub @izzarzn 🥳


Update GitHub Action Versions

GitHub Actions Version Updates

actions/delete-package-versions published a new release v5.0.0 on 2024-01-16T13:50:21Z
docker/build-push-action published a new release v5.1.0 on 2023-11-17T12:46:31Z
actions/cache published a new release v4.0.0 on 2024-01-16T22:17:55Z

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Alten - Instead of the city the message "Cluj Napoca is not a city in Romania" is written

ENVIRONMENT: production
URL:  https://scrapers.peviitor.ro/src/alten/index.html
Browser: Chrome
Device: Laptop
OS: Windows 11
STEPS TO REPRODUCE:

Open the URL in the browser
Click on the "Run Scraper" button
Check the job location/city for job "Mainframe Production Engineer"
Open https://careers.altenromania.ro/ and click the "See More" button 6 times
Check the job location for job "Mainframe Production Engineer" from the Alten website
EXPECTED RESULTS:
The city of the job is written in accordance with the city on the company website

ACTUAL RESULTS:
Instead of the city, the message "Cluj Napoca is not a city in Romania" is written

Fixed!

Verificat!


[Refactor]: Remove code-editor support

I'm submitting a...

[x] Bug Fix
[ ] Feature
[x] Other (Refactoring, Added tests, Documentation, ...)

Does this PR introduce a breaking change?

[x] Yes
[ ] No

Does this PR affects any existing issues?

[x] Yes
[ ] No

closes #83, closes #94, fix #96, closes #101

@isaac2004 .
Not now. currently code-mirror makes a lot of problems while compilation and also felt unnecessary.
Do you have any justification why do we need one?

@Sibiraj-S I would like to use the editor as part of a blog. Basically use the editor and store the html in some datasource. I write technical blog posts, so at times I want to copy/paste the code from some editor and have it formatted in a eye friendly way. Having an option to paste formatted source code and it formatted correctly on the html side would be very valuable.

such as?


The behavior of one node multi-gpus with webdataset

Hello, this issue just want to clarify the concept of the behavior of one node multi-gpus with webdataset (in torchlightning + DDP strategy). According to several test i run, I can not let 2 gpus take the shard data from the "identical dataset".
Suppose I have 200 shard data, how to setup webdataset or torch.utils.DataLoader, so that 2 gpus share with 200 shard data (per gpus only take 100 shard for each epoch ?) ?
( In torchlightning, we let Lightningmodule handle the  DDP strategy. So we're also not sure webdataset can realize the num_gpus and correct its behavior.

I'm not sure what you're asking.
For splitting, you have two options: webdataset and wids.
webdataset
You can specify any nodesplitter you like to split shards among nodes. You can do this as an argument to the WebDataset class, or you can do it explicitly, since WebDataset is just a wrapper for DataPipeline:
    dataset = wds.DataPipeline(
        wds.SimpleShardList(list(map(str, range(10)))),
        wds.split_by_node,
        wds.split_by_worker,
        wds.non_empty,
	    ...
    )
```

# wids

In the `wids` library, you don't have explicit access to shards; everything is specified by index. If you want to ensure that each GPU only sees a subset of the shards, you need to arrange for the sampler you use to do that. Look at `DistributedChunkedSampler` on how to use rank and worker arguments to choose a subset of the indexes.

I'm not sure what you're asking.
For splitting, you have two options: webdataset and wids.
webdataset
You can specify any nodesplitter you like to split shards among nodes. You can do this as an argument to the WebDataset class, or you can do it explicitly, since WebDataset is just a wrapper for DataPipeline:
    dataset = wds.DataPipeline(
        wds.SimpleShardList(list(map(str, range(10)))),
        wds.split_by_node,
        wds.split_by_worker,
        wds.non_empty,
	    ...
    )

wids
In the wids library, you don't have explicit access to shards; everything is specified by index. If you want to ensure that each GPU only sees a subset of the shards, you need to arrange for the sampler you use to do that. Look at DistributedChunkedSampler on how to use rank and worker arguments to choose a subset of the indexes.
I haven't tried wids with Lightning, but you should be able to use it directly, since it is just a regular indexed dataset.

webdataset solution looks good, but I have the following form :
pair_ds = wds.WebDataset(str(shard_path), resampled=True).decode('pilrgb', handler=wds.warn_and_continue).to_tuple("jpg", "txt", handler=wds.warn_and_continue)

Is there a way to add wds.split_by_node, wds.split_by_worker in pair_ds ?

I'm not sure what you're asking.
For splitting, you have two options: webdataset and wids.
webdataset
You can specify any nodesplitter you like to split shards among nodes. You can do this as an argument to the WebDataset class, or you can do it explicitly, since WebDataset is just a wrapper for DataPipeline:
    dataset = wds.DataPipeline(
        wds.SimpleShardList(list(map(str, range(10)))),
        wds.split_by_node,
        wds.split_by_worker,
        wds.non_empty,
	    ...
    )

wids
In the wids library, you don't have explicit access to shards; everything is specified by index. If you want to ensure that each GPU only sees a subset of the shards, you need to arrange for the sampler you use to do that. Look at DistributedChunkedSampler on how to use rank and worker arguments to choose a subset of the indexes.
I haven't tried wids with Lightning, but you should be able to use it directly, since it is just a regular indexed dataset.

Yes, You can specify any nodesplitter you like to split shards among nodes. is what I'm trying to do. But, it got stuck with training in torchlightning.
I have tried 2 kind of method :
(1) official website pipeline API
dataset = wds.DataPipeline(
    wds.SimpleShardList(urls),

    # at this point we have an iterator over all the shards
    wds.shuffle(100),

    # add wds.split_by_node here if you are using multiple nodes
    wds.split_by_node,
    wds.split_by_worker,

    # at this point, we have an iterator over the shards assigned to each worker
    wds.tarfile_to_samples(),

    # this shuffles the samples in memory
    wds.shuffle(1000),

    # this decodes the images and json
    wds.decode("pil"),
    wds.to_tuple("png", "json"),
    wds.map(preprocess),
    wds.shuffle(1000),
    wds.batched(16)
)

(2) There're a japanes ninja give a clear example similar with your method, but it still got stuck..
class Liaon115M(pl.LightningDataModule):
    ...
    def setup(...):
        def wrap_wds(shard_path, ratio):
            trfs = lambda tup : (self.img_transforms(tup[0]), self.txt_transforms(tup[1]))
            raw_web_ds = wds.WebDataset(str(shard_path), nodesplitter=wds.split_by_node)
            # I have set `handler=wds.warn_and_continue` in decode & to_tuple, but it's not stuck in here...
            pair_ds = raw_web_ds.decode('pilrgb').to_tuple("jpg", "txt")
            return pair_ds.batched(self.batch_size, partial=False)
        self.train_ds = wrap_ds(...)

    def train_dataloader(self):
        sample_loader = wds.WebLoader(
            self.train_ds,
            batch_size=None,
            shuffle=False,
            pin_memory=True,
            num_workers=self.num_workers
        )
        dataset_size = self.total_num_ds * self.split_ratio[0]
        num_batches = int( dataset_size // (self.batch_size * self.n_gpu) )

        sample_loader.length = num_batches
        sample_loader = sample_loader.with_length(num_batches)

        sample_loader = sample_loader.repeat(nbatches=num_batches)
        sample_loader = sample_loader.slice(num_batches)

        print(f"train steps : {num_batches}")
        return sample_loader

unbalanced GPU utilization..


Filter tasks that include a #todo tag and were created before a certain date

Hi,
I have a several pages tagged with #RA and inside are a bunch of tasks (todos) that are tagged with #todo and I am trying to get a task list of all the tasks with the #todo from #RA pages but I also only want the tasks from #RA pages that were created after a certain date.
How do I go about having 2 where clauses.
This is what I have so far
dv.taskList(dv.pages("#RA").file.tasks .where(t => t.text.includes("#todo")))
This is only able to get me a task list of all the tasks with the #todo in #RA pages but I am unable to factor in the date that the #RA page was created
Any help is greatly appreciated!
Thank you!

If all you care about is the date that the page was created, you can do a double where:
dv.taskList(dv.pages("#RA").where(p => p.file.ctime < dv.date("2020-08-10")).file.tasks.where(t => t.text.includes("#todo")))

Hi blacksmithgu,
Firstly, thank you for such a prompt response!
I am getting an error where dv.date is not a function.
Error for using dv.date

I read another post where you mentioned a workaround was dv.call('data', ) but then I got an error that dv.call is not a function.
dv.taskList(dv.pages("#RA")
    .where(p => p.file.ctime < dv.call('date',"2020-08-10"))
    .file.tasks
    .where(t => t.text.includes("#todo")))

Error for using dv.call

Am I missing something here?
Thanks in advance!
Really loving this plugin!

Oh whoops, I jumped the gun... dv.date() is coming in the next release tommorow...

I see! Looking forward to the release! Thanks so much for your help!

The new release is out - see if the query works now.


Add support for using gateway.spec.addresses as service external ips

since service.externalIP only support IP addresses, in this implementation, only the gateway.spec.addresses with type IPAddress has been used.
refer to #360

I've run some simple test in local.

 apply config altered from quickstart.yaml
apiVersion: gateway.networking.k8s.io/v1beta1
kind: GatewayClass
metadata:
name: eg
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
name: eg
spec:
gatewayClassName: eg
listeners:
  - name: http
    protocol: HTTP
    port: 80
addresses:
  - type: IPAddress
     value: <IP_ADDRESS>
  - type: IPAddress
    value: <IP_ADDRESS>   # repeat is okay, will only store once
  - type: IPAddress
    value: <IP_ADDRESS>
  - type: Hostname   # do not support, so will skip
     value: foo.bar
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: backend
---
apiVersion: v1
kind: Service
metadata:
name: backend
labels:
  app: backend
  service: backend
spec:
ports:
  - name: http
    port: 3000
    targetPort: 3000
selector:
  app: backend
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
spec:
replicas: 1
selector:
  matchLabels:
    app: backend
    version: v1
template:
  metadata:
    labels:
      app: backend
      version: v1
  spec:
    serviceAccountName: backend
    containers:
      - image: gcr.io/k8s-staging-ingressconformance/echoserver:v20221109-7ee2f3e
        imagePullPolicy: IfNotPresent
        name: backend
        ports:
          - containerPort: 3000
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
name: backend
spec:
parentRefs:
  - name: eg
hostnames:
  - "www.example.com"
rules:
  - backendRefs:
      - group: ""
        kind: Service
        name: backend
        port: 3000
        weight: 1
    matches:
      - path:
          type: PathPrefix
          value: /

checkout backend service
NAME                           READY   STATUS    RESTARTS   AGE
pod/backend-7f6b74f5b7-78xp4   1/1     Running   0          113m

NAME                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE
service/backend      ClusterIP   <IP_ADDRESS>   <none>        3000/TCP   113m
service/kubernetes   ClusterIP   <IP_ADDRESS>     <none>        443/TCP    162m

NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/backend   1/1     1            1           113m

NAME                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/backend-7f6b74f5b7   1         1         1       113m

checkout gateway-class and gateway
NAME   CONTROLLER                                      ACCEPTED   AGE
eg     gateway.envoyproxy.io/gatewayclass-controller   True       114m

NAME   CLASS   ADDRESS   PROGRAMMED   AGE
eg     eg      <IP_ADDRESS>   True         115m     # it has two addresses, but only one is showing

$kubectl describe gtw eg
Name:         eg
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  gateway.networking.k8s.io/v1beta1
Kind:         Gateway
Metadata:
  Creation Timestamp:  2023-04-17T13:41:50Z
  Generation:          2
  Managed Fields:
    API Version:  gateway.networking.k8s.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:addresses:
        f:conditions:
          k:{"type":"Accepted"}:
            f:lastTransitionTime:
            f:message:
            f:observedGeneration:
            f:reason:
            f:status:
          k:{"type":"Programmed"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:observedGeneration:
            f:reason:
            f:status:
            f:type:
        f:listeners:
          .:
          k:{"name":"http"}:
            .:
            f:attachedRoutes:
            f:conditions:
              .:
              k:{"type":"Accepted"}:
                .:
                f:lastTransitionTime:
                f:message:
                f:observedGeneration:
                f:reason:
                f:status:
                f:type:
              k:{"type":"Programmed"}:
                .:
                f:lastTransitionTime:
                f:message:
                f:observedGeneration:
                f:reason:
                f:status:
                f:type:
            f:name:
            f:supportedKinds:
    Manager:      envoy-gateway
    Operation:    Update
    Subresource:  status
    Time:         2023-04-17T13:49:42Z
    API Version:  gateway.networking.k8s.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:addresses:
        f:gatewayClassName:
        f:listeners:
          .:
          k:{"name":"http"}:
            .:
            f:allowedRoutes:
              .:
              f:namespaces:
                .:
                f:from:
            f:name:
            f:port:
            f:protocol:
    Manager:         kubectl-client-side-apply
    Operation:       Update
    Time:            2023-04-17T13:49:42Z
  Resource Version:  4881
  UID:               c119f71e-b445-4e2c-8a27-cdce7e969b8e
Spec:
  Addresses:
    Type:              IPAddress
    Value:             <IP_ADDRESS>
    Type:              IPAddress
    Value:             <IP_ADDRESS>
    Type:              IPAddress
    Value:             <IP_ADDRESS>
    Type:              Hostname
    Value:             foo.bar
  Gateway Class Name:  eg
  Listeners:
    Allowed Routes:
      Namespaces:
        From:  Same
    Name:      http
    Port:      80
    Protocol:  HTTP
Status:
  Addresses:
    Type:   IPAddress
    Value:  <IP_ADDRESS>
    Type:   IPAddress
    Value:  <IP_ADDRESS>
  Conditions:
    Last Transition Time:  2023-04-17T13:49:42Z
    Message:               The Gateway has been scheduled by Envoy Gateway
    Observed Generation:   2
    Reason:                Accepted
    Status:                True
    Type:                  Accepted
    Last Transition Time:  2023-04-17T13:49:42Z
    Message:               Address assigned to the Gateway, 1/1 envoy Deployment replicas available
    Observed Generation:   2
    Reason:                Programmed
    Status:                True
    Type:                  Programmed
  Listeners:
    Attached Routes:  1
    Conditions:
      Last Transition Time:  2023-04-17T13:49:42Z
      Message:               Sending translated listener configuration to the data plane
      Observed Generation:   2
      Reason:                Programmed
      Status:                True
      Type:                  Programmed
      Last Transition Time:  2023-04-17T13:49:42Z
      Message:               Listener has been successfully translated
      Observed Generation:   2
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
    Name:                    http
    Supported Kinds:
      Group:  gateway.networking.k8s.io
      Kind:   HTTPRoute
      Group:  gateway.networking.k8s.io
      Kind:   GRPCRoute
Events:       <none>

checkout service in namespace: envoy-gatewat-system
kubectl get svc -n envoy-gateway-system
NAME                            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)        AGE
envoy-default-eg-64656661       LoadBalancer   <IP_ADDRESS>    <IP_ADDRESS>,<IP_ADDRESS>   80:32146/TCP   118m
envoy-gateway                   ClusterIP      <IP_ADDRESS>   <none>            18000/TCP      122m
envoy-gateway-metrics-service   ClusterIP      <IP_ADDRESS>   <none>            8443/TCP       122m

kubectl describe -n envoy-gateway-system svc envoy-default-eg-64656661
Name:                     envoy-default-eg-64656661
Namespace:                envoy-gateway-system
Labels:                   app.gateway.envoyproxy.io/name=envoy
                          gateway.envoyproxy.io/owning-gateway-name=eg
                          gateway.envoyproxy.io/owning-gateway-namespace=default
Annotations:              <none>
Selector:                 app.gateway.envoyproxy.io/name=envoy,gateway.envoyproxy.io/owning-gateway-name=eg,gateway.envoyproxy.io/owning-gateway-namespace=default
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       <IP_ADDRESS>
IPs:                      <IP_ADDRESS>
External IPs:             <IP_ADDRESS>,<IP_ADDRESS>
Port:                     http  80/TCP
TargetPort:               10080/TCP
NodePort:                 http  32146/TCP
Endpoints:                <IP_ADDRESS>:10080
Session Affinity:         None
External Traffic Policy:  Local
HealthCheck NodePort:     30880
Events:                   <none>

thanks for this PR @shawnh2 , the code looks good, added some minor comments
also hoping this PR lands after https://github.com/envoyproxy/gateway/pull/1259 which is making changes in a similar area

Can you resolve failues on CI ? @shawnh2

The CI has been fixed, @arkodg @Xunzhuo PTAL

Can you add a doc for this ? @shawnh2
And also need to resolve some conflicts, thanks.
Others LGTM

Done

Ops ... Conflicts again : (
Mind resolve it one more time ? Then I can merge it : ) @shawnh2

Ops ... Conflicts again : (
Mind resolve it one more time ? Then I can merge it : ) @shawnh2

Done


[Problem]: Images newer than v7.1.2 are broken on ghcr.io - Docker hub is working!

Build is still not working with ghcr.io. But it's working fine with Docker Hub. So the workaround is pulling the image from there.
Seems the image is pushed successfully, but there is some issue with the image manifest.
It's more or less a bug of buildx or build-and-push action. I have no more ideas and not enough time to work into this at the moment. Have subscribed some related issues. Will come back to this when there are some news coming up.
Any help appreciated.

I recently changed the title of this issue for documentation purpose.
Investigation result: Looks like the manifest generated for multiarch images by "build and push" action (with buildx) is not supported by Github container repo (ghcr.io). As I don't want to change the build process to build the images in separate steps (takes a lot longer!) this will remain broken until build and push and/or buildx fixes this.
@viceice provenance is now disabled, but did not fix this.
WORKAROUND: Pull Image from Docker Hub.
Regards,
André

that's strange, i use buildx for multi arch too and those works fine on ghcr.
so it must be something with the action.
🤔
https://github.com/containerbase/base

Tell me what's wrong. :)
https://github.com/buanet/ioBroker.docker/blob/main/.github/workflows/build-debian-image-dev.yml
This is where I build the dev tag. It's perfect for testing.
Maybe you can show me your action, so I can compare. If it is private, you can give it to me in private or simply try fix it an make a pr.

[EDIT] Sorry, maybe I got you wrong. You mean it might be a problem of build and push Action, right?

yes, we use docker buildx ... directly, which is working fine for the linked repo

I would smell that it is rather related to https://github.com/buanet/ioBroker.docker/blob/main/.github/workflows/build-debian12-latest.yml#L152 - removing untagged manifests.
Maybe, right after the build, the ref count is yet not update as fast as expected, counting the just pushed manifests as untagged and thus removing them alltogether.

Investigation result: Looks like the manifest generated for multiarch images by "build and push" action (with buildx) is not supported by Github container repo (ghcr.io).

Is there any source to that? I had / have no such issues in https://github.com/EugenMayer/docker-image-unison

I would smell that it is rather related to https://github.com/buanet/ioBroker.docker/blob/main/.github/workflows/build-debian12-latest.yml#L152 - removing untagged manifests.
Maybe, right after the build, the ref count is yet not update as fast as expected, counting the just pushed manifests as untagged and thus removing them alltogether.

Good point. I'll give it a try.

@EugenMayer Thanks for your hint! My test looks good. Deleting the untagged images was never a problem as I pushed a separate manifest before changing build process to multiarch build with build and push action...  I'll fix that with next release.

With latest release pulling from ghcr.io is now working.


Where am I supposed to find .jar file to execute or precompiled binaries to download? Readme mentions them but I'm unable to find any links.

Hi :) First, I'd like to thank you for making this great software.
However, I'm totally unclear as to where I'm supposed to obtain the version that is ready to run.
The readme says:

Zoom Firmware Editor application is a self-executable jar-file. If you have Java JRE installed, then double-click on "ZoomFirmwareEditor.jar" should launch it.
If you dont have Java installed on your computer, you can download and run "ZoomFirmwareEditor-setup.exe". That setup application will install the compiled Windows executable with embedded JVM (created using Excelsior JET).

As you can see, these aren't links and these files are nowhere to be found. I'm totally puzzled.
Thanks.

It's under Releases. (facepalm)
I'm sorry, I'm not an experienced GitHub user, maybe you should make it more obvious for those who are not "in the know". Leaving the issue open so you can notice this.


Limitlessled leaves user in the dark after HA restart.

Home Assistant release (hass --version): 0.59.2
Python release (python3 --version): 3.5
Component/platform: light/limitlessled
Description of problem:
If you restart HA, it switches all limitlessled lights off and does not restore the previous state (or leave them untouched). PR #10587 tried to fix this and works perfectly but the change was rejected. So the problem remains.
Expected:
The lights should either keep the state or get configured to the previous state.

I think the platform should leave the device alone on startup. If some user wants to turn all lights off during restart, writing an automation for that is simple enough.
Where is the code that turns lights off, I was unable to find it? (I do not have these lights, I am just curious)

It is exactly here: https://github.com/home-assistant/home-assistant/blob/1f1115f631cb3fae99f1c68a3812e2ac79814e92/homeassistant/components/light/limitlessled.py#L262
I think the design by now is correct, because there are two possibilities:
a) State restauration (as a generic light feature) --> not implemented
b) Sync the state at start-up
The other possible solution would be to have wrong states in HASS.

Thanks! I was searching for "off" :-)
If these bulbs cannot report their state, I guess Home Assistant will also be wrong if the lights are changed with e.g. a smartphone app?
There is also:
c) Use  assumed_state and avoid pretending to have the right state

'assumed_state' is more for switches like command_line switches where you don't have real state. Then whatever you do state is always on or always off.
It's like:

From the documentation:
For switches with an assumed state two buttons are shown (turn off, turn on) instead of a switch. By setting assumed_state to false you will get the default switch icon.

That would eliminate the use case.
Yes, Home Assistant will not recognize the change, but I see no other possibility for now. Why we should change the behaviour? Everytime you restart HASS you will get a possible wrong state.
The solution would be to implement a state restoration, which is I guess a major change.

The pr used the state restore infrastructure. If you want, you can just use the patch. I use the esp-milight-hub now and configure the devices via mqtt. So I have no need to provide a solution for this bug anymore.
One idea might be to just set the internal state and not configure the lamps at all on init.

One idea might be to just set the internal state and not configure the lamps at all on init.

@happyleavesaoc suggested the same thing in #10587 and I think this is the way to do it (along with your assumed_state() -> True addition).
Your PR was rejected because it tried to fix something wrong by adding more wrong things. Removing the initial wrong is much better :-)
Again, if somebody wants the current behavior of forcing a specific external device state on startup, he can do so with a HOMEASSISTANT_START automation.

I agree with you, but then we need to identify all 1-way devices, change that behaviour and classify it as a breaking change.

Agreed that it's a breaking change but this is not a global problem – only limitlessled is acting in this way during startup.

do you know other 1-way components?

The Sense Hat light is one.

So how do I apply the patch to fix this issue? Im having the same problem..

@badstraw360 The easiest way is to copy this file from the PR to your config/custom_components/light directory and then restart Home Assistant. That should override the default limitlessled platform and will opt you out of future updates until the custom component is deleted again.
@tringler Since @janLo appears to have lost the motivation, are you up for this challenge? It seems fairly simple to remove the initialization (like you already did above with comments) but I am unable to test it, so I will not make an attempt.

@amelchio - Yes, I can do the job. Let me test and review your code this evening, so I can create a PR.

@amelchio Please check https://github.com/home-assistant/home-assistant/pull/11204/commits/35790f7ebbc53e0c12c21f15747a31ed9222d82c

If you remove all the initial state setting from the device you need to set the old state from the history. Otherwise you will most definite end up having a wrong state.
Also consider to add a service to "resync" the devices or add a paragraph in the documentation how the user can achieve this.


SVG vector Logo

Which @ngneat/elf-* package(s) are relevant/releated to the feature request?
No response
Description
Hi. Do You have SVG logo? For shirts, websites, etc.
Proposed solution
If You have other vector format, I can convert to svg.
Alternatives considered

Do you want to create a pull request?
No

No. I took the logo from https://www.flaticon.com/


Frontpage

Simple approach of adding a frontpage defined as markdown file.
Works for my use case.

@MaxMelcher Any reason why this is still open?
Something I need to change/adjust?

Sorry for the delay - busy time for me, had no chance to look at it just yet.
Get Outlook for iOShttps://aka.ms/o0ukef

From: cpetry @.>
Sent: Thursday, April 28, 2022 11:43:07 AM
To: MaxMelcher/AzureDevOps.WikiPDFExport @.>
Cc: Max Melcher @.>; Mention @.>
Subject: Re: [MaxMelcher/AzureDevOps.WikiPDFExport] Frontpage (PR #100)
@MaxMelcherhttps://github.com/MaxMelcher Any reason why this is still open?
Something I need to change/adjust?
—
Reply to this email directly, view it on GitHubhttps://github.com/MaxMelcher/AzureDevOps.WikiPDFExport/pull/100#issuecomment-1111984616, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAH3LN6NAQIOYOFQKM5JU6DVHJMSXANCNFSM5PWPFZOQ.
You are receiving this because you were mentioned.Message ID: @.***>

I cant merge this because of the significant refactorings in #105 - i will try to merge it in soon.


Ionic Packaging services fails to sign ios apps

More recently, we get errors during the last code signing step of the Packaging process.
It is either:
Signing Identity:     ""
Provisioning Profile: ""
()
/usr/bin/codesign --force --sign <thumbnail shown> --entitlements /Users/package/workspace/<appid>/cordova/platforms/ios/build/<app name>.build/Release-iphoneos/<app name>.build/<app name>.app.xcent --timestamp=none /Users/package/workspace/<appid>/cordova/platforms/ios/build/device/<app name>.app

: no identity found
Command /usr/bin/codesign failed with exit code 1
** BUILD FAILED **
Or:
=== BUILD TARGET  OF PROJECT  WITH CONFIGURATION Release ===
Check dependencies
Code Sign error: No code signing identities found: No valid signing identities (i.e. certificate and private key pair) were found.
** BUILD FAILED **
In between, the security profiles have not changed, but seem to be unavailable or corrupted somehow for the signing process.
Resetting the Security Profiles with same certificates and provisioning profiles as before solves the problem, but of course we cannot start to do this every day or on every build.

I'm having the same issue. Additionally, ionic package info crashes on builds that have failed in this manner.
Failed build of this type:
czapka:/tmp/ionic-app-deploy-70001/crm-app$ ionic package info 42
An uncaught exception occurred and has been reported to Ionic

Unexpected end of input (CLI v1.7.4)

Your system information:

Cordova CLI: 6.3.1
Gulp version:  CLI version 3.9.1
Gulp local:   Local version 3.9.1
Ionic Version: 1.2.4-nightly-1917
Ionic CLI Version: 1.7.4
Ionic App Lib Version: 0.6.0
ios-deploy version: 1.9.0
ios-sim version: 5.0.8
OS: Mac OS X El Capitan
Node Version: v4.5.0
Xcode version: Xcode 8.0 Build version 8A218a

Build that failed for a different reason:
czapka:/tmp/ionic-app-deploy-70001/crm-app$ ionic package info 1

  id       │ 1
  status   │ FAILED

Successful build for the same app:

  id        │ 41
  status    │ SUCCESS

I can send the app id in private if it helps.


Enhance the script for version above 20.6

Jira ticket
https://jira-eng-sjc4.cisco.com/jira/browse/SDWANAURA-49

Assigned to Harshal Khairnar, waiting for him to create a git account


Raise a helpful error if users try to use the cloud resolver when the server doesn’t support it

There are multiple ways to deploy the Sematic server: one allows for cloud execution while the other does not. If a user tries to use the CloudResolver when the server can’t support it, we should give a message that indicates why it isn’t working and also points to the deployment docs.

From where it will run the cloud server

From where it runs cloud resolver


GenericTabularInline doesn't work properly

Styles support does not work when adding GenericTabularInline class to inlines variable of Admin class. When adding other TabularInline classes they work natively, it is GenericTabularInline that has the problem

Try to inherit from unfold.admin.TabularInline as well.

I just tested an example below and it works without any problems
from django.contrib.contenttypes.admin import GenericTabularInline
from unfold.admin import TabularInline

class TagGenericTabularInline(TabularInline, GenericTabularInline):
    model = Tag


Improvement: RAG Search

lol.
Incase you're curious we use named entity recognition models to extract key words / phrases. Then use bm25 + vector search to identify the top results!

So, set your expectations accordingly. You're not getting your perfect RAG in an afternoon.

"What do you mean difficult? It's three lines of code with LangChain, lol, noob," Sure, getting started is
simple but making it actually good? lol right back at you.

You see plenty of people disappointed with RAG, claiming it doesn't meet business needs and isn't as
great as hyped.  They put together
the most overengineered data transformation pipelines possible, using the latest "meta" they picked up from
 Medium or TikTok, all held together by the shittiest code lifted straight from LangChain and LlamaIndex.

Or they opt for the lazy approach... Just chunk it up, rely on large context windows, dump everything into a
 single vector store, and trust in the magic of the LLM to somehow make the result good.
This approach has also zero real-world applicability.

So yeah, it's tough, but I promise, you can make any use case work. But getting there is real work, and you
need to decide if the effort is worth it.

My Recommendations:

    Set Up an Evaluation Pipeline Early: You need to know how good your system is. Don’t start any real work
without the ability to quantify performance. You must be able to say, "If I switch this prompt for that one,
performance improves by 7% but hallucination increases by 10%."

    Educate Yourself: Dive into resources like ragas, dspy, and probably also libraries providing
structured output guardrails like outlines.

    Choose Your Tracing/Logging Framework: Make friends with a framework like W&B, Langsmith, Langfuse,
and my personal favorite, Phoenix.

    ragas docs: https://docs.ragas.io/en/stable/

    DSP< GitHub repository: https://github.com/stanfordnlp/dspy

    Outlines GitHub repository: https://github.com/outlines-dev/outlines

    wandb site: https://wandb.ai/site

    Langsmith: https://www.langchain.com/langsmith

    Phoenix: https://phoenix.arize.com/

Now you have the tools to really measure and optimize your system!

This is mostly trial and error, as dumb as it sounds. The tech is still too new to have a solid
 "best practice" guide. Data varies so much from Source A to Source B that the strategy has to be
tailored for each client, for each use case. However, some ideas from a couple of successful projects:

if you need over 8k tokens, your chunking strategy, retrieval process, ranking, or whatever, SUCKS.
That's why it blows my mind every time I hear people complain that Llama3 only has an 8k token
context. What do you even need more tokens for?  Perhaps not in your "passphrase"
search test, but in real life with real end-users, they will.

Less Is More: Don’t get caught up in merging 20 different RAG pipelines from LangChain and 30
 from LlamaIndex all put together into one frankenstein monster of a pipeline just because you
see in the docs that it's a direct implementation of some new paper. Stick to what works and use
your own brain. Financial data, for example, often just boils down to a few key numbers, but
understanding them requires a web of related data across documents. Classic RAG will let you
down here. You need a system that maps entities via relationships like a Knowledge Graph.

The paper you read yesterday about RAPTOR, HYDE or some new "hot shit" RAG stuff... yeah try it out, but keep
 in mind they are usually not tested with real use cases.  Don't be desillusionized if it don't
 work out. Also never go to [r/machinelearning](https://www.reddit.com/r/machinelearning/) for your
LLM questions. Saddest place on reddit lol.

Have Stamina! Good Luck!

TL;DR: Set up an evaluation pipeline, improve your process step by step. Keep it simple and
chill with context size and optimization craziness.

----->
arthurwolf

    Just chunk it up, rely on large context windows, dump everything into a single vector store,
 and trust in the magic of the LLM to somehow make the result good.

We gather data *very* liberally (using both a keyword and a vector based search), get anything
 that might be related. Massive amounts of tokens.

Then we go over each result, and for each result, we ask it « is there anything in there that
matters to this question? <question>. if so, tell us what it is ».

Then with only the info that passed through that filter, we do the actual final prompt as
you'd normally do (at that point we are back down to pretty low numbers of tokens).

Got us from around 60% to a bit over 85%, and growing (which is fine for our use case).

It's pretty fast (the filter step is highly parralelizable), and it works for *most* requests
(but fails miserably for a few, something for which we're implementing contingencies).

However, it is expensive. Talking multiple cents per customer question. That might not
 be ok for others. We are exploring using (much) cheaper models for the filter and seeing good results so far.

----->----->
nightman

I recommend to try Reranking (like Cohere reranking and filtering based on relevance_score)
instead of current filtering. It might not work for you but it's a middle ground between naive
 vector store retreival and checking each document with LLM if it fits.

LongRAG
https://github.com/TIGER-AI-Lab/LongRAG

Add Wikipedia + X wiki support.

Un-examined
ColBERT
https://www.answer.ai/posts/2024-08-13-small-but-mighty-colbert.html
https://arxiv.org/abs/2406.11706
https://arxiv.org/abs/2303.00807
RAG Papers
https://arxiv.org/abs/2408.05025
https://arxiv.org/pdf/2408.02545
https://arxiv.org/pdf/2408.03910
https://arxiv.org/pdf/2408.03325
https://arxiv.org/abs/2407.19813
https://arxiv.org/abs/2407.19825
https://arxiv.org/abs/2407.21712
https://arxiv.org/abs/2407.19825
https://arxiv.org/abs/2407.13481
https://arxiv.org/abs/2408.02545
https://arxiv.org/abs/2408.06778
https://arxiv.org/abs/2408.03910
RAG Techniques Implemented
https://github.com/ShishirPatil/gorilla/tree/main/raft
https://arxiv.org/html/2408.06941v1
https://github.com/NirDiamant/RAG_Techniques
https://github.com/NirDiamant/Controllable-RAG-Agent
https://github.com/EdwardDali/erag
https://github.com/gusye1234/nano-graphrag
Graph Reasoning
https://jdsemrau.substack.com/p/graph-reasoning-structures-and-concepts
Eval
https://arxiv.org/pdf/2408.02666v1
https://arxiv.org/abs/2407.18416
https://arxiv.org/html/2408.02666v1
Reasoning
https://arxiv.org/abs/2408.06195

Well, I aint waiting 6 months.
RAG
Systematically Improving RAG Applications prep:

**Week 1: Cold Start Problem**

- Overview of the Playbook and RAG System Inference Flywheel
- Understanding and Justifying the Playbook
- LLM Synthetic Data Generation for Evaluation
- Leading vs Lagging Metrics
- Evaluation and Improvement Strategies

**Week 2: Identifying Failure Modes**

- Segmenting input space of questions
- Developing specialized systems based on topics and  capabilities
- Improving precision and recall before considering generation
- Working with domain experts to decompose problem space

**Week 3: Extending Capabilities**

- Understanding limited inventory and capabilities issues
- Metadata filtering
- Rerankers w/ COhere
- Fine-Tuning Embedding Models
- Multimodal data handling (PDFs, PPTs, Tables, Images)

**Week 4: Representations**

- Understanding Embeddings and Search
- Routing queries using classifiers
- Evaluating routing effectiveness
- Handling multiple routes
- Parallel Routing with instructor
- Identifying what tools to provide agents

**Week 5: Product**

- Generation techniques
- Streaming responses
- Managing citations
- Feedback UI and collecting labels
- Prompt engineering and rejecting wor

Fuck it. we'll do it live.
https://github.com/asg017/sqlite-lembed
https://github.com/asg017/sqlite-vec

https://research.trychroma.com/evaluating-chunking
https://github.com/brandonstarxel/chunking_evaluation
https://research.trychroma.com/embedding-adapters

Using RAGBuilder with one of their pre-defined datasets:
from langchain_openai import ChatOpenAI

from langchain_community.document_loaders import WebBaseLoader
from langchain_openai import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_chroma import Chroma
from langchain_community.retrievers import  BM25Retriever
from langchain.retrievers.multi_query import MultiQueryRetriever
from langchain.retrievers import ParentDocumentRetriever
from langchain.storage import InMemoryStore
from langchain.retrievers.document_compressors import CrossEncoderReranker
from langchain_community.cross_encoders import HuggingFaceCrossEncoder
from langchain.retrievers.document_compressors import EmbeddingsFilter
from langchain.retrievers import ContextualCompressionRetriever
import os
from operator import itemgetter
from langchain import hub
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough, RunnableParallel, RunnableLambda
from langchain.retrievers import MergerRetriever
from langchain.retrievers.document_compressors import DocumentCompressorPipeline

def rag_pipeline():
    try:
        def format_docs(docs):
            return "\n".join(doc.page_content for doc in docs)

        llm=ChatOpenAI(model='gpt-4o')

        loader = WebBaseLoader('https://github.com/KruxAI/ragbuilder/blob/main/SampleInputFiles/docs-get_started-quickstart.md')
        docs = loader.load()

        embedding=OpenAIEmbeddings(model='text-embedding-3-large')

        splitter = RecursiveCharacterTextSplitter(chunk_size=1800, chunk_overlap=200)
        splits=splitter.split_documents(docs)
        c=Chroma.from_documents(documents=splits, embedding=embedding, collection_name='testindex-ragbuilder-1724574175',)
        retrievers=[]
        retriever=BM25Retriever.from_documents(docs)
        retrievers.append(retriever)
        retriever=MultiQueryRetriever.from_llm(c.as_retriever(search_type='similarity', search_kwargs={'k': 10}),llm=llm)
        retrievers.append(retriever)

        parent_splitter = RecursiveCharacterTextSplitter(chunk_size=5400, chunk_overlap=600)
        splits=parent_splitter.split_documents(docs)
        store = InMemoryStore()
        retriever = ParentDocumentRetriever(vectorstore=c,docstore=store,child_splitter=splitter,parent_splitter=parent_splitter)
        retriever.add_documents(docs)
        retrievers.append(retriever)
        retriever=MergerRetriever(retrievers=retrievers)
        arr_comp=[]

        model = HuggingFaceCrossEncoder(model_name="BAAI/bge-reranker-base")
        arr_comp.append(CrossEncoderReranker(model=model, top_n=5))
        pipeline_compressor = DocumentCompressorPipeline(transformers=arr_comp)
        retriever=ContextualCompressionRetriever(base_retriever=retriever,base_compressor=pipeline_compressor)
        prompt = hub.pull("rlm/rag-prompt")
        rag_chain = (
            RunnableParallel(context=retriever, question=RunnablePassthrough())
                .assign(context=itemgetter("context") | RunnableLambda(format_docs))
                .assign(answer=prompt | llm | StrOutputParser())
                .pick(["answer", "context"]))
        return rag_chain
    except Exception as e:
        print(f"An error occurred: {e}")

##To get the answer and context, use the following code
#res=rag_pipeline().invoke("your prompt here")
#print(res["answer"])
#print(res["context"])

Tracking RAG-related research
https://arxiv.org/pdf/2408.14906
https://arxiv.org/pdf/2407.14482
https://www.turingpost.com/p/longrag
https://arxiv.org/pdf/2408.14717

https://pub.towardsai.net/reliable-agentic-rag-with-llm-trustworthiness-estimates-c488fb1bd116
https://docs.datacommons.org/papers/DataGemma-FullPaper.pdf

https://huggingface.co/lightblue/kurage-multilingual

Closing since RAG has been added. And all links have been shuffled/sorted to their proper issues page.


Add missing global 'security' declaration to openapi.proto

Fixes ApiKeyAuth method being used by clients generated via OpenAPI generator.
Original idea: https://github.com/authzed/authzed-go/pull/255

Gonna have a look at this today. Thanks for putting it up!

How does it differ from what's right above it? https://github.com/authzed/api/pull/120/files#diff-2c89108fef442d02f29499535203f20595b6fdd3cff830a5b3fd40e979eece0bR34-R41

Okay maybe I have messed up the syntax a bit. Let's take it different, as I am not familiar with buf.
This is what the target swagger.json should look like:

I have updated the PR, hope that makes more sense now.

Hmm... the thing I'm getting hung up on is that there's two security blocks. Is that correct/expected? Should we change the existing security block rather than adding a new one?

No I think this is totally fine - even if not being really straight forward at fist. The official docs unfortunately use yaml for examples already.
So you have essentially two elements:
securityDefinitions defines what is actually available from your API, but it does not apply that auth method to anywhere.
Here it just says: The API supports an Authentication named ApiKeyAuth that is of type apiKey and will be applied by using a header named Authorization.
In the security section you just name where your defined security methods are going to be applied by just cross referencing them. Line 3035 means, take the auth method named ApiKey and then apply it to all endpoints ([]).
Even if that looks unfamiliar, it is confirmed working. So you might want to take a look here:
https://github.com/ewerk/authzed-http-client/blob/fc74dc40f8d81720f976b1d866106d4e8907bfed/.github/workflows/main-build.yml#L34

@tstirrat15 Could you also please review - I am not able to request that and it seems like a second review is required.

Yep, this makes sense now. Thank you!

I checked this out and it looks good. I'm going to go ahead and merge this, and then I'll do the work to pull it into authzed-go.


Merge main

Description
Merged main
Type of Change

[ ] ✨ New feature (non-breaking change which adds functionality)
[ ] 🛠 Bug fix (non-breaking change which fixes an issue)
[ ] ❌ Breaking change (fix or feature that would cause existing functionality to change)
[ ] 🧹 Code refactor
[ ] ✅ Build configuration change
[ ] 📝 Documentation
[x] 🗑 Chore

@soeren-schmaljohann-2denker think you made the changes from the wrong branch. Changed the target to fixes but now there are conflicts

Created new PR #2 with correct branch


[feat] System Brightness listener

Description (What is the problem)
Would be good to have a possibility to listen to system brightness change
Solution suggestion
Add a Stream<double> onSystemBrightnessChanged like  the way we have it for the current brightness

Please wait for feature finish

@WorkNedoPrograMax this feature should be added in version 2.0.0+1


Uninstall unused dependencies

What needs to happen?
Find unused npm dependencies and uninstall them (remove from package.json)
Why should we do this?
There are some dependencies that we forgot to uninstall as we deleted unused code over time. We should remove them to remove bulk and reduce confusion.
When does this need to get done?
Not urgent but shouldn't be a big effort

Are there tools that can identify unused package.json dependencies?

Yeah, I ran this tool called depcheck and got this but I think I'll need to go through and human-check these too. Pretty sure some of these are used.
Unused dependencies
* @aics/simularium-observables-manager
* @types/victory
* momentjs
* rc-motion
* react-player-controls
* url-loader
* victory
Unused devDependencies
* @babel/cli
* @types/enzyme
* @types/jest
* @types/jsdom
* babel-loader
* css-loader
* enzyme
* eslint-plugin-react-hooks
* file-loader
* husky
* identity-obj-proxy
* jsdom
* jsdom-global
* less
* less-loader
* lint-staged
* postcss-loader
* prettier
* ts-node
* webpack-cli
Missing dependencies
* moment: ./src/components/ViewerTitle/index.tsx

might try this one too: https://www.npmjs.com/package/npm-check

Ok, I tried both. Just for future reference, neither are perfect but depcheck worked much better. npm-check listed way more deps under "unused" that were actually used (like lodash!)


No AX.25 port data configured

Hi,
I'm not sure that bower is the problem, but each time I try to launch it, i get this message in the syslog.
for information :
npm -v
1.3.5
nodejs -v
v0.10.15
regards,

You're probably installing and running the node package which is an amateur radio related tool - I had this issue installing node-red.
Try this to fix it:

sudo ln -s /usr/bin/nodejs /usr/bin/node


Python 3 Compatibility

This was tested on reading a gse SOFT file. Also, no more tests are failing in python 2 and python 3 than before. Fixes #4

This is really good improvement. I wanted to make it for the new release as anyway python 2 support is ending and everybody is moving to python 3.

I already pushed it to pypi :)

It looks from this thread that geoparse if python 3 compatible, but I am having an issue with conda installing where it says it only works with python 3.5*. Looks from the Travis build info, its being tested on 3.7-dev. Any idea as to why it may only be compatible with 3.5 (at least with the conda install?
`70.44s$ conda install --yes -c gusdunn geoparse
Fetching package metadata .............
Solving package specifications: .
UnsatisfiableError: The following specifications were found to be in conflict:

geoparse -> python 3.5*
python 3.6*
Use "conda info " to see the dependencies for each package.
The command "conda install --yes -c gusdunn geoparse" failed and exited with 1 during .`

Hi, at least for my setup it is compatible. All the test are running. It could be something with the conda environment. I am not familiar with it. Can it be that the conda package requires something else than pip?

I looked into it more -- still no solution for the conda build in Travis, but it seems to work with all versions of python when pip installing geoparse and other dependencies.

This looks like a conda problem. Maybe you would try to contact the uploader for the package (gusdunn)?


Hello, I have another question. I see some codes related to testing in the code, but I don't see any commands for using the test. Does point cloud classification require testing? If so, can you tell me the command for the test?

Hello, I have another question. I see some codes related to testing in the code, but I don't see any commands for using the test. Does point cloud classification require testing? If so, can you tell me the command for the test?

Apologies for the inconvenience. You can try this command:
CUDA_VISIBLE_DEVICES=0 python main.py --config path/to/config --test --exp_name saving/dir/path --ckpts path/to/ckpt-best.pth

Thank you so much!


Build remotely on Ubuntu 20.04

What is the goal of this PR?
In order to speed up our builds, we should do them remotely on BuildBuddy
What are the changes implemented in this PR?
Tweak .bazelrc settings to build on BuildBuddy with our own Ubuntu 20.04 image

@vmax @alexjpwalker Is client-python build still not working?

Replaced with #167

Replaced with #167


fix: queue size as specified count

Summary by CodeRabbit

Enhancements

Improved configuration of the stream buffering behavior for better performance.

Bug Fixes

Ensured core functionality of chunk enqueuing and stream termination remains intact.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 97.45%. Comparing base (cf40f9d) to head (29192f7).

Additional details and impacted files
@@           Coverage Diff           @@
##           master      #20   +/-   ##
=======================================
  Coverage   97.44%   97.45%
=======================================
  Files          52       52
  Lines        1293     1296    +3
  Branches      193      193
=======================================
+ Hits         1260     1263    +3
  Misses         32       32
  Partials        1        1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.