Outreachy December 2024 Guidance

Hi all,
here's some information about the Outreachy application period. First of all: welcome! 🥳
For applicants
Thanks for contributing to ocaml-api-watch! Here's some information that's additional to the project description on the Outreachy site.
Contributions
Where to find issues
You can look for issues in the issue tracker of this project. In particular look for issues labelled with good first issue
First Steps
First of all, to contribute you'll need to get OCaml up and running. If you haven't already done that, you can have a look at the Installing OCaml page on the ocaml.org site.
How to Contribute
Once you got OCaml up and running, you can head to the CONTRIBUTING.md guide on how to setup the project, building it, running the test suites and everything you need to know to make your first contribution.
Where to ask for help
You should not hesitate to ask questions if you need help. You can ask directly on this issue if you need help with this project, how to contribute, etc...
If you need help with a particular issue, you can ask directly on that issue.
We'll add links to other communication channels here later on.
Mentors
The mentors for the application period and for the internship are @NathanReb and @panglesd. We're both very excited about having Outreachy applicants and later an Outreachy intern contributing to this project!
On weekends we tend to be non or less available.
For Non-applicants
Outreachy is an initiative to give folks from under-represented groups in open-source the opportunity to do a remote internship in an open-source project. So for open-source communities, Outreachy is a great opportunity to work together with folks from lots of different backgrounds and maybe even grow the community and become more diverse! Outreachy provides the infrastructure for the internships and application process, and the open-source communities need to provide projects and mentoring. One of the projects we -i.e. the OCaml community- is participating with in this Outreachy round consists in writing a structural diffing algorithm for OCaml interfaces in ocaml-api-watch. We might get some Outreachy applicants contributing to this repo from October 1st to 29th 2024 as part of the application process to that project :)

Welcome here outreachy applicants!
Do not hesitate to say hello in this thread, and to ask any question on this specific internship!
Setting up OCaml
The first thing you'll want to do is to setup OCaml up and running into your machine. There are good resources for that in the official ocaml.org website, such as this one. If you are new to OCaml, you'll likely benefit from reading some docs on the language there too!
We won't provide general assistance for setting up OCaml on this thread. However there is a discord server with an outreachy channel where you'll find help when needed!
Being assigned a good first issue
Once you are ready to contribute, the contributing.md file is a good start. But I wanted to emphasize that during the contribution period, in order to avoid having multiple applicant working at the same time on a single issue, we will require you to be assigned an issue. And, in order to avoid an issue being assigned for too long to a single person, we require you to have successfully built the project before being assigned one!
So, whenever you have built the project and are ready for contributing, simply find an unassigned "good first issue", post a message in the thread with a screenshot of the man page of api-diff (that you can get with dune exec api-diff -- --help), and we'll assign you the issue!
Conclusion
Once again, welcome, we are glad to have you interested in the project!

I'm really excited to get the chance to contribute to the OCaml API Watch project again! It's been amazing to see the progress so far, and I'm thrilled to help make diffing tools more accessible for OCaml developers. Looking forward to learning more and working with the team to keep things moving forward!

I am very excited to join and contribute toward the realization of this project. I look forward to working closely with the mentors and colleagues to make valuable contributions to the OCaml community.

Hello Team,
I have successfully set up my environment, and the build works well. The tests run without any output(dune runtest), as expected. However, when I run the command dune exec api-diff -- --help, I receive a strange output, which I have attached below.

I'm using macOS.

Hi @marcndo ! I'm glad to see you successfully setup your environment. Please pick an issue that hasn't been assigned already and that you'd like to tackle and ask to be assigned in the issue comments directly!
PS: note that your dune exec invocation is wrong, the -- part should go after dune exec: dune exec -- api-diff --help. The -- part tells dune exec not to interpret any command line flags following the -- and that anything that comes after that must be interpreted by the program invoked throuh dune exec.

OK, thank you for the correction. I tagged you already  on an issue i would like to work on sir. However i will proceed to comment again on the issue for beter visibility.


fix: do not requeue old placement data

When placement related data is deleted the placement data is retrieved from the database and queued to be processed.
If the queue already has a placement update or delete message then we will process the new details, before then processing the old details.
Change the operation type from LOAD to LOOKUP so we always grab the latest data available in the database. This will remove the ability for overwriting data with old data, but we will still get duplicate effort as we will process the placement twice. This should be considered acceptable until the sync process can be overhauled.
TIS21-6701

LGTM. I am looking forward to the funky integration test to demonstrate the original issue is resolved 😉


Use tox to run pytest-copie unit tests locally on multiple python versions

Change Description
This proof of concept PR shows that we can use tox with a simple configuration file to execute the pytest-copie-based unit tests locally on a group a set of different python versions.
There's a fair amount of dev environment setup that is required:

Install tox in your working environment.
Install pyenv - typically this is done with brew install pyenv.
Run pyenv init to see what updates to make to your .bashrc file to finish the installation.
More instructions for pyenv install and setup are here: https://github.com/pyenv/pyenv?tab=readme-ov-file#installation

With pyenv installed you can now install multiple versions of python. Note however, that pyenv actually builds python on your system, so you need to make sure that you have the python build dependencies installed as well. Instructions for doing that are here: https://devguide.python.org/getting-started/setup-building/#install-dependencies

You can now build and install various versions of python like so pyenv install 3.11.
You can also run pyenv versions to see the versions of python that are available. Note that this doesn't include the python in your conda environment, or the default system python version.
Now from the command line, change to the directory that has the tox.ini file in it.
Run pyenv local 3.11.7 so that the version of python is "available" (i.e. it can be discovered by tox)
Run tox.

The output from the configuration file here will show that it tries to find python versions 3.8-3.11. For each version that tox finds, it will run pytest tests and report the results.
Checklist

[x] This PR is meant for the lincc-frameworks/python-project-template repo and not a downstream one instead.
[x] This change is linked to an open issue
[x] This change includes integration testing, or is small enough to be covered by existing tests

@delucchi-cmu yours is a very fair point. There is certainly more effort involved in setting up tox to run. However, once you have tox installed in your virtual environment (even if you haven't added multiple additional python versions), running tox at the command line will be very similar to running pytest tests at the command line.
tox will simply skip over the versions of python that it can't find, but will run the tests for the versions that it can find.
Putting all that aside though, I really want to have more confidence when we assert "The PPT works for Python >= 3.8".


SEPG Automated Release - v3.3.2

SEPG Details
Release Build Link: https://sou-jenkins2.swinfra.net/job/Auto-Release/756/
Build Token: github.com/CAFapi/opensuse-opensearch1-image/main/450fc577e02c1148692f06a921c8973ff52b022e
Versioning Behaviour: Roll on to next minor
Artifacts For Release
Docker

cafapi/opensuse-opensearch1:3.3.2 (Static Tag)
cafapi/opensuse-opensearch1:3.3 (Dynamic Tag)
cafapi/opensuse-opensearch1:3 (Dynamic Tag)
cafapi/opensuse-opensearch1:latest (Dynamic Tag)

 Build Stage
Action
Status

Release Checks
completed
:heavy_check_mark:

Gather Release Data

:white_small_square:

QA Documentation Site Deployment

:white_small_square:

QA Deployment Branch Promotion

:white_small_square:

Docker Image Promotion

:white_small_square:

Maven Artifact Promotion (Closed Source)

:white_small_square:

NuGet Artifact Promotion

:white_small_square:

Artifactory Release Artifact Tagging

:white_small_square:

Generic Artifact Promotion

:white_small_square:

Maven Artifact Promotion (Open Source)

:white_small_square:

Maven Staging Repository Removal (Open Source)

:white_small_square:

Git Tagging

:white_small_square:

Git Release

:white_small_square:

Git Release Asset Deployment

:white_small_square:

Pull Request Merge

:white_small_square:

Release Branch Deletion

:white_small_square:

view the build log here


Read mail in plain text

Hello,
I might have missed it, but is there an option to view all mails in plain text first ? I only found the option for composing or answering mails, but not for viewing. I would like to get the plain text first and then - if wanted - switching to HTML view.
Is there such an option ?
RainLoop version: 1.12.0
Plugin: custom mapping
Thanks in advance!

There's no option for it but it has been previously requested.


animated gifs - hardcoded frame limit of 50?

Hi,
first of all, thanks for this awesome work!
I've encountered a problem though:
I'm using ImageCraft to add a partitally transparent PNG image
to an animated GIF, it work perfect for any GIF between 2 and 50 frames,
but whenever I'm using an animated GIF with more than 50 frames, the finished
image is just the added PNG, the GIF "background" image is missing.
Is there a hard coded limit of 50 frames?
Thanks in advance!

You mean the PNG watermark lost its transparency? If so, yes, it is intended. Imagecraft has two mode for animated GIF, the high quality mode and low quality mode, when the GIF image is considered too big (too many frames or dimensions are too large), Imagecraft will fall back to low quality mode or "protection" mode, see this. The reason for this is that PHP has its memory limit and PHP is not that good at manipulating images. Without this protection, when a GIF image is too big, it will take a very long time to finish the manipulation and it has a high chance to crash your server.
If you want to fix this, my suggestion is to switch to another library like imagemagick or its cli tools, or even  another language, nodejs or golang is a good choice, they are 100x faster than PHP. Anyway, thanks for reporting this, if you find any other issues, feel free to post here. Thanks.

Hi coldume,
thanks for your reply, the line of code you linked explained why it's not working like expected for me:
The GIFs are 468x100pixels, so anything above 51 frames (= 2 386 800 pixels) would be larger than the pixel limit you set.....
Unfortunately, due to several reasons, I'm bound to PHP, so I have to try to increase that limit a bit......


CardElement on* events aren't firing

I'm attempting to use the CardElement and none of the on* events (onBlur, onFocus, onChange, etc.) seem to be properly connected. Though everything seems fine with submission when calling createPaymentMethod().
I'm porting over React code and was using the onChange event to handle errors from Stripe in real-time, but that doesn't seem possible atm.
Versions
"@stripe/stripe-js": "^4.9.0",
"solid-js": "^1.9.3",
"solid-stripe": "^0.7.0"

Extremely basic repro
import { createSignal, onMount } from 'solid-js';
import { loadStripe } from '@stripe/stripe-js';
import { CardElement, Elements } from 'solid-stripe';

import type { Stripe } from '@stripe/stripe-js';

const STRIPE_KEY = '';

const App = () => {
  const [stripe, setStripe] = createSignal<null | Stripe>(null);

  onMount(() => { void loadStripe(STRIPE_KEY).then(setStripe); });

  return (
    <div style={{
      'background-color': '#282c34',
      'min-height': '100vh',
      display: 'flex',
      'flex-direction': 'column',
      'align-items': 'center',
      'justify-content': 'center',
      color: 'white',
    }}
    >
      <div
        style={{
          width: '640px',
          padding: '20px',
          'background-color': 'white',
        }}
      >
        <Elements stripe={stripe()}>
          <CardElement
            onBlur={() => console.log('onBlur')}
            onFocus={() => console.log('onFocus')}
            onEscape={() => console.log('onEscape')}
            onLoadError={(event) => console.log('onLoadError', event)}
            onNetworksChange={(event) => console.log('onNetworksChange', event)}
            onReady={() => console.log('onReady')}
            onChange={(event) => console.log('onChange', event)}
          />
        </Elements>
      </div>
    </div>
  );
};

export default App;

What I expect is that the events fire when expected: onFocus when focus is granted, onBlur when focus is lost, etc. Most importantly, onChange when the Card Element detects a change in state (error, completion, etc.)
What I am experiencing is that no events are firing. Though submission is still possible, the app is unable to see the state of the Card Element and so is unable to do things like disable / re-enable buttons when there's a change in errors or completion.

I think the issue here is that I'm invoking the accessors here and the custom primitive never reacts to it.
Opened a PR that should fix this

Most of the events are firing now (I'm seeing onBlur, onFocus, onEscape, and onReady), however onChange is still not firing.
Also, the component is throwing an error when mounting:
Uncaught (in promise) IntegrationError: When adding an event listener, the second argument should be a function callback.
Upon doing some digging, it looks like any undefined handlers cause this error and any useAttachEvent calls after the error fail to bind. (Or maybe they bind but are immediately unbound because of the error? Either way, they're not attached by the time the form is interactable.)
Changing line https://github.com/wobsoriano/solid-stripe/blob/0119f06c847a0f71aedb5549d83748ecb99acd6c/src/components/createElementComponent.tsx#L102 to
if (!element() || !cb) {

seems to do the trick.

As an aside, options doesn't appear to be reactive. There's a disabled prop in there used to disable the form input. If this would be better off as a separate issue, lmk.

Awesome, thanks for this. I'll release a quick patch for the event listener callback issue. Will do a separate one for the options. Will investigate

Awesome, thanks for this. I'll release a quick patch for the event listener callback issue. Will do a separate one for the options. Will investigate

Decided to see if I couldn't sus it out. Seems like a createEffect with a call to element().update() is what we're looking for. Example:
createEffect(() => {
  if (element()) {
    element()!.update(props.options as any)
  }
});

Yup that's we're looking for. I'll make sure we only update it when needed

@mr-tech should be available in the latest release

@mr-tech should be available in the latest release

Hey. Thanks for the quick responses! I just got around to checking out the releases. Seems as though there's a slight oversight in the callback check.
https://github.com/wobsoriano/solid-stripe/blob/040419061ae4f1f2e319a9942e25506d11f74b54/src/components/createElementComponent.tsx#L118
This should be || not &&.

You want to do a quick PR for that @mr-tech ? 😄


Automated cherry pick of #5620

Cherry pick of #5620 on release-1.46.
/cc  @enahum

/release-note-none


Adding Group Names as Swarm_Labels

Maybe this could be helpful for someone.
Add this at the top of swarm_cluster.yml to set group_names as swarm labels for each swarm_node.
---
- name: "Set SWARM LABELS"
  set_fact: swarm_labels="{{ group_names }}"
  tags:
    - skip_ansible_lint
    - swarm_labels

Thanks! This should now have been merged in master


AgentPod misses serviceaccounts of plugin in controlller namespace

Pre-requisites

[X] I have double-checked my configuration
[X] I can confirm the issue exists when I tested with :latest
[X] I have searched existing issues and could not find a match for this bug
[X] I'd like to contribute the fix myself (see contributing guide)

What happened/what did you expect to happen?
Related  discussion: https://github.com/argoproj/argo-workflows/discussions/12566
Controller runs in namespace: argo,  there is a plugin named khaos-executor-plugin and a serviceaccount named khaos-executor-plugin.
The sample workflow runs in namespace: khaos-workflow, there is a plugin named hello-executor-plugin and a serviceaccount named hello-executor-plugin.
argo get hello-7t78h -n khaos-workflow
Name:                hello-7t78h
Namespace:           khaos-workflow
ServiceAccount:      unset (will run with the default ServiceAccount)
Status:              Error
Message:             serviceaccounts "khaos-executor-plugin" not found
Conditions:
 Completed           True
 PodRunning          False
Created:             Wed Feb 28 17:32:52 +0800 (2 hours ago)
Started:             Wed Feb 28 17:39:01 +0800 (2 hours ago)
Finished:            Wed Feb 28 17:39:01 +0800 (2 hours ago)
Duration:            0 seconds
Progress:            0/1

STEP            TEMPLATE  PODNAME  DURATION  MESSAGE
 ◷ hello-7t78h  main

Version
v3.4.14
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-
spec:
  entrypoint: main
  templates:
    - name: main
      plugin:
        hello: { }

Logs from the workflow controller
time="2024-02-28T09:39:01.368Z" level=info msg="Task-result reconciliation" namespace=khaos-workflow numObjs=0 workflow=hello-7t78h
time="2024-02-28T09:39:01.368Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=khaos-workflow workflow=hello-7t78h
time="2024-02-28T09:39:01.368Z" level=warning msg="[DEBUG] boundaryID was nil" namespace=khaos-workflow workflow=hello-7t78h
time="2024-02-28T09:39:01.368Z" level=info msg="was unable to obtain node for , letting display name to be nodeName" namespace=khaos-workflow workflow=hello-7t78h
time="2024-02-28T09:39:01.368Z" level=info msg="Plugin node hello-7t78h initialized Pending" namespace=khaos-workflow workflow=hello-7t78h
time="2024-02-28T09:39:01.368Z" level=info msg="TaskSet Reconciliation" namespace=khaos-workflow workflow=hello-7t78h
time="2024-02-28T09:39:01.368Z" level=info msg="Creating TaskSet" namespace=khaos-workflow workflow=hello-7t78h
time="2024-02-28T09:39:01.438Z" level=info msg=reconcileAgentPod namespace=khaos-workflow workflow=hello-7t78h
time="2024-02-28T09:39:01.458Z" level=error msg="error in agent pod reconciliation" error="serviceaccounts \"khaos-executor-plugin\" not found" namespace=khaos-workflow workflow=hello-7t78h
time="2024-02-28T09:39:01.458Z" level=info msg="Updated phase Running -> Error" namespace=khaos-workflow workflow=hello-7t78h
time="2024-02-28T09:39:01.458Z" level=info msg="Updated message  -> serviceaccounts \"khaos-executor-plugin\" not found" namespace=khaos-workflow workflow=hello-7t78h
time="2024-02-28T09:39:01.458Z" level=info msg="Marking workflow completed" namespace=khaos-workflow workflow=hello-7t78h
time="2024-02-28T09:39:01.458Z" level=info msg="Marking workflow as pending archiving" namespace=khaos-workflow workflow=hello-7t78h
time="2024-02-28T09:39:01.458Z" level=info msg="Workflow to be dehydrated" Workflow Size=1516
time="2024-02-28T09:39:01.466Z" level=info msg="cleaning up pod" action=deletePod key=khaos-workflow/hello-7t78h-1340600742-agent/deletePod
time="2024-02-28T09:39:01.468Z" level=info msg="Workflow update successful" namespace=khaos-workflow phase=Error resourceVersion=85399559051 workflow=hello-7t78h
time="2024-02-28T09:39:01.491Z" level=info msg="archiving workflow" namespace=khaos-workflow uid=5c4aa38d-4d94-4def-b189-b6908d3133e4 workflow=hello-7t78h
time="2024-02-28T09:39:01.513Z" level=info msg="Queueing Error workflow khaos-workflow/hello-7t78h for delete in 72h0m0s due to TTL"
time="2024-02-28T09:39:56.542Z" level=info msg="Alloc=17143 TotalAlloc=4920922 Sys=61565 NumGC=947 Goroutines=282"
time="2024-02-28T09:40:17.314Z" level=info msg="Performing periodic workflow GC"
time="2024-02-28T09:40:17.316Z" level=info msg="Deleting old offloads that are not live" len_wfs=0
time="2024-02-28T09:40:17.316Z" level=info msg="Workflow GC finished"

Logs from in your workflow's wait container
kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

As mentioned in https://github.com/argoproj/argo-workflows/discussions/12566, I think plugins in controller namespace is useless if workflow runs in other namespace. So, it should not be loaded into AgentPod.

@alexec Do you know why plugins in controller namespace are needed?

A plugin outside the controller namespace is a user plugin. In many set ups, the users cannot modify the controller because they only have access to their own namespace. Loading plugins from the user’s namespace allows the user to self-serve their plugins. I’m not sure a user plugin should try and use a controller service account, so this seems to be a bug.

I’m not sure a user plugin should try and use a controller service account, so this seems to be a bug.

Controller plugins are loaded by default, so controller service account will be access if AutomountServiceAccountToken is true.
https://github.com/argoproj/argo-workflows/blob/af2cacb365a6cc03cc35ed9749976e095f9a03f7/workflow/controller/agent.go#L273-L302
I'm not pretty sure why controller plugins are needed for user workflow. Do you know why?

This seems odd to me too. Agents are part of the workflow, not part of the control-plane, so should run using the service account of the workflow.

This seems odd to me too. Agents are part of the workflow, not part of the control-plane, so should run using the service account of the workflow.

@alexec I see that you support the feature of executor plugin, executor plugins in control-plane are loaded by default, so I wonder why you do this or can you confirm if we can only load plugins in user‘s workflow namespace with this PR:#12724.
https://github.com/argoproj/argo-workflows/blob/889a9d24be072c5d04e853db3d4c40a04f939d28/workflow/controller/agent.go#L270


Google Home unable to pair with HA Bridge

I am using the latest HA Bridge (4.5.6) and tried with the latest Google Home app (<IP_ADDRESS>) and older one as well (1.19.29, 1.19.26).
I have tried various other suggestions provided in other threads, but unfortunately not able to pair it.
I had a working setup until beginning of May, but since then its not working and I am trying hard to reinstate it.
Any help will be highly appreciated.
Please let me know if you need any additional information to debug.

Don't need any other info as this is not the bridges issue, rather google is making it difficult to pair with an emulated hue bridge that is not part of Philips....

Thanks for your response. So is there a work around to work through this issue, or we can't connect to the bridge from Google Home anymore at all? Any suggestion would be welcome.

bumping the thread, as I didn't see any responses. Any help will be highly appreciated

Anyone?
So are people out there with GM not at all able to connect to the HA Bridge? Any solution?

Had this issue but somehow Google must've made another change as I've been able to connect GH to the bridge for the past few weeks. Latest GH and 4.5.6 installed. I just tested again by deleting and re-added Philips Hue back on the GH app. I'd suggest trying it again.

I am using <IP_ADDRESS> GH version and 4.5.6 HA Bridge and tried many times, with no luck :(

When you add Philips Hue in GH, does it start a browser session and redirect you to https://hue-adpter.appspot.com/client/#/activate/pair or does it redirect you to account.meethue.com? When I had this issue back then, GH was directing me to account.meethue.com which requires you to have a hue account. And I assume you have a physical GH hub as it doesnt pair without the hub.

For me its consistently going to https://hue-adpter.appspot.com/client/#/activate/pair, but then eventually never able to pair. The started with going to account.meethue.com, but ever since I have downgraded the App version and played around with various options. Now its consistently goign to the appspot link, but no luck in the pairing.
It end up saying that its not able to find the Hue Bridge.

+1 Just added a new phone and no pairing ..........used to work google must have changed the program again......

My devices hadn't been working for at least a few days (I have an Echo right next to GH so usually use it for devices anyway and GH for queries and Chromecast) but last night I went into the app to see why and it said I had 0 devices! I removed the hue account and re added it, it went to the pairing web page and I clicked pair. It told me to press my hue button which I did not do on my actual hue hub. But it did find all of my 56 ha bridge 4.5.6 devices running on 2 separate pi on port 80...
My girlfriend comes back Tuesday, I'm waiting to see if I can add the devices to her Google account too because in the past it wouldn't find any when I tried on her phone so she can use them from Google Assistant.

So does it pair without a hue bub in the system or you need a hub to confirm pairing?

robertw... dont waste your $$. I stupidly bought a real Hue bridge when I had this issue, thinking the real Hue bridge will act as the go-bwtn HA-bridge and GH.  If your GH is not pairing now, it will still not see the emulated "lights" set up in HA-bridge. It will only see the real Hue lights connected to the real Hue bridge. And when you install the Philips Hue app, it does see the HA-bridge and the emulated lights but only for local use within the same IP subnet.
I dont understand why the pairing is working for some and not others. I've seen others successfully pair in other forums like Smarthings and HASS. Maybe try uninstalling the Home app and re-installing??

I just removed my hue account in GH, unplugged my real hue hub, and readded hue to GH and it still found the ha bridge devices. So shouldn't be related to having a real hue on the network.

I found the root cause of my problem, may work for others as well.
I was trying to use the HA Bridge new binaries, but never recreated the data folder (and it was always using the old configuration and device metadata as stored in there already). I went ahead and deleted the data folder and let it re-create and viola, my GH was able to pair without any issue.
Please give it a try, if you are in the same situation.

You are correct if you del data it will pair but if you try to pair another account it fails
Example pair to my gmail account then try to pair to my wife will not pair the second account......

Interesting. I didn't try with the multiple accounts.

Before i go thru the effort. Will GH running on a RPI connect to HA-bridge?  Alexa wont

@DeetAus Why would think that the Alexa won't connect to ha-bridge. That is the one thing that we know does work will. It does require an Echo or Echo Dot though.

Alexa running on a Rasp Pi
Regards,
David Traino

On 16 Aug 2017, at 11:01 pm, BWS Systems<EMAIL_ADDRESS>wrote:
@DeetAus https://github.com/deetaus Why would think that the Alexa won't connect to ha-bridge. That is the one thing that we know does work will. It does require an Echo or Echo Dot though.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/bwssytems/ha-bridge/issues/696#issuecomment-322771480, or mute the thread https://github.com/notifications/unsubscribe-auth/AMLZ99mOfKgHnTioK0-d20-ZXtoJHlGCks5sYu8hgaJpZM4OSBj2.

That would be true as the wiki says it won't. The software Alexa does not have the upnp connect code for hue.

Referencing the more accurate description in #764  and closing this


(PDB-3911) Add primary key to resource_events

Prior to this commit, resource_events had a unique constraint
on four columns.
After this commit, we add a primary key on those same four columns
and drop the unique constraint.  A primary key is enforced by a
unique constraint behind the scenes so there should be no difference
other than 3rd party tools like pglogical or pg_repack will now
be happy that there is a primary key when they used to fail.

Test PASSed

CLA signed by all contributors.


Option to limit CRs the operator watches

We should be able to restrict what CRs an operator handles, as example, by using a selector:
public @interface Controller(
    LabelSelector[] selectors() default {}
)

This could be required when a canary deployment of operators is required so at certain point in time, there will be more than one operator running in a cluster each one is taking into account a subset of the CRs.

@metacosm I have some spare cycles this week so if you need any help on this feature , I'd be happy to take a stab

@lburgazzoli This sounds awsome!

Ideally, however we implement this, it should boil down to generating a Predicate. Other aspects to keep in mind as well:

I don't think it will be possible (nor desirable, for that matter) to use LabelSelector on the Controller annotation
Whatever configuration we add to the annotation, it should be possible to have similar configuration in application properties as well

I don't know if a predicate would be the best approach as then it would filter events after we receive them, instead a label selector (or anything similar) would configure the watcher to only watch some resources

I don't know if a predicate would be the best approach as then it would filter events after we receive them, instead a label selector (or anything similar) would configure the watcher to only watch some resources

Ah, right. Should we also filter on annotations?

I guess on quarkus a possible representation using properties could be:
quarkus.operator-sdk.controllers."foo".selectors[0] = "environment=production"
quarkus.operator-sdk.controllers."foo".selectors[1] = "tier!=frontend"

or
quarkus.operator-sdk.controllers."foo".selectors = "environment=production,tier!=frontend"

I'd prefer the second option (which I think Quarkus supports out of the box, iirc).

I don't know if a predicate would be the best approach as then it would filter events after we receive them, instead a label selector (or anything similar) would configure the watcher to only watch some resources

Ah, right. Should we also filter on annotations?

As far as I know, you can't use annotation to select objects, only labels can be used.

Yep annotations are not indexed in etcd AFK, so we cannot filtern on them.
I particulary like the idea to the the filtering on server side (K8S API) if possible based on labels, this is much more efficient, then doing it with the predicate in the operator. Although it's limited to labels, so probably we need both.
An interesting thing is that admission controllers can do this based on namespaces so namespace selector:
https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-namespaceselector
What we support to some extent, just like naming a list of namespaces, not based on labels. But for canary deployments and testing purposes I think could be quite nice too.

@metacosm @csviri I'm working on a POC for this issue so we can discuss about code and wonder if we could add some tests based on the KubernetesServer in addition to the mocked tests as it simplifies things a lot.

I agree that we need to beef up our testing story and adding tests based on KubernetesServer would make sense to explore that space and see what we're missing in terms of testing support.

Don't know much about that mock server, but worked with the one from kube builder - that was awsome. For development purposes I'm happy personally with minukube / kind. Very happy to discuss this, maybe on weekly, improving tests is always a good thing :)


mdadm: allow sending mail when using opensmtpd

opensmtpd does not require the setuid bit for its sendmail. This works around it by wrapping the called sendmail so that the wrapper falls back on either the setuid sendmail or the non-setuid sendmail depending on what's available.
I don't know whether that's something that happens often, but maybe it'd make sense to generate a linkfarm to both current-system/sw/bin and wrappers/bin, for when an absolute path is requested?
Tested on 18.03.

Success on x86_64-linux (full log)
Attempted: mdadm
Partial log (click to expand)
shrinking RPATHs of ELF executables and libraries in /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0
shrinking /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/sbin/mdadm
shrinking /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/sbin/mdmon
gzipping man pages under /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/share/man/
strip is /nix/store/fzcs0fn6bb04m82frhlb78nc03ny3w55-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/lib  /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/sbin
patching script interpreter paths in /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0
checking for references to /build in /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0...
moving /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/sbin/* to /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/bin
/nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0

Success on x86_64-linux (full log)
Attempted: mdadm
Partial log (click to expand)
shrinking RPATHs of ELF executables and libraries in /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0
shrinking /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/sbin/mdadm
shrinking /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/sbin/mdmon
gzipping man pages under /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/share/man/
strip is /nix/store/fzcs0fn6bb04m82frhlb78nc03ny3w55-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/lib  /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/sbin
patching script interpreter paths in /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0
checking for references to /build in /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0...
moving /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/sbin/* to /nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0/bin
/nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0

Success on aarch64-linux (full log)
Attempted: mdadm
Partial log (click to expand)
/nix/store/3g02fy269y31jkn3yr5r1v91k9ffia3m-mdadm-4.0

Just added information to the commit message. Also copying it here as it'll be easier to access:
The solution of relying on `$PATH` to be set is unfortunately unreliable, as `mdadm --monitor` will likely be executed from a `systemd` unit, that runs with a clean `$PATH`.

So I thought it'd be better to just follow what the previous mdadm did, and depend on the current environment.

Success on x86_64-linux (full log)
Attempted: mdadm
Partial log (click to expand)
/nix/store/xzymic7fcjdcxw88v10j4amnyz09bl79-mdadm-4.0

Success on aarch64-linux (full log)
Attempted: mdadm
Partial log (click to expand)
/nix/store/3g02fy269y31jkn3yr5r1v91k9ffia3m-mdadm-4.0

Both approaches have their benefits.
I personally prefer to not leak NixOS specific paths into nixpkgs (the packages). Sometimes it doesn't really matter, other times it makes packages break on non-NixOS. By the way, here you can see the $PATH approach implemented in the kbfs NixOS module. Does mdadm work with sendmail PATH lookup and something like path = [ "/run/wrappers" "/run/current-system/sw/bin" ]; in the systemd service definition?

I can confirm it does work with PATH, actually that's the way I initially chose to develop. The issue I came upon with that is that there are likely people with services such as:
systemd.services.mdadm-monitor = {
  description = "Monitor RAID disks";
  wantedBy = [ "multi-user.target" ];
  script = "${pkgs.mdadm}/bin/mdadm --scan --monitor -m root";
};

And such a script would be silently broken by such an update: it would no longer send any email were one of the disks to fail. This sounds like an easy way to have someone hate nixpkgs, unfortunately.
And I don't really know of any way to handle the switch cleanly. Using lib.warn to warn on all installs of mdadm until 18.09 seems… a bit overkill. And I'd rather not just break everything silently until then.
Do you have any idea how to handle this better?

Good point. (I have a hand-written service like that myself.) The problem is that we don't have a NixOS module for mdadm, https://github.com/NixOS/nixpkgs/issues/31840. Let's merge.

Applied to master (ec9a51d0cc6fab730bfe10d354b47c98ddb37460).

Thanks!


TSS and Water Quality

Resolves #63
Resolves #62
Processing code for TSS and water quality
L1 datasets include rows for data not collected (i.e. L1 is a full dataset)

tss remove rows for any NAs since measured on same instrument

ask PR where we got ORP ranges, seems like in some systems it can go negative
https://www.ysi.com/parameters/orp-redox

ask PR where we got ORP ranges, seems like in some systems it can go negative
https://www.ysi.com/parameters/orp-redox

changed ORP range to -250 to 500


Move Selu Constants to separate file

This is a follow up from #621 to move constants to separate file.

This change is

Review status: 0 of 4 files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.

src/math/unary_ops.ts, line 193 at r1 (raw file):

          const mask = x.greater(ops.scalar(0));

          const scaleAlpha = Scalar.new(unary_utils.SELU_SCALEALPHA);

Scalar.new is deprecated, can you use ops.scalar like we're doing on the left?

src/math/unary_utils.ts, line 1 at r1 (raw file):

export const SELU_SCALEALPHA = 1.7580993408473768599402175208123;

can we call this file selu_util.ts?

Comments from Reviewable

 Thanks for following up with this!

Review status: 0 of 5 files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.

Comments from Reviewable

Review status: 0 of 5 files reviewed at latest revision, 3 unresolved discussions, some commit checks failed.

src/math/unary_ops.ts, line 24 at r2 (raw file):

import * as ops from './ops';
import * as selu_util from './selu_util';
import {Scalar, Tensor} from './tensor';

travis failed because of unused variables. log: https://travis-ci.org/PAIR-code/deeplearnjs/builds/338364424?utm_source=github_status&utm_medium=notification
Everything else is looking good

Comments from Reviewable


Fix for METAMODEL-128

Please review the code fix for the mentioned defect. One test case gives false failure because we are comparing query's string form when an alias of item is used. I have modified that.

I thought returning the same reference might prove risky and have unexpected behavior. But yes, what you said makes sense.

Yes normally I would also be in favor of making a copy. But in this case it is really a "reference" and not an independent item. So if you change that then I think we can make the general change a lot less - then you don't need to make the clone method public and you don't need to change the existing unittest.

No that existing unit test still changes because the alias name in query will be replaced by ..i.e. "foobarbaz" gets replaced by "a.foo"

Anyhow I'll make that change you suggested

Hi, I have made the change to return same item instead of clone.

Fine I'll remove that getAllSelectItems() method but isn't subquery in from and where class there in to do list of MetaModel. I remember seeing it in jira for improvement.

SELECT... FROM (SELECT name as n FROM ...) as q where n='foo';  is perfectly valid but will fail if we don't consider select items of other clauses if and when we support sub queries for from and where clause in MetaModel.

Hey, I noticed one thing just now. We can actually use subqueries in from clause at least for JdbcDataContext. Below is the query I ran which was successful.
Query query = dataContext.parse("SELECT password as X from (select username,password from AMASS_USER)  S");

Correct, that's the one type of subquery we support. One that is in the FROM clause. We don't support it in select or where (currently).

We can also use subqueries in where clause. I successfully executed below query.
Query query = context.parseQuery("select username from AMASS_USER where password in (select password from AMASS_USER where enabled=1)");
Considering all the places where we can have alias name for columns wouldn't it be better to have the getAllSelectItems() method. Or shall I still remove the method and use only items from select clause.

I believe that where clause will only work on JDBC because it gets carried along as one big expression. But MM unfortunately will not understand it so no select items will be there anyway. Hmm how about then making a getSelectItemByAlias(...) method. We could start with just items from SELECT but could eventually expand it if subquery support increases.

Ya.. that sounds like a good plan.. I'll make the necessary changes and commit it today

Committed with recommended changes...


Enhance logging: Add support of storing logs in local files.

Is it possible to write output to file? like ./dbs-source-reader > log.txt. Now it save only launch message.

Added in DBS release 0.8.5


[Bug]: The settings in the desktop version did not disappear after clicking save

ODC version
odc424
OB version

What happened?
https://github.com/oceanbase/odc/assets/143492234/4fea8293-0696-4449-9419-d7d62f83a353
What did you expect to happen?
Settings disappear after clicking save
How can we reproduce it (as minimally and precisely as possible)?
Click save in the settings section
Anything else we need to know?
No response
Cloud
No response

fixed

pass


feat(stdlib,interp): Allow multiple overloads for stdlib functions

Some standard library functions require full-fledged signature overloads!  Implementing that in brs basically requires multiple dispatch to be implemented.  But now multiple signatures for a single stdlib function is possible!  The correct signature and its associated implementation will be dynamically chosen at runtime for each function invocation.
A neat side-effect is that type-mismatch error messages are much nicer now!

New error message format:
[Line 1] Provided arguments don't match any of Mid's signatures.
    function Mid(s as String, p as Integer) as String:
        * Mid requires at least 2 arguments, but received 1.
        * Argument 's' must be of type String, but received Integer.
    function Mid(s as String, p as Integer, n as Integer) as String:
        * Mid requires at least 3 arguments, but received 1.
        * Argument 's' must be of type String, but received Integer.
Error occurred

Just gonna merge this — there was a ton of change in the stdlib tree to match the new Callable constructor, and a lot of logic moved around from BrsFunction.ts and interpreter/index.ts into Callable.ts, but that's a good thing I guess.
Otherwise there were basically no test changes other than a few checks that looked for specific error messages.  It doesn't seem like we've broken anything!


SWC vs Babel in Vite

Vite 4.0 has built-in support for SWC and supports using it for react project, and Vercel is pushing Turbopack which is also based on SWC with no Babel. Should this project support SWC to be forward compatible with new build tools?

Would love to see SWC support in the future for this preset 🙏

Bump!

Please don't "bump" issues, it has absolutely no positive effect.
If someone wants to contribute something, we'll consider it, but right now the team doesn't have the resources to invest into such a thing.


Make OAuth scopes configurable

Title: Make OAuth scopes configurable
Description:
It's currently hard-coded to scope but the scopes can vary widely, e.g. see Github's OAuth scopes: https://developer.github.com/apps/building-oauth-apps/understanding-scopes-for-oauth-apps/#available-scopes
cc @rgs1 @fishcakez @williamsfu99

FYI you tagged a random William

🤦 woops, fixed

I am getting openid is required scope from my IdP (actually a locally running test one).
openid is a required scope for oidc. Would be great to be able to specify them.
Is this filter intended to be used for oidc (i.e. propegating ID tokens, cookies, etc.) or only for oauth2 (subtle difference I know).

@Smirl
I don't see an issue running this filter with OIDC/SAML for enterprise apps, although I don't have any experience running the filter with any sort of SSO gateway to speak more on the subject. I would not recommend federating the access tokens for consumer apps though, as internal admins of the filter should be the ones controlling who can access what.

@Smirl I'm currently working on a fix to make the scope parameter configurable, and I'll open a pull request soon. Let you all know by posting it here.
In my local tests, I can see the cookies and Authorization header forwarded in the headers to the upstream service.


Add QuickNode as a company using ViewComponent

Summary
I was looking at the index page and saw:
If your team starts using ViewComponent, [send a pull request](https://github.com/github/view_component/edit/main/docs/index.md) to let us know!

Since we use it heavily at QuickNode, I thought I would send a PR to include us in the list.
Other Information
I've updated the CHANGELOG file as well, as requested by the CONTRIBUTING document.
Thank you for maintaining this great project!

@joelhawksley Thanks for merging! How come I don't see it in the list when I visit viewcomponent.org ?  🙏

@joelhawksley Thanks for merging! How come I don't see it in the list when I visit viewcomponent.org ?  🙏

The website will be updated on the next release of the gem.

Thanks @Spone

@luccastera here you go: https://viewcomponent.org 🎉

@Spone Thank you!


Whitelist for broken websites

Hi, thank you for this useful addon.
I wonder if a simple whitelist mechanism could be implemented, as some websites work incorrectly when resist fingerprinting is enabled.
Thanks a lot in advance.

I could do something, the main problem with it is that resist fingerprinting is an all or nothing setting.  So, if I make it be disabled when you are on a whitelisted site, it will also be disabled for every other site you are on at the time.  Do you have an example of a site that is broken with it on that I can look at?

Do you have an example of a site that is broken with it on that I can look at?
It breaks profile and video pages on TikTok.

Hi, @Aaron-P . Sorry, I forgot to reply before.
For instance testvelocidad.es (a spanish web to measure net speed) always shows "0 ms" for ping and upload test freezes wit resist.fingerprinting enables.

Bumping this, resist fingerprinting completely breaks the ui of pixiv and I use it quite frequently. I currently have firefox esr installed just for pixiv so i dont have to toggle but using another browser for it is a bit annoying lel

I could do something, the main problem with it is that resist fingerprinting is an all or nothing setting. So, if I make it be disabled when you are on a whitelisted site, it will also be disabled for every other site you are on at the time.

After https://bugzilla.mozilla.org/show_bug.cgi?id=1824235 has been solved, resistFingerprinting can be toggled per-site using the privacy.resistFingerprinting.exemptedDomains  property. Perhaps this add-on could be changed to use this property instead. (Mozilla plans to implement a GUI setting, but that's not done yet: https://bugzilla.mozilla.org/show_bug.cgi?id=1450398)

Unless they are planning on adding it soon the exemptedDomains property does not seem to be exposed via the web extensions API, so an extension cannot manage it. You would need to manually edit in about:config.

Oh okay, I see... Anyway, thanks for the quick analysis and response!


Some Updates RFC

Hi i have spent a little bit of time adding to this library in my fork. we simply need to change the URLs back and pull to update these repos.
If anyone is looking for some newer stuff take to speak and feel free to open an issue.  I think ill basically leave it here the goal is to make it a 1.4.2 release and have that be final as far as what is needed fro this component.
The major things added are:

countTokens (a faster way if you don't care about the contents)

tokenStats : some interesting insights into encoded strings such as frequency and position maps.

jsdocs: for implementation interface docs

browserify: for browser support

The major things to test are: if we can get a good working version of the original python version, I added it to the npm package in case it just works but it would be good to have the package support Node, Browser and Python

Also check out the browser demo browser demo

Compatible with Node >= 12
overall just lmk what you think.

https://github.com/latitudegames/GPT-3-Encoder/pull/30
Check out the pull here if interested in a diff

#30
Check out the pull here if interested in a diff

Hey man. I'm just a nobbie but I'm trying to learn with your project.
After I run npm install @syonfox/gpt-3-encoder
I can't import your project.
import {encode, decode, countTokens, tokenStats} from "gpt-3-encoder"
I am getting the error 'Cannot find module 'gpt-3-encoder' or its corresponding type declarations."
:(
I'm using next js

https://github.com/latitudegames/GPT-3-Encoder/pull/31#discussion_r1102012743

Hey @syonfox, I've also spent some time fixing up this package in #38. I don't have the tokenStats implemented in my fork, but I've added some other optimizations and features. See gpt-tokenizer for more info. Open to collab. on further work/features.

It seems we are running into memory issues in our app using GPT-3-Encoder.  Going to try gpt-tokenizer @niieani and will report back if it solves our issue as identified in #38.  Seems there are some improvements in your package that might be good to bring into GPT-3-Encoder.


Missing unvell.ReoGridEditor.dll in ReoGrid-2.0.0-All-Packages.zip

Hi,
Is unvell.ReoGridEditor.dll missed in ReoGrid 2.0? Please check & advise, Thanks

The unvell.ReoGridEditor.exe is the DLL in demo folder. If you really need a DLL of editor library rather then EXE you can download the source code and build a DLL manually.

Hi Jing,
From your ReoGrid-2.0.0-All-Packages download, it contains ReoGrid-2.0.0-Demo-Source.zip the source-code for demo program only, there is no source code for the following DLL, unvell.ReoGrid.dll, unvell.ReoGridEditor.dll, unvell.ReoScript.dll & unvell.ReoScript.EditorLib.dll, but these DLL source were available in previous version.
Question 1: It is your license policy changed?
Question 2: Can I obtain these DLL’s source code? If these DLL sources require extra license, please inform us the license cost, terms & conditions.
Regards,
Richard Hau
From: Jing<EMAIL_ADDRESS>Sent: 2017年8月9日 22:26
To: unvell/ReoGrid
Cc: RichardHau; Author
Subject: Re: [unvell/ReoGrid] Missing unvell.ReoGridEditor.dll in ReoGrid-2.0.0-All-Packages.zip (#34)
The unvell.ReoGridEditor.exe is the DLL in demo folder. If you really need a DLL of editor library rather then EXE you can download the source code and build a DLL manually.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/unvell/ReoGrid/issues/34#issuecomment-321271538 , or mute the thread https://github.com/notifications/unsubscribe-auth/AE0VVlKcpeZ3m6KFh6Coqe1ji7Fz3muOks5sWcGMgaJpZM4OufSd .https://github.com/notifications/beacon/AE0VVreuD4gy-RR-Tk66paliGJKKcDA5ks5sWcGMgaJpZM4OufSd.gif

All source code of ReoGrid and Editor library can be accessed from here
https://github.com/unvell/ReoGrid
(You may have to use git or GitHub desktop to download it)
License changed to MIT since 2017. There is no extra licenses required. Hope ReoGrid helps you.

I have downloaded, thanks.
From: Jing<EMAIL_ADDRESS>Sent: 2017年8月10日 11:33
To: unvell/ReoGrid
Cc: RichardHau; Author
Subject: Re: [unvell/ReoGrid] Missing unvell.ReoGridEditor.dll in ReoGrid-2.0.0-All-Packages.zip (#34)
All source code of ReoGrid and Editor library can be accessed from here
https://github.com/unvell/ReoGrid
(You may have to use git or GitHub desktop to download it)
License changed to MIT since 2017. There is no extra licenses required. Hope ReoGrid helps you.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/unvell/ReoGrid/issues/34#issuecomment-321441248 , or mute the thread https://github.com/notifications/unsubscribe-auth/AE0VVkr2e3qVfbpJ3cjloq1JyJ0YqHdCks5sWnn4gaJpZM4OufSd .  https://github.com/notifications/beacon/AE0VVmaxwNkF5igsmznNJmhphgYjG7g6ks5sWnn4gaJpZM4OufSd.gif


Form builder generate hidden fields without file field.

It would be more convenient if user could generate direct s3 upload form without file input field (just form and hidden fields). I needed it when wanted to use custom file upload field (from jquery-fileupload).
Current version does not allow it because hidden fields generation is sewed into file_field method and required_base_fields are hidden for the user in private scope.

@dwilkie I don't think that failed build is related to my PR's changes. Could you re-run CI build ?

@dwilkie I've rebased my fork against your master and specs pass again


Check the boot entries after the backup restore

https://github.com/sysrich/tik/pull/11 addresses this! thanks!


Add Gradio interface of PwW for image inpainting

Hi, @cloneofsimo
I made few update of this repo.:
(1) Adding UI to prepare "color content" in gradio interface
(2) Add gradio interface of PwW for image inpainting
(3) Update requirement.txt: specifically,
(a) set diffusers==0.10.0 as the current implementation is not supported for higher version
(b) Add gradio
Please let me know if there's any comment to the gradio interface.
Thanks in advance :)

Awesome, I don't know what to say!

I'll just invite you to become a maintainer

No problem :)


Date time does not work

I set my correct time zone during installation, but the Date/Time is incorrect on my installed OS. There does not seem to be any way to correct it, or set to NPT time ..

sudo service ntpdate onestatus
sudo service ntpd onestatus

The following quits ntpdate/ntpd and then runs ntpd to sync the time once, then exits:
sudo service ntpdate onestop
sudo service ntpd onestop
sudo ntpd -g -n -d -q

Does this help?

Wait, did @probonopd merge my localise fix before 0.5?
If so, I regret to inform you that it sets the timezone every boot based not on settings but keyboard layout: english (US) is common in New Zealand but the time zone gets set to New York.
You need to edit localise:
sudo nano /usr/local/sbin/localize

and everywhere it says 'America/New York' you need to change to your timezone, in my case Pacific/Auckland

@linuxkettle
Seems little old New Zealand gets missed out on the timezone settings again ... (I am in NZ as well, so greetings fellow Kiwi) :)
I tried the above and logged out/in but no change, will try re-booting ..
EDIT: Nope, after reboot the same.
@probonopd
I tried the ntpdate/ntpd restart commands above, but still nothings, stays stubbornly on NY time

greetings fellow Kiwi

:~)

Seems little old New Zealand gets missed out on the timezone settings again

Yeah, you really shouldn't need to edit a system program to change the timezone.
We almost need localise to source a configuration file so that you can set a different timezone to your keyboard layout.

Still no way to set the correct locale, I see others in ther threads are havin the same issue .. even using LXQT Config to set the locale does not wok ..

I had to manually set the locale for each of my apps; this might work:
sudo nano /usr/local/sbin/localize

then push Ctrl+M and Ctrl+N so that you can use the mouse to scroll and select, and see the line numbers. Scrolling down, you should see this:
 64 | # Define default values
 65 | default_country = "US"
 66 | default_language = "en"
 67 | default_locale = "%s_%s.UTF-8" % (default_language, default_country)
 68 | default_timezone = "America/New York"

change it to this:
 64 | # Define default values
 65 | default_country = "NZ"
 66 | default_language = "en"
 67 | default_locale = "%s_%s.UTF-8" % (default_language, default_country)
 68 | default_timezone = "Pacific/Auckland"

If you end up spending a lot of time in nano, you can run
sudo nano /usr/local/etc/nanorc

and mess about with the options here:
https://opensource.apple.com/source/nano/nano-3/nano/nanorc.sample.auto.html.
I've enabled mouse support and line numbers by default, changed the keyboard shortcuts and messed about with colours.

I had to manually set the locale for each of my apps; this might work:
sudo nano /usr/local/sbin/localize

then push Ctrl+M and Ctrl+N so that you can use the mouse to scroll and select, and see the line numbers. Scrolling down, you should see this:
 64 | # Define default values
 65 | default_country = "US"
 66 | default_language = "en"
 67 | default_locale = "%s_%s.UTF-8" % (default_language, default_country)
 68 | default_timezone = "America/New York"

change it to this:
 64 | # Define default values
 65 | default_country = "NZ"
 66 | default_language = "en"
 67 | default_locale = "%s_%s.UTF-8" % (default_language, default_country)
 68 | default_timezone = "Pacific/Auckland"

Yep I already tried that a few days back, nothing has changed ... you must have done something else that worked if you have correct timezone shown with "date" in the console or GUI top/right of the screen ..

How did you show the Day/Month in the clock on the screen top/right, in that way

I'm not entirely sure.

Maybe a duplicate of January's https://github.com/helloSystem/Utilities/issues/37

https://github.com/helloSystem/hello/discussions/406#discussioncomment-4245391

https://gist.github.com/mlconnor/1887156
I found this list of date formats for various countries, I hope you can understand that it is very important to have a date and time setting.
image


Event Request - OnBeforePostResJnlLineConsume - codeunit 5987 "Serv-Posting Journals Mgt.

🕮 Describe the request
Please could we have a OnBeforePostResJnlLineConsume event in codeunit 5987 "Serv-Posting Journals Mgt."
procedure PostResJnlLineConsume(var ServiceLine: Record "Service Line"; var ServShptHeader: Record "Service Shipment Header")
var
    ResJnlLine: Record "Res. Journal Line";
IsHandled: Boolean;

begin
IsHandled := false;
OnBeforePostResJnlLineConsume (ServiceLine,ServShptHeader,IsHandled);
if IsHandled then
exit;
[IntegrationEvent(false, false)]
local procedure OnBeforePostResJnlLineConsume (var ServLine: Record "Service Line"; var ServShptHeader: Record "Service Shipment Header"; var IsHandled: Boolean)
begin
end;
Additional context
This will allow us to post the usage resource journal and timesheet using another method which happens before the document is posted,

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 11525.


Tweets are garbled

Hi,
Today I opened the app in Chrome, and found that the tweets are garbled. Attached is the snapshot. Checked with other browser like Firefox, there seems fine. Was anything changed?
https://gofile.io/d/n0Nuas

Looks like this is a local issue with my browser


Use correct msg when showing delete blocks confirm dialog.

Resolves
The incorrect text "Delete all 1 blocks?" and replaces it with "Delete block".
Proposed Changes
Simply checks the number of blocks to delete and if only 1 then use another message.

@picklesrus Would you mind taking a look at this over the next week or so?

The fact that were a dialog when there was only 1 block was actually a bug. We only showed it for blocks with inputs (e.g. move (10) steps) but not for blocks without them (e.g when green flag clicked).  #1990 fixed that which should mean we never show this message when deleting a single bock.  Let me know if I'm misunderstanding though!  Thanks!


Harmonise UKB PPP data

Notes on data
Storage

UKB PPP data is fully mirrored in the Gentropy Vault.
For each study we have one TAR file which contains per-chromosome gzipped tabular files. Because of that, some additional processing will need to happen before we can feed the data into Spark.

Format

The primary columns are: CHR, POS, ALLELE0, ALLELE1, ID (encoded as CHR:POS:ALLELE0:ALLELE1:imp1)
ID is computed based on GRCh37 while POS is based on GRCh38. This means that the existing ID will field be ignored and re-assembled from other fields.

Processing approach
Scope

At this stage, we will only use the European (discovery) cohort.

Harmonisation
ALLELE0 and ALLELE1 do not necessarily correspond to REF and ALT. Hence, we adopt the following algorithm:

If CHR_POS_ALLELE0_ALELLE1 variant is present in the variant index, record it as is.
If CHR_POS_ALLELE1_ALLELE0 variant (flipped) is present in the variant index:

Flip the alleles
Allele frequency = 1 − original allele frequency
Beta = − original beta

If neither variant is present in the variant index, discard it.

Filtering
We will use hard filtering as described below, meaning that variants that don't pass the filters are fully discarded and will not be present in the output

Discard variants with INFO < 0.8
Discard variants with minor allele frequency < 1e−4
Discard variants that are not present in the variant index (see above)

After filtering, we expect to observe anywhere from 13 to 20 million SNPs
Special cases

Data contains X chromosome, it should be processed in the same way as autosomes
Data contains pseudoautosomal region of Y chromosome, designated as XY. This needs to also be processed and, if present in the variant index, recorded as chromosome Y.

CC @addramir @d0choa @DSuveges

Just double checking, when you refer to variant index, it is the full GnomAD4 variant annotation table, right? These decisions sounds solid, just confirming we are dropping variants if they are interpreted on the reverse strand? (I think this is fine choice)

After filtering, we expect to observe anywhere from 13 to 20 million SNPs

Just curious, what is the starting number?

Just double checking, when you refer to variant index, it is the full GnomAD4 variant annotation table, right?
Thank you for bringing this up, this is actually something I also wanted to confirm with @addramir. I didn't work with the variant index before, so I'd appreciate the link to which exact dataset I should use.

These decisions sounds solid, just confirming we are dropping variants if they are interpreted on the reverse strand?
I think (@addramir also might be better suited to confirm/deny this) that we expect all variants to be reported on the forward strand, the main issue is that sometimes alleles will be flipped so that ALLELE1 is reference and ALLELE0 is reference. So we don't expect a lot of cases where neither ID is found in the variant index and we have to drop it.

as an exploratory step, it would be great to see some stats on
I will definitely include the detailed statistics on how many variants dropped for which reason, not just for one study but generally in the harmonisation logs. It's going to be useful for debugging and for downstream analysis.

Just curious, what is the starting number?
I've chosen a few random studies and the results were: 15 933 877, 15 933 061, 15 952 258.

@tskir as for the gzipped files. This worked quite smoothly for the eqtl catalogue.
But those templates are based on Dataflow, and since I worked with it recently, I should be able to write a simple pipeline to do the decompression, so thank you for the pointer either way!

Three questions about populating the study index

For pQTL studies like UKB PPP, what do we put into traitFromSource / traitFromSourceMappedIds fields? Do we just not populate them? Do we put some term for “protein expression / levels”? I distinctly remember there used to be a discussion about this but I don't think we decided on anything.
For the studyId field, do I just use the original ID and the UKB_PPP prefix? The result would look like this: UKB_PPP_AARSD1_Q9BTE6_OID21311_v1.
For each study, the original metadata also contains panel information, for example “Oncology” or “Inflammation”. Do we need this information at all? If so, where do I put it? I don't think there is a field for it, so it might need to be created if we want this.

@addramir
2, 3 — thank you, got it

I thought it should be liek the name of the file or the name of the GWAS from their meta-data file without adding anything else (like "levels" or similar). What is "traitFromSourceMappedIds" btw?

The problem is that there is no trait explicitly specified. Here are all of the metadata fields for one example study:

UKBPPP_ProteinID
AARSD1:Q9BTE6:OID21311:v1

olink_target_fullname
Alanyl-tRNA editing protein Aarsd1

OlinkID
OID21311

UniProt
Q9BTE6

Assay
AARSD1

Panel
Oncology

Panel_Lot_Nr
B04412

UniProt2
Q9BTE6

HGNC.symbol
AARSD1

ensembl_id
ENSG00000266967

chr
17

gene_start
42950526

gene_end
42964498

Strand
-1

Dilution_factor
1:1

block
B

expansion
no

In theory, traitFromSource should contain the original description of a trait being measured (which it is lacking in this case), and traitFromSourceMappedIds would contain one (or many) mapped ontology IDs representing the same trait. So
for regular GWASes, traitFromSource might be Alzheimer's disease and traitFromSourceMappedIds could be [EFO:1001870].
Here, the implied trait is “expression of a (given protein)”, which I'm not sure how to correctly represent. Especially given that we already have the separate "gene" column in the study index. Since the context should be clear from the type of study (pQTL) + gene, we might just drop those fields and it would be fine.
@d0choa Do you have any thoughts on this?

I see. Well, my guess, that traitFromSource should be "olink_target_fullname" and traitFromSourceMappedIds should be the same for all: http://www.ebi.ac.uk/efo/EFO_0007937 . Don't forget we need a column that represents the target_id and links the protein and the gene (ensembl_id?), I don't know what is the name for this column in study_index.

Without going into detail, we want to be consistent with what we have for eQTL Catalogue pQTLs (Sun et al.) code.
In [14]: df.filter(f.col("studyType") == "pqtl").show(3, vertical=True, truncate = False)
-RECORD 0------------------------------------------------------------------------------------------
 studyId              | Sun_2018_plasma_HLA.DQA2.7757.5.3..1
 projectId            | Sun_2018
 studyType            | pqtl
 traitFromSource      | HLA.DQA2.7757.5.3..1
 geneId               | ENSG00000237541
 tissueFromSourceId   | UBERON_0001969
 nSamples             | 3301
 summarystatsLocation | https://ftp.ebi.ac.uk/pub/databases/spot/eQTL/sumstats/QTS000035/QTD000584
 hasSumstats          | true
-RECORD 1------------------------------------------------------------------------------------------
 studyId              | Sun_2018_plasma_TAC1.9337.43.3..1
 projectId            | Sun_2018
 studyType            | pqtl
 traitFromSource      | TAC1.9337.43.3..1
 geneId               | ENSG00000006128
 tissueFromSourceId   | UBERON_0001969
 nSamples             | 3301
 summarystatsLocation | https://ftp.ebi.ac.uk/pub/databases/spot/eQTL/sumstats/QTS000035/QTD000584
 hasSumstats          | true
-RECORD 2------------------------------------------------------------------------------------------
 studyId              | Sun_2018_plasma_IL5RA.13686.2.3..1
 projectId            | Sun_2018
 studyType            | pqtl
 traitFromSource      | IL5RA.13686.2.3..1
 geneId               | ENSG00000091181
 tissueFromSourceId   | UBERON_0001969
 nSamples             | 3301
 summarystatsLocation | https://ftp.ebi.ac.uk/pub/databases/spot/eQTL/sumstats/QTS000035/QTD000584
 hasSumstats          | true
only showing top 3 rows

The critical fields are:

studyId needs to be unique
studyType needs to be pqtl
geneId will be used for L2G

Knowing what protein isoforms (e.g. Olink ID or uniprot accession) are nice to have but are not critical.
As you can see, traitFromSourceMappedIds is not populated. That's fine because we don't use QTLs to derive Target-Disease/Phenotype evidence, only variant/locus to gene.

@d0choa @addramir Test runs revealed two further problems with the data. One is very easy and the other is quite complex, so I'll post it as a separate comment.
1. One entry from metadata table does not correspond to a summary stats file
For all ancestries (I'm now ingesting just Europeans but checked for all), all but one entities in the original metadata table correspond to a summary stats file. Likewise, all summary stats file have a corresponding metadata entity (some more than one — more on that below!)
However, the GLIPR1:P48060:OID31517:v1 entry in the metadata table does not correspond to any summary stats files. There's not much to do other than manually mark and skip it, which I will do.

2. Some summary stats correspond to a measurement of a mixture of proteins
For some summary stats files (to be exact, 15 of them) the metadata table contains more than one entry.
Example 1: α-Amylase

UKBPPP_ProteinID
olink_target_fullname
OlinkID
UniProt
Assay
Panel
Panel_Lot_Nr
UniProt2
HGNC.symbol
ensembl_id
chr
gene_start
gene_end
Strand
Dilution_factor
block
expansion

AMY1A_AMY1B_AMY1C:P0DUB6_P0DTE7_P0DTE8:OID30707:v1
Alpha-amylase 1A_Alpha-amylase 1B_Alpha-amylase 1C
OID30707
P0DUB6_P0DTE7_P0DTE8
AMY1A_AMY1B_AMY1C
Inflammation_II
B20704
P0DUB6
AMY1A
ENSG00000237763
1
103655760
103664554
1
17:40
C
yes

AMY1A_AMY1B_AMY1C:P0DUB6_P0DTE7_P0DTE8:OID30707:v1
Alpha-amylase 1A_Alpha-amylase 1B_Alpha-amylase 1C
OID30707
P0DUB6_P0DTE7_P0DTE8
AMY1A_AMY1B_AMY1C
Inflammation_II
B20704
P0DTE7
AMY1B
ENSG00000174876
1
103687415
103696454
-1
17:40
C
yes

AMY1A_AMY1B_AMY1C:P0DUB6_P0DTE7_P0DTE8:OID30707:v1
Alpha-amylase 1A_Alpha-amylase 1B_Alpha-amylase 1C
OID30707
P0DUB6_P0DTE7_P0DTE8
AMY1A_AMY1B_AMY1C
Inflammation_II
B20704
P0DTE8
AMY1C
ENSG00000187733
1
103745323
103758726
1
17:40
C
yes

My understanding is that, in the assay, they are measuring total alpha amylase, which in reality consists of three subunits coded by three genes, and so they report it this way.
Example 2: Interleukin-12
Note that, in the first example, all subunits are on the same chromosome and even roughly in the same region. This is true for all but one examples. But for Interleukin-12, its two subunits are on different chromosomes:

UKBPPP_ProteinID
olink_target_fullname
OlinkID
UniProt
Assay
Panel
Panel_Lot_Nr
UniProt2
HGNC.symbol
ensembl_id
chr
gene_start
gene_end
Strand
Dilution_factor
block
expansion

IL12A_IL12B:P29459_P29460:OID21327:v1
Interleukin-12
OID21327
P29459_P29460
IL12A_IL12B
Oncology
B04412
P29459
IL12A
ENSG00000168811
3
159988835
159996019
1
1:1
B
no

IL12A_IL12B:P29459_P29460:OID21327:v1
Interleukin-12
OID21327
P29459_P29460
IL12A_IL12B
Oncology
B04412
P29460
IL12B
ENSG00000113302
5
159314780
159330487
-1
1:1
B
no

All affected targets
Creatine kinase U-type, mitochondrial
Neutrophil defensin 1
Beta-defensin 4A
Interleukin-27
Galactoside 3_5
Interleukin-12
Galectin-7
MHC class I polypeptide-related sequence B_A
Alpha-amylase 1A_Alpha-amylase 1B_Alpha-amylase 1C
BolA-like protein 2
Choriogonadotropin subunit beta 3
Cancer/testis antigen 1
Beta-defensin 103
Beta-defensin 104
Sperm acrosome-associated protein 5
Possible solutions

Simply ignore those 15 targets for now and do not ingest
Create duplicate study index entries, so that N entries in the original metadata table = N entries in the study index table, all linking to the same summary stats file. This will be problematic because my understanding is that we have always assumed that 1 study index record = 1 summary stats file, which are linked by study ID, and this will almost certainly break something.
Cram all information into a single study index entry. This will be even more problematic because, for one, we currently assume that 1 study index entry for pQTL = 1 gene, which will no longer be the case.
For each set of “duplicated” metadata table entries, keep only one. This leads to loss of information and potential inconsistencies. We'll also need to decide how to choose which gene to keep.
Massively rework study index / summary stats structure and their relationship (just for 15 genes in one study?..)

TBH none of those solutions strike me as very good, so @d0choa @addramir I'd love to hear your opinions on this

I had a brief discussion with @DSuveges and we concluded a solution around #2 is probably the safest bet here.
The rationale is that we don't want to lose gene assignments for the main use of this data (colocalisation/MR). Having replicated signals for multiple genes producing the same gene product is probably safer than losing QTL assignments. The mistake we will be committing is that we don't know if each of these genes is the one being transcribed but, if not, it would be another gene producing the same gene product.
To ensure this doesn't break anything, the best option would be to use a non-redundant studyIndexId. In that way, you would ensure that they are treated as independent studies (even if they come from the same summary stats). That would be safe downstream, but let us know if this causes any issues with the harmonisation.

Transformation into a Spark-compatible format complete
The data for all 2 957 studies can be found here: gs://gentropy-tmp/batch/output/ukb_ppp_eur. Each now has a unique study ID. The data is pre-partitioned by study and chromosome.
Next steps
I am now working on the final action which is the actual data harmonisation pipeline. I expect to have test runs this week and some results next week.

Status update
A lot of the developments and discussions have been happening internally on this issue, so just to provide a status update:

[x] Draft harmonisation pipeline was complete 2024-04-05.
[x] Data exploration stage including AF/AF plots was complete 2024-04-10, and decision was reached for final harmonisation pipeline parameters and which variant types to ingest.
[x] Harmonised data v1 processed and delivered for all studies 2024-04-16.
[x] Harmonised data v2 processed and delivered for all studies 2024-04-18, which included a fix to p-value conversion from original data.

The variant annotation dataset was updated by @DSuveges 2024-04-18 as well. I'm currently investigating the data against the updated dataset, as well as looking into including chromosome X data in addition to autosomes.

Metrics and plots with the previous version of the variant annotation dataset
This is for comparison/baseline purposes only, I'll post the updated metrics with the new dataset version shortly.
Comments on metrics and plots

This includes all variant types on all autosomes only.
Variants are categorised according to how their alleles matched the variant annotation dataset:

Direct: A0 in UKB PPP = REF, A1 = ALT
Flip: A1 in UKB PPP = REF, A0 = ALT

Variants are also split into three types:

snp_c: SNP which replaces a nucleotide with its complement, so four types: A → T, T → A, G → C, C → G
snp_n: All other SNPs
indel: Indels, defined as variants where the length of at least one allele is > 1.

Metrics

All variants: 16 044 498
INFO ≥ 0.8: 14 496 921
FREQ ≥ 1e–4: 14 496 921 (nothing excluded because data seems to be already pre-filtered)
Variant index match: 14 354 033

Original alleles (A0 = REF): 13 896 197
Flipped alleles (A1 = REF): 457 836

Have allele frequencies in both UKB PPP and VA datasets: 14 353 489 (only these variants can be plotted on AF/AF plots below)

Plots

Decision
For the v1 and v2 versions of the harmonised UKB PPP data, as discussed with @addramir and @Daniel-Considine, the decision was to harmonise all variant types except for [flip] [snp_c], which clearly display a lack of AF concordance.

@tskir for some reason image plot dosn't work.

Metrics and plots with the new version of the variant annotation dataset
Notes

This is the version of variant annotation dataset produced by @DSuveges 2024-04-18.
Because the indel positions were fixed with this update, the harmonisation pipeline now does not add +1 to indel positions.
The data here still only covers the autosomes.

Metrics

All variants: 16 044 498 (no change, as expected)
INFO ≥ 0.8: 14 496 921 (no change, as expected)
FREQ ≥ 1e–4: 14 496 921 (no change, as expected)
Variant index match: 14 352 842 (0.01% reduction)

Original alleles (A0 = REF): 13 894 891 (0.01% reduction)
Flipped alleles (A1 = REF): 457 951 (0.02% increase)

Have allele frequencies in both UKB PPP and VA datasets: 14 352 842 (<0.01% reduction)

As we can see, the metrics are essentially unchange, which is good. This means the new harmonisation pipeline handles indels from the new variant annotation dataset well.
Plots

Conclusion
For autosomes, it looks like we are good to go with the updated variant annotation dataset with the same decision re: variant types as before.

@addramir Thank you! Fixed the plots in both comments now

Thank you! All looks good for me.

Checking up on chromosome X
UKB PPP data likes to spice things up a bit occasionally, so while in the file name the chromosome is called chrX (for example, discovery_chrX_VWF:P04275:OID20256:v1:Cardiometabolic.gz), inside the file the value for the chromosome name is actually recorded as 23. I didn't quite see this coming, so the original quickly prepared harmonised data (v1, v2) did not include chromosome X.
But this actually presented a good opportunity to plot it separately now and to verify that the allele frequencies behave in the same way as for the autosomes, which luckily they are. So for the upcoming v3 data release, I will proceed to add chrX and process it in the same way as the autosomes.

:zap: UKB PPP (EUR) harmonised data v3 is ready
List of changes compared to v2
General

Improved processing performance and output partitioning structure. This is achieved by pre-computing the variant_annotation dataset, by tweaking the number of RDDs for shuffling, and by optimising joins. The entire harmonisation pipeline now runs in <1 hour on ~70 (average number) 16-core VMs, and costs just ~$10 per run in compute time.

Study index

Corrected sample size. It's now computed from peeking into the first row of every corresponding original summary stats file.
Added population structure. Assuming everyone is European (NFE).

Summary stats

Using the updated variant annotation dataset. Correspondingly, not adding 1 to indel positions anymore.
Added chromosome X data. It is recorded as “chromosome 23” at source.
Added sample size. This is simply the N column from the original data.
Proper p-value calculation. Now using neglog_pvalue_to_mantissa_and_exponent on the LOG10P column.

Metrics for the sample study
As for all previous metrics, I'm using UKB_PPP_EUR_A1BG_P04217_OID30771_v1.

All variants: 16 044 498
INFO ≥ 0.8: 14 496 921
FREQ ≥ 1e–4: 14 496 921
Variant index match: 14 873 176

Original alleles (A0 = REF): 14 391 059
Flipped alleles (A1 = REF): 482 117

Kept after filtering out low quality variants (snp_c in the flipped orientation): 14 867 681

The first three metrics are unchanged because previously, chrX variants were present at these stages and were counted, but they were labeled as chromosome 23, so they didn't go into subsequent stages.

:zap: Clumping & locus collection is done
Results

Clumped data: gs://ukb_ppp_eur_data/clumped, total 20 814 records
Collected loci: gs://ukb_ppp_eur_data/collected, total 18 524 records

Code and parameters used
Import
from gentropy.common.session import Session
from gentropy.method.window_based_clumping import WindowBasedClumping
from gentropy.dataset.summary_statistics import SummaryStatistics
from pyspark.sql.functions import col
from gentropy.dataset.study_locus import StudyLocus

Set up
session = Session()

Clump
%%time
path_sumstats="gs://ukb_ppp_eur_data/summary_stats"
L=SummaryStatistics.from_parquet(session=session,path=path_sumstats)
slt=WindowBasedClumping.clump(L,gwas_significance=1.7e-11,distance=5e5)
slt.df.write.parquet("gs://ukb_ppp_eur_data/clumped")

Collect
%%time
sl=StudyLocus.from_parquet(session=session,path="gs://ukb_ppp_eur_data/clumped")
df=sl.df
df = df.filter(~((col("chromosome") == "6") & (col("position").between(25000000, 34000000))))
df.count()
sl.df=df
wbc=sl.annotate_locus_statistics(summary_statistics=L,collect_locus_distance=1000000)
wbc.df.write.parquet("gs://ukb_ppp_eur_data/collected")

Overview: full UKB PPP (EUR) ingestion & harmonisation code
I've now finished uploading all code that I have related to the UKB PPP (EUR) datasource. Currently, three stages are required for the entire processing pipeline.
1. Mirror

Input. Original data is hosted on Synapse.
Transformation.

As we decided in the past, we want to keep the copy of the original data in the Vault.
Protocol is available here: https://github.com/opentargets/gentropy-vault/blob/main/datasets/ukb-ppp.md.
The protocol must be run manually.

Output. The output of this step is kept forever in the Vault.

2. Preprocess

Input. The mirrored data from the previous step.
Transformation.

The data which we mirrored during the previous steps has to undergo several specific transformations which aren't achievable in Spark (especially the first one):

Extract gzipped per-chromosome files from inside the individual TAR files, uncompress, partition by chromosome
Recreate the study ID. This is required because multiple rows in the study index can reuse the same summary stats file
Drop certain rows which don't have a corresponding summary stats file

This transformation is done using Google Batch. The code can be found in this new repository: https://github.com/opentargets/gentropy-input-support. The UKB PPP-specific part is this one: https://github.com/opentargets/gentropy-input-support/blob/dc5f8f7aee13a066933f3fd5b18a9b3a5ca71069/data_sources.py#L43-L103.
This step must also be triggered manually, how to do this is described in the repository.

Output. Precursors of study index and summary stats datasets are output. Because we decided that we don't want to keep the data twice, the output of this step is only kept temporarily and is deleted after 60 days according to the gs://gentropy-tmp bucket lifecycle rules.

3. Ingest & harmonise

Input. The preprocessed data from the previous step.
Transformation.

The data is ingested according to the usual study index & summary stats schemas.
The variant positions, alleles, and allele frequencies are also harmonised according to all of the discussions which happened during this issue.
The code is available in this pull request: https://github.com/opentargets/gentropy/pull/652. Once this PR is closed, that concludes this issue.

Output. Study index and harmonised summary stats, kept forever as usual.


Support Arithmetic function CEILING

Backgrounds

Kotlin JDSL does not yet support an arithmetic function CEILING.

Objects

Support an arithmetic function CEILING.

Requirements

Creates ceiling() function in com.linecorp.kotlinjdsl.dsl.jpql.Jpql for supporting an arithmetic function CEILING.

Contribution

Please check if there is an assignee in this issue. And if it don't have an assignee, please leave comment to assign.
Please check the commit convention through the commitlint.config.js file. The commit  convention also described in the PR template.
When you use intellij, please set the code style through the .editorconfig file.
Please create the PR for develop as the base branch.
If you have any questions, please leave comment in Kotlin-JDSL Discord Channel in LINE Open Source

Can I contribute to this issue?

Hi esperar!
I assigned this issue to you. Thank you for your help!


How does the scroll-on-drag work?

I'm trying to implement both dragula and dnd in a page. While the scroll on drag is working with dnd elements, the same is not working for dragula elements. How does it work?
EDIT : There seems to be an issue with dragula messing with the default scrolling behaviour of the browser, and I thought dnd had a custom implementation. Closing.

Hi @paw1 did it work for you? If so how did you fix it?

dom-autoscroller


Added search bar to Asset Loader

Implemented QLineEdit for searches.
Text changed trigger proxy.setFilterRegExp
Proxy set to CaseInsensitive

This one actually contains #251, should we close one of them?

Close the other, because I believe this also has fixes for problems that the other one introduced. :)

@aardschok could you check the above comments made here and after changing test whether the functionality is all there? :) Then we should be safe to merge this.

@BigRoy Everything works with said feedback implemented

Would it be possible to make a small gif showcasing this new functionality? Ideally I'd like for all visual changes to come with a gif or still image.

Results:

Great! That'll make our future selves and any followers happy. :)
I try and do that to encourage comments without actually having to (1) clone, (2) checkout your PR branch, (3) set it up locally and (4) find a way to run it. A simple picture takes all that away.

Great work @aardschok!


Theme the frontend to resemble australis

Fixes https://github.com/mozilla/tofino/issues/722

Addressed comments.


Do I have to use firebase?

Apologies for what I suspect is a fairly basic question...  One of the steps listed for this project is to create a firebase configuration.
I might have missed it over the last while, but is this as a result of Google favouring Firebase as its primary mechanism of delivery for on-device APIs & services?
(I'm not really opposed to using Firebase, although I'm wondering what the minimal paths are and am also curious about overlap between Visual Studio App Center, Facebook and Firebase for things like analytics.)
If Firebase is a requirement, are there any comparisons between this library and react-native-firebase worth making clear?

For Google OAuth 2.0 authentication you need firebase configuration.
react-native-firebase is not comparable with this library.
react-native-firebase suggests to use this library for Google OAuth 2.0 authentication.
docs here.

@atrauzzi this library does not depend on firebase, but to get the config file, we documented how to do it via firebase since it's fairly straightforward and many use it anyways. I don't think you need a firebase project though - I think you can just create everything in the google cloud console.

That's interesting. I wonder if it's worth making this clear in the docs or documenting both methods would help avoid potential confusion.  Again, I'm only asking this because based on the instructions, I was lead to believe that I needed Firebase in some way.
If I wasn't told to go to Firebase, I'd have never thought about it.

@atrauzzi Agreed! From my experience following the docs, you actually need to go to the cloud console for other portions (getting the web client ID, configuring the OAuth consent screen), so we should update the docs with the simplest path. @vonovak is working on documentation now, so we can probably clean this up.

I totally agree with @atrauzzi, it was confusing for me as well when I started using the library

I just went to google cloud console and created my own credentials for my project

I followed the instructions

and then downloaded the json configs

still could not get this to work

@fabriziobertoglio1987 sorry to bump an old thread, but did you ever figure out a simple process?
Is it still a case of creating the google-services.json manually from credentials supplied when you create an app in the Cloud Platform?

https://pub.flutter-io.cn/packages/google_sign_in this plugin for flutter doesnt not say to include google-services.json for google signin ... the only docs for android is
To access Google Sign-In, you'll need to make sure to register your application.

You don't need to include the google-services.json file in your app unless you are using Google services that require it. You do need to enable the OAuth APIs that you want, using the Google Cloud Platform API manager. For example, if you want to mimic the behavior of the Google Sign-In sample app, you'll need to enable the Google People API.

How is this being done without firebase?

this library is not compatible with react-native 0.60.6

@fabriziobertoglio1987 I noticed your comments around the repo; please do not make unsubstantiated claims about the library not working

this library (@react-native-community) is not compatible with react-native 0.60.6

Do you have any proof for that? (a runnable reproduction of the issue)? Looking at the issues, I don't see one.

also the library does not correctly uninstall itself and causes issues. I had to remove two lines from my AppDelegate.m

Any react native library itself does not have a way to remove all of its code from the project.

The latest release is 2.1.1, but the readme says that the library is compatible with react native > 0.60

Please take a better look at the readme, it clearly states the difference between v2 and v3.
Thank you.

@vonovak Thanks for your reply
there is no need to open an issue. There is already an open issue
https://github.com/react-native-community/react-native-google-signin/issues/755

@vonovak  Version 3.0 does not work, that is why you made 2.1.1 the latest release.
You should reopen this and all unsolved issues, as version 2.1.1 does not work with react native < 0.60 (I tested it personally)
https://github.com/react-native-community/react-native-google-signin/issues/755

Is there any guide to help me to achieve this without firebase?

@Uttu316 no one has created such guide yet, but you can be that person! :) I think what you need is just create the client IDs in the cloud console and then provide them to the configure() call.

If it is mentioned in the docs that we can use this plugin without firebase, there should be clear instructions on how to accomplish that otherwise you should remove the option that using this library without firebase is possible.
It keeps confusing the developers who are integrating the plugin without firebase.

@Uttu316 no one has created such guide yet, but you can be that person! :) I think what you need is just create the client IDs in the cloud console and then provide them to the configure() call.
this might help too #883 (comment)

the option that you have mentioned does not work at all.

May I ask, why does some people try and implement this sign-in stuff without Firebase? Any drawbacks at sight? I always thought that to use Firebase is the way we are supposed to do to get this thing right...
Anyway, thanks in advance,
Fabricio

@FabricioGanzert

why does some people try and implement this sign-in stuff without Firebase

in my case, I don't wanna pay for firebase authentication to manage auth user data, but I wanna implement google sign-in with my own backend.

Still looking for a solution here.

@KManiKumarReddy Seemingly, that person who had answered my question removed his answer - Which means it might o longer a valid answer.
As you, I'm still looking for a working solution.

hello, documentation for non-firebase users was added here: https://github.com/react-native-google-signin/google-signin/commit/b7a70b8de054a504e7a7806db491a19d9d543f64
I'm going to close this as resolved, please open a PR or issue if you find those instructions are not correct :)
Thank you!


Update nf-winuser-getrawinputdevicelist.md

Update example code. Call GetRawInputDeviceList in a loop to properly handle device list change between calls to GetRawInputDeviceList.

@DJm00n : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Jest mocks calls are not recording object parameters as copies, but references, which results in bad assertions, when the object mutates later.

#10373  🐛 Bug Report
When asserting multiple calls to the same mock function using objects as parameters to the mock function, Jest will not record the values of the object as they were at the moment of the call, but only the reference to the object.
When tested code mutates the object after the call, it will also mutate the recorded calls within Jest, resulting in failing tests, that should be green.
To Reproduce
Using Jest 26.2.2, consider this test:
let doingSomething = jest.fn();

function myFunction() {
    let myParam = { myField: 'testValue' };
    doingSomething(myParam);
    myParam.myField = 'differentValue';
    doingSomething(myParam);
}

describe('something', () => {
    test('should work',  () => {
        myFunction();
        expect(doingSomething).toBeCalledTimes(2);

        // this assert will fail, because myField == 'differentValue', even though it shouldn't:
        expect(doingSomething).toHaveBeenNthCalledWith(1, { myField: 'testValue' });

        // this is okay:
        expect(doingSomething).toHaveBeenNthCalledWith(2, { myField: 'differentValue' });
    });
});

Expected behavior
Both expects should be green, since the object parameter was correct at the time of the first call.

Hello @marcelb
I don't think it's a but. This is how javascript programing language works with object and other mutable objects.
Here you can find other approach to the same problem: https://codesandbox.io/s/practical-water-26cf0x?file=/main.test.js
To better understand this problem you can imagine what is happening in similar pure js example:

// VERY simple ala jest.fn
function makeMockFn () {
  const callHistory = []

  const mockedFn = (...args) => {
    callHistory.push(args)
  }

  // helper to get function call history
  mockedFn.getCallHistory = () => callHistory

  return mockedFn
}

// creating ala jest.fn()
const fn = makeMockFn()

// create params to pass to fn
const myParams = { a: 1, b: 2 }

// running fn
fn(myParams)
fn('string')
fn(123456)

// check what we have in call history
console.log(fn.getCallHistory())

// lets mutate myParams
myParams.a = 'CHANGED a key'

// try to run fn
fn(myParams)

// check what happen with first and last position of history array
console.log(fn.getCallHistory())


Running date operations on several inputs

This seems like something I should be able to do, but I can't figure out how to get it to work.
I am trying to do date comparisons between different properties in the input and set an output as a result. I've tried referencing properties by name, and wrapping in a date() function, but can't get any output. I'm also not able to confirm from the trace whether my inputs were parsed at all.
I'm using gorules.io editor in a project to test all this.

Hi @psamaan, could you try using:
date(property1.date)

and in some cases
> date(property2.date)

Yup, that worked. I must have only wrapped the rows in date() and not the column selectors. Thanks!


Fix migration classname

Fixes bug in migration class name which cause db:migrate to fail with NameError looking for CreateResearchDomains.
Changes proposed in this PR:

change class name from CreateFos to CreateResearchDomains

No - not master - developoment - AAAhhhhhh