Possible to support displaying all authors on PyPI page?

What's the problem this feature will solve?

:wave: Hi. Our project pyhf just switched (c.f. https://github.com/scikit-hep/pyhf/pull/2095) from having our PyPI metadata in setup.cfg to pyproject.toml. In doing so, we also changed from having our author metadata for the 3 authors be across author and author_email to having it be contained in authors following PEP 621's requirements of

These fields accept an array of tables with 2 keys: name and email. Both values must be strings. The name value MUST be a valid email name (i.e. whatever can be put as a name, before an email, in RFC 822) and not contain commas. The email value MUST be a valid email address. Both keys are optional.

pip is recognizing all the metadata as we would expect
$ python -m pip show pyhf
Name: pyhf
Version: 0.7.1.dev43
Summary: pure-Python HistFactory implementation with tensors and autodiff
Home-page:
Author:
Author-email: Lukas Heinrich<EMAIL_ADDRESS>Matthew Feickert<EMAIL_ADDRESS>Giordon Stark<EMAIL_ADDRESS>License: Apache-2.0
Location: /home/feickert/.pyenv/versions/3.10.6/envs/pyhf-dev-CPU/lib/python3.10/site-packages
Requires: click, jsonpatch, jsonschema, numpy, pyyaml, scipy, tqdm
Required-by:

However, when we published this to TestPyPI to check how things looked after switching over we noticed that TestPyPI is displaying only the first author and linking their email

Previously when we shoved all our names and emails into author and author_email we could at least have all our names be displayed (no surprise there as we were abusing the field)

I assume that this behavior with authors is because warehouse uses only the core metadata here (?) following PEP 621's instructions of:

Using the data to fill in core metadata is as follows:

If only name is provided, the value goes in Author/Maintainer as appropriate.
If only email is provided, the value goes in Author-email/Maintainer-email as appropriate.
If both email and name are provided, the value goes in Author-email/Maintainer-email as appropriate, with the format {name} <{email}> (with appropriate quoting, e.g. using email.headerregistry.Address).
Multiple values should be separated by commas.

Would it be possible for warehouse to display all authors information if it exists? Or is that something that is outside the scope of how warehouse interacts with metadata?
Describe the solution you'd like

Have warehouse be able to parse the existence of PEP 621 authors and display all names and associated emails of authors on the package webpage.
Additional context

Is this a duplicate of #9400?

Is this a duplicate of #9400?

Hm I think so. I'll assume yes and close here in favor of it, but actually check when I'm out of my next meeting.
Apologies for the noise: I searched "authors" in the Issues and so missed this as I didn't check all the ones just matching "author".


Add DuckDuckGo crawlers to bot list

See github issue: https://github.com/WhichBrowser/Parser-PHP/issues/568

DuckDuckGo Https Bot
Mozilla/5.0 (compatible; DuckDuckBot-Https/1.1; https://duckduckgo.com/duckduckbot)

DuckDuckGo Bot
DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)

DuckDuckGo Favicons Bot
Mozilla/5.0 (compatible; DuckDuckGo-Favicons-Bot/1.0; +http://duckduckgo.com)

IP Address Rules:
https://help.duckduckgo.com/duckduckgo-help-pages/results/duckduckbot/

Coverage increased (+0.02%) to 99.97% when pulling 16fe39a48cbcb7a751240504b7eea2ec5d005c2c on ayumi-cloud:DuckDuckGo into 880b9fa797401d14b28956442944c3daa70240ff on WhichBrowser:master.

Fixed by d5ad2cedb037fb0df31e7a8512564539a6c83fb9

Fixed by d5ad2cedb037fb0df31e7a8512564539a6c83fb9


Testing xen-arm-builder automatically

I imagine that the current way that PRs for this project are tested is:

Verify that the Travis build was successful.
Manually clone the PR to a developer's PC.
Start the Ubuntu-based privileged Docker container (possibly rebuild it first).
Manually start the build, e.g. make clone && make build && make cubieboard2.img
When it eventually finishes, exit the Docker container.
Physically insert a microSD card into the PC and "dd" the image to it.
Unplug the microSD card and plug it into a powered down cubieboard2.
Optionally connect a serial cable to the console port.
Power on the board. I assume that it is already connected to the LAN.
If it boots OK then ssh cubieboard2.local (fiddling with known_hosts).
Run some kind of manual test as a sanity check, like building and running a unikernel.
Possibly repeat all steps for BOARD=cubietruck.

My proposal for automating this would be:

A physical machine is designated as a build slave and is permanently connected to the serial console of a cubieboard2 and/or a cubietruck.
A device allows the power to the board to be controlled by software.
Automatically clone the PR, start the build container, build the image, and stop the container.
Automatically power off the device, power it on, and simulate a keypress on the serial console to interrupt U-Boot.
Send commands to U-Boot to download the built kernel, an initial ramdisk, and the built dtb to the board (in RAM only) via TFTP. No Xen yet...
Boot the downloaded image into the initramfs.
If everything worked OK, run tftp -g -r cubieboard2.img -l - 192.168.x.y | dd of=/dev/mmcblk0 bs=4096 (or equivalent) via the serial console.
After it completes, power the board off and on and verify that the complete image has booted.
Run some tests over SSH.

But do you consider xen-arm-builder to be important enough to warrant setting this up?
I've done some experiments including:

Using a second cubieboard2 as the build machine, since it has suitable UARTs.
-- You would know that the normal USB-to-serial device supplied with the cubieboard2 is not suitable for automation since it interferes with the ability to power off the board. That was why I tried out a second board initially.
-- Builds are slow of course, but that could be done elsewhere, maybe even on Travis.
Using one of these cheap wifi-controllable power plugs to control the power: http://hackaday.com/2014/11/13/hacking-a-20-wifi-smart-plug/ (configuring it can be painful but after that it's cool).
Automation of most of the U-Boot commands can be done with pyboot
-- Depends on conmux.

So, is this worthwhile?

Here's a log of the build: https://jenkins-lukedunstan.rhcloud.com/job/xen-arm-builder_external/1/console
Here's a log of a script that writes the image to the sdcard and then boots it: https://jenkins-lukedunstan.rhcloud.com/job/load-image_external/4/console
(Jenkins is only used as a place to store log files)

Hi @infidel; As it happens, after the refresh we're starting to look at automating more of the build infrastructure to support better testing, including looking again at some of our local machine room infrastructure. Is this something you managed to push any further on in the meantime? It would be great to get something setup to test like this!

I didn't spend much more time on it but it did work. Some shell scripts are at https://github.com/infidel/cubiebuild (jenkins-ssh.sh is the entry point) but it could potentially be much simpler, because the manner in which I used Jenkins is not normal, and the reason for that is because I was using a cubieboard2 as the build machine, installing the resulting image on a second cubieboard2.
It would be simpler to just have a proper server do the build and then have a serial device server (aka terminal server) to control the serial console of the target board, plus another network relay device to control the target power.
The basic concept is the same as what I did though: build an image file, power off the target board, power it on, send serial console commands to U-Boot to load the newly built kernel into RAM, then use tftp ... | dd ... to download the full image to SD card, then power cycle to boot it. The main missing piece is to run tests to verify that the new image is fully working.


Upcoming maintenance on bots.reinhart1010.id

We will be performing a software update for our server at November 22, 2022 16:00-17:00 UTC.

Resolved: BOTS Misskey Instance is back up in de2a1cb.


{ Silly question} VirtualDOM compability with Elmish

Sorry for putting this probably obvious topic here in the issues:
Is it possible to see preact as a drop in replacement for react?
Since its API is the same?
Can I simply use Preact in Elmish, since its ported to React already?
Thanks :D

In most projects preact can be used as a drop in replacement for react. In case some 3rd party library needs react-specific things we have a compat layer called preact-compat that can be used in conjunction. Note that the latest react has made some API changes in 16.3. though, but most libraries haven't switched yet.
I'm not sure how Elmish works under the hood. It looks really interesting! I don't have any experience with F#, perhaps someone else can chime in here?

You can indeed use preact, inferno or any library compatible with react using Elmish and modifying only your webpack config.
See their migration guide: https://preactjs.com/guide/switching-to-preact
{
  "resolve": {
    "alias": {
      "react": "preact-compat",
      "react-dom": "preact-compat"
    }
  }
}


cool

just checking it out now but I think you guys have a winner.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4a36d9da-7510-64b5-f937-cc3d9f0c2325
Version Independent ID: 43badf5d-f042-1bad-6e50-7a42100dbada
Content: Windows UI library - Windows UWP applications
Content Source: winrt-related-src/toolkits/winui/index.md
Product: uwp
GitHub Login: @mijacobs
Microsoft Alias: mijacobs

Thanks for the vote of confidence! :)


3.1.2 Create API (?) to create chat

Transferred to issue #73

This has been transferred to issue #73


[Bug] better discord not appear in the discord settings Mac

when i install better discord, then i open discord and it doesnt appear the settings for better discord

Starting Install...
Creating required directories...
✅ Directory exists: /Users/rolo/Library/Application Support/BetterDiscord
✅ Directory exists: /Users/rolo/Library/Application Support/BetterDiscord/data
✅ Directory exists: /Users/rolo/Library/Application Support/BetterDiscord/themes
✅ Directory exists: /Users/rolo/Library/Application Support/BetterDiscord/plugins
✅ Directories created
Downloading asar file
✅ Package downloaded
Injecting shims...
Injecting into: /Applications/Discord.app/Contents/Resources
✅ Injection successful
✅ Shims injected
Restarting Discord...

Fully close Discord manually and relaunch it.

i tried twice but nothing happens

Fully close Discord manually and relaunch it.

i tried twice but nothing happens

Please reinstall

Please reinstall

i tried too but nothing happen

Please show some more info if you'd like this debugged. Things like the console logs, versions, etc.

Is this issue still valid with the latest version?

Yes, this is still valid with the latest version of BetterDiscord. It's happening to me too. I tried using BandagedBD, but that only made me unable to use my microphone, and made me unable to screenshare.

I've also tried reinstalling BetterDiscord, reinstalling BandagedBD, Repairing Betterdiscord, Repairing BandagedBD, running discord as an administrator, logging out and back into my account, and restarting discord, all of which didn't fix the problem. I had to uninstall BandagedBD to get my microphone and screenshare working again.

reinstalling BandagedBD

BandagedBD is long dead, any use of that installer now can drastically mess up your install


Removed regexp for socket.io check

Actually this check is still not fully cover socket.io usage. I mean /socket.io prefix could be redefined.

Actually this check is still not fully cover socket.io usage.

@nkt Not quite sure what you're trying to do here -- mind elaborating a bit?

@yunong I've replaced regexp-driven check to indexOf check, which is faster.

The perf benefits are negligible, and using the regex is a bit clearer in terms of documentation.


could dependency on onnxruntime be avoided?

I'm working on a port of NeuralNote to visionOS and hitting some trouble getting onnxruntime to compile for visionOS.
Comment here says that Features is doing a CQT (what does "harmonically stack" mean?). I was wondering if other CQT code could be used instead of the onnx model? For example: https://github.com/cannam/constant-q-cpp

Hi,
Yes the ONNXRuntime dependency could be avoided, but getting the CQT working exactly the same in another framework/in pure C++ might be a bit of work.
Concerning this library, I don't know if we could get exactly the same thing, but with a few tweaks probably yes.
Harmonic stacking is kind of a reshape operation of the CQT array so that bins with frequency being a multiple of the same fundamental frequency stacks on a new dimension. You can look on the BasicPitch repo here to see how it's implemented.
If you manage to get the features working without ONNXRuntime, please let me know, I'd be pretty happy to integrate it.


MarketPane.shouldComponentUpdate returns undefined on account page

When I go to the account page on the demo, this warning appears thousands of times in chrome's console:
[app.js:106707] Warning: MarketPane.shouldComponentUpdate(): Returned undefined instead of a boolean value. Make sure to return true or false.
1718 console messages are not shown.

Waiting on my local chain to sync so I can see if this is happening on the downloaded client; will update this issue once it's caught up.

Confirmed that this happens on localhost, as well.

If #111 doesn't fix it, I'll dive deeper when I get back to my dev machine.


Feature/ccloud rest client

===========WIP================ DO NOT MERGE !!!!
Implement a java native ccloud client that talks directly to the ccloud api, so you dont have to have the CLI tool manually installed, and manually/automate outside the tool login before hand.
==============================
As well:
Refactor so that ccloud client is shared between ccloud acl and ccloud principle provider .
Make PricipleProvider configurable in same way as other providers so its aligned .
Remove old solution for principle translation, to clean things up.

note this is built on top of the work done in https://github.com/kafka-ops/kafka-topology-builder/pull/188

note this is built on top of the work done in https://github.com/kafka-ops/kafka-topology-builder/pull/188

Hi. @michaelandrepearce can you please create an issue for this change? thanks a lot.

Hi. @michaelandrepearce can you please create an issue for this change? thanks a lot.

sure will do, kinda needs the other PR merged first anyhow.

sure will do, kinda needs the other PR merged first anyhow.

https://github.com/kafka-ops/kafka-topology-builder/issues/191

https://github.com/kafka-ops/kafka-topology-builder/issues/191

@purbon from using this, it seems all stable and working nicely, only trouble i have is how do we want to go about adding tests for this as there is no free tier with no credit card option, so no way of doing some testing against the real endpoint in an open source way.
Options i can offer:

Simply have to manually smoke test
You have some contacts to get an account that could be used with credentials that can run a tests against
We simply mock the rest calls and check the mocks being called, but it wont validate anything in reality.

Im guessing option 1 is most pragmatic in the circumstances.

@purbon from using this, it seems all stable and working nicely, only trouble i have is how do we want to go about adding tests for this as there is no free tier with no credit card option, so no way of doing some testing against the real endpoint in an open source way.
Options i can offer:

Simply have to manually smoke test
You have some contacts to get an account that could be used with credentials that can run a tests against
We simply mock the rest calls and check the mocks being called, but it wont validate anything in reality.

Im guessing option 1 is most pragmatic in the circumstances.

@purbon ill rebase this and complete it off in next day or two so hopefully it can make 1.5 too.  You ok with what i raised above and my thoughts on how to handle?

@purbon ill rebase this and complete it off in next day or two so hopefully it can make 1.5 too.  You ok with what i raised above and my thoughts on how to handle?

happy to make a 1.5.1 :)
On Sat, 16 Jan 2021, 19:37 Michael André Pearce<EMAIL_ADDRESS>wrote:

@purbon https://github.com/purbon ill rebase this and complete it off
in next day or two so hopefully it can make 1.5 too. You ok with what i
raised above and my thoughts on how to handle?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kafka-ops/kafka-topology-builder/pull/190#issuecomment-761613155,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAQXPBA7AHBYR4EEXC6Z63S2HMHJANCNFSM4VYNARVQ
.

happy to make a 1.5.1 :)
On Sat, 16 Jan 2021, 19:37 Michael André Pearce<EMAIL_ADDRESS>wrote:

@purbon https://github.com/purbon ill rebase this and complete it off
in next day or two so hopefully it can make 1.5 too. You ok with what i
raised above and my thoughts on how to handle?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kafka-ops/kafka-topology-builder/pull/190#issuecomment-761613155,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAQXPBA7AHBYR4EEXC6Z63S2HMHJANCNFSM4VYNARVQ
.

Thoughts about the options you propose.

Simply have to manually smoke test

What do you mean?  if what you mean is small and quick running integration tests vs a confluent cloud instance, in here we still have the problem of getting an instance to do it, or basically allow only to run this tests with anyone who got credentials for it.

You have some contacts to get an account that could be used with credentials that can run a tests against

That would be a great option, but not sure. Let me work some magic.
Might be we can have the integration tests somehow ready, but flag out, somehow like the RBAC is right now, so it can only be run if proper credentials are there.

We simply mock the rest calls and check the mocks being called, but it won't validate anything in reality.

Agree, not sure this is even a good strategy for this component as the logic (in KTB) is kinda minimal.
So, thinking about this I am thinking we can get first the smoke tests, we could second get the integration tests, but for this, we need to sort out access (leave this with me).

Thoughts about the options you propose.

Simply have to manually smoke test

What do you mean?  if what you mean is small and quick running integration tests vs a confluent cloud instance, in here we still have the problem of getting an instance to do it, or basically allow only to run this tests with anyone who got credentials for it.

You have some contacts to get an account that could be used with credentials that can run a tests against

That would be a great option, but not sure. Let me work some magic.
Might be we can have the integration tests somehow ready, but flag out, somehow like the RBAC is right now, so it can only be run if proper credentials are there.

We simply mock the rest calls and check the mocks being called, but it won't validate anything in reality.

Agree, not sure this is even a good strategy for this component as the logic (in KTB) is kinda minimal.
So, thinking about this I am thinking we can get first the smoke tests, we could second get the integration tests, but for this, we need to sort out access (leave this with me).

it would be lovely to land this before I cut https://github.com/kafka-ops/kafka-topology-builder/issues/195, what do you think?

it would be lovely to land this before I cut https://github.com/kafka-ops/kafka-topology-builder/issues/195, what do you think?


bahaviour of params with dict and list as values

When sending a request as a client I expected query params to be expanded as follows:
In [32]: _encode_params({"aigentk": ["TEST_ONE", "TEST_TWO"]})
Out[32]: 'aigentk=TEST_ONE&aigentk=TEST_TWO'

However aiohttp does the following:
In [35]: update_path({"aigentk": ["TEST_ONE", "TEST_TWO"]})
Out[35]: aigentk=%5B%27TEST_ONE%27%2C+%27TEST_TWO%27%5D

So it encoded the list as an actual list.
If I a use a list of tuples it's working like I expected:
In [33]: update_path([("aigentk", "TEST_ONE"), ("aigentk", "TEST_TWO")])
Out[33]: aigentk=TEST_ONE&aigentk=TEST_TWO'

Is this a design choice or an inconsistency?
Thanks.

Hi @jgelens,
It is so by design.
http://aiohttp.readthedocs.io/en/stable/client.html#passing-parameters-in-urls
MultiDict has the same behavior also.

ok thanks! I will update my code. I encountered this issue when porting an application which was using the requests library.


Fix compatible

Pull Request Test Coverage Report for Build 89

27 of 37 (72.97%)  changed or added relevant lines in 5 files are covered.
2 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.4%) to 20.134%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

core/src/main/java/com/alipay/sofa/common/log/factory/LogbackLoggerSpaceFactory.java
4
5
80.0%

core/src/main/java/com/alipay/sofa/common/utils/ReportUtil.java
0
2
0.0%

core/src/main/java/com/alipay/sofa/common/log/factory/Log4j2LoggerSpaceFactory.java
8
11
72.73%

core/src/main/java/com/alipay/sofa/common/log/env/LogEnvUtils.java
3
7
42.86%

Files with Coverage Reduction
New Missed Lines
%

core/src/main/java/com/alipay/sofa/common/log/env/LogEnvUtils.java
1
65.88%

log-sofa-boot-starter/src/main/java/com/alipay/sofa/common/boot/logging/initializer/DefaultLogbackReInitializer.java
1
86.15%

Totals

Change from base Build 86:
0.4%

Covered Lines:
873

Relevant Lines:
4336

💛  - Coveralls


Create the skeleton using design file

Add the following text from design file
Create a button to direct back to login screen


Update deps, adapt to newer versions, fix clippy

I am not sure whether this project, in its current state, accepts PRs, but the idea behind it is very nice and since I just used it for some testing I want to contribute some updates back upstream.

I am also very curious what you found it useful for!

I need a virtual kubelet and wanted to use krustlet due to me and the Rust language usually being in more agreement than with Go. I also needed to update the dependencies there and make it buildable with newer rust/cargo since that project is not actively maintained.
I understand the possible issues, however I'm just using these to do a proof of concept. Moreover the interesting functionality will be in another binary/repository so I'm using krustlet to really simply fake that pods are running. Feel free to check out my repository of krustlet but bear in mind that the code is very rough, the commits are not descriptive, and so on.
I am unable to send a proper PR to krustlet because in order to update it properly I would have to rewrite the WASM handling due to the need to switch to another backend. In my branch I removed the wasi part completely.
What I might use krator for later is handling of the state machine of my CRD objects, unless I have the need for finalizers and other things that are, to the best of my knowledge, not handled in krator.
On the other hand, to be honest, if the other idea ends with the PoC, or if I need a current, stable, and updated stack I might not use krator (or krustlet) at all in the end.  I just figured I might as well polish the update to krator and contribute it back, just in case it helps someone else or someone wants to get back to it at some point =)

Looks like the failures are all from features not being enabled on the openapi crate

The documentation tests fail because markdownlint finds issues in target/doc/static.files/SourceSerif4-LICENSE-3bb119e13b1258b7.md which would not fail before building since that does not exist there.
Build tests fail because the tests do not enable any of the 'v1_*' features of k8s-openapi and the test runs cargo build which will not use dev-dependencies from Cargo.toml.
I have an idea how to fix it, but I'm not sure if it's the best way.

@nertpinx Feel free to try whatever feels best!


Update mod-standard

@modulus/standard is currently pulling the latest version, which breaks tests. Need to update mod-standard, set it to a version in package.json, and fix linting errors

Best solution was to set it to an older version so that there wouldn't have to be a major refractor.


Stop cfunc/ccall/class applying to inner funcs/classes

Fixes https://github.com/cython/cython/issues/4092
Inner classes isn't tested because they don't officially work
at this stage (see https://github.com/cython/cython/issues/4572).
It should be tested if/when they're fully supported

I've had a fairly thorough rewrite to move the work to InterpretCompilerDirectives. I think they have to be processed there because that's where all the directives are initially set up.
I've created a list of directives that I believe shouldn't be applied to nested functions/classes when used as a decorator and created a separate CompilerDirectivesNode for the body that excludes those.
One change is that this only applies to decorators, and with statements are kept all the way down. I'm not sure whether that's a good idea or not be it's more natural to do this way

I personally haven't seen with used much in the wild when it comes to defining these specific directives. It seems more natural to use them as decorators, which is the situation that this PR aims to improve.
That's both good (because it's less likely to break existing code) and bad (because it's less likely that we get helpful feedback to take an informed decision). But let's see what happens between now and 3.0 final.

Thanks!


Examples "missing script: start"

When I try to npm start to checkout the examples I get the following error:
npm ERR! Darwin 14.5.0
npm ERR! argv "node" "/usr/local/bin/npm" "start"
npm ERR! node v0.12.7
npm ERR! npm  v2.12.1
npm ERR! missing script: start

You can launch the dev server with examples by running npm run examples. Also always you can check out the list of available scripts in package.json.

Yes, realized that after I posed the issue. Someone may want to update the documentation.


nil error in HTML writer (allocation)

Got an error in the HTML writer in the allocation section. Looks like it might be choking on non-existent contact ids? Probably should capture those errors. This is a different use of contact ids outside of the responsible party - probably why the error isn't trapped.
.rvm/gems/ruby-2.3.3/gems/adiwg-mdtranslator-2.3.0/lib/adiwg/mdtranslator/writers/html/sections/html_allocation.rb:51:in `+': no implicit conversion of nil into String (TypeError)

I added a trap for instances where the source or recipient IDs are not found in the contact array.  The HTML how displays 'Contact {...} not found!'
In general, this level of testing is not done in the HTML writer.  It's too difficult to interrogate the HTML output.


ARROW-9828: [Rust] [DataFusion] Support filter pushdown optimisation for TableProvider implementations

I've got a use case for this with a custom TableProvider implementation, so thought I'd give this a go :)
This PR allows TableProviders to optionally indicate that they support handling filter expressions either:

Inexactly, to simply optimise data retrieval in an approximate fashion; e.g. pruning in your classic chunked storage system with min/max column metadata stored per chunk
Exactly, in which case the relevant filter plan nodes can be optimised out entirely

Some preemptive concerns from my side:

Most of these concepts could probably have better names, open to suggestions here.
I'm not sure whether expressions are the correct thing to be pushing down to the provider.
I've had to update quite a few scan callsites with empty filter lists. Could this be handled in a better way?
Currently, only table scans using TableSource::FromProvider are supported, because we need a reference to the provider at optimisation time. #8910 removes the provider/named-based reference distinction entirely so I can rebase this once that's merged and add an extra test using an ordinary sql statement, rather than just a ctx.read_table(provider) call.

I'd appreciate any thoughts or feedback!

...aaand a new conflict has already landed since I started writing up this post, will rebase shortly :D

Thank you @returnString ! This looks awesome -- I will try and review it carefully tomorrow.

FYI this will conflict with parts of #8910. IMO we should merge #8910 first, as it simplifies some of the code, making this PR easier.

FYI this will conflict with parts of #8910. IMO we should merge #8910 first, as it simplifies some of the code, making this PR easier.

Yeah definitely; like I mentioned, that's actually required for this to work with ordinary query methods (not just read_table(provider) as the tests are using currently) because we need a reference to the provider whilst doing logical plan optimisation :)

Can you take a look at ARROW-10902. I would be very interested by your input as it is very closely related.

#8910 removes the provider/named-based reference distinction entirely so I can rebase this once that's merged and add an extra test using an ordinary sql statement, rather than just a ctx.read_table(provider) call.

https://github.com/apache/arrow/pull/8910 is merged

#8910 is merged

Awesome, thanks for the heads up! I'll hopefully get some time to rebase this tomorrow evening.

I am very happy with the general heading of this PR. Thanks for your great work @returnString 😄 .
I can't shake the filling that we are adding too much complexity to the TableProvider enum (1 extra method + 1 signature change + extra enum type). Are we really sure that we couldn't achieve the same with less? (cf comment and comment)

Rebase is done, plus I've augmented the test coverage to also use the SQL code paths now that providers are resolved eagerly (thanks @rdettai 🥇)

One more rebase for some minor test changes that just landed :)

Codecov Report

Merging #8917 (2ba19ba) into master (d65ba4e) will increase coverage by 0.01%.
The diff coverage is 87.41%.

@@            Coverage Diff             @@
##           master    #8917      +/-   ##
==========================================
+ Coverage   83.25%   83.27%   +0.01%
==========================================
  Files         196      198       +2
  Lines       48116    48251     +135
==========================================
+ Hits        40059    40181     +122
- Misses       8057     8070      +13

Impacted Files
Coverage Δ

rust/datafusion/src/datasource/csv.rs
75.00% <ø> (ø)

rust/datafusion/src/datasource/empty.rs
52.94% <ø> (ø)

rust/datafusion/tests/custom_sources.rs
75.00% <ø> (ø)

rust/datafusion/src/datasource/memory.rs
83.59% <75.00%> (ø)

rust/datafusion/src/physical_plan/planner.rs
80.45% <80.00%> (-0.10%)
:arrow_down:

rust/datafusion/tests/provider_filter_pushdown.rs
85.24% <85.24%> (ø)

rust/datafusion/src/optimizer/filter_push_down.rs
97.65% <88.33%> (-1.74%)
:arrow_down:

rust/datafusion/src/datasource/datasource.rs
100.00% <100.00%> (ø)

rust/datafusion/src/datasource/parquet.rs
95.31% <100.00%> (ø)

rust/datafusion/src/execution/context.rs
89.98% <100.00%> (+0.02%)
:arrow_up:

... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d65ba4e...2ba19ba. Read the comment docs.

Nobody replied to this comment. The question is whether it is worth doing the filter pruning at the logical plan level or if it is more something for the physical plan. Otherwise you can merge!

Thanks again @returnString  -- @rdettai I believe I understand your concern -- and we can always enhance this API / trait to do pruning at the physical level as well.


fix(weaviate): fix weaviate terraform modules

Summary

fix weaviate service account by rerouting recipes to correct terraform modules
fix dagster helm version

Next step

Add authentification layer (api key and/or OIDC)

Test Plan

Local tf link

Checklist

[ ]  No images hosted from dockerhub
[ ]  Are dashboards present to understand the health of the application.  There must be at least 1 of these

[ ]  all databases should have dashboards
[ ]  ideally also have at least cpu/mem utilization dashboards for webserver tier of the app
[ ]  you can use plural from-grafana to convert a grafana dashboard found via google to our CRD

[ ]  Are scaling runbooks present

[ ]  all databases must have scaling runbooks
[ ]  you can use the charts in pluralsh/module-library to accelerate this

[ ]  do you need to add config overlays?

[ ]  inputing secrets
[ ]  configuring autoscaling

[ ]  If there’s a web-facing component to the app, we need to support OIDC authentication and setting up private networks if no authentication option is viable
[ ]  All major clouds must be supported

[ ]  Azure
[ ]  AWS
[ ]  GCP

@michaeljguarino added api key authentication


Not getting conflicts when expected in JavaScript

While investigating #2699, I was trying to simulate the conflict, and I found this more basic problem in conflict detection in javascript's database.commit():
First patch in: https://github.com/attic-labs/noms/pull/2706
Then:
cd samples/js/counter
npm install

Then run node . http://localhost:8000::c1  in one shell, wait a few seconds, then run same thing in a separate shell.
Here's what I see. In both shells, output is the same:
!? node . http://localhost:8000::c1
current value is 7
waiting 10 seconds...
done
succeeded, new val is: 8

What I expect is that second shell will have a conflict.

Note that this is committing to same dataset both times ... so this actually should conflict.

So this is working as intended. If you comment out the following in _doCommit():
          if (equals(commitRef, currentHeadRef)) {
            break;
          }

you'll get a Merge Needed error.

Ah. Makes sense. I think maybe we should take that feature out now, but
that's a separate bug.
On Fri, Oct 14, 2016 at 11:31 AM cmasone-attic<EMAIL_ADDRESS>wrote:

Closed #2707 https://github.com/attic-labs/noms/issues/2707.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/attic-labs/noms/issues/2707#event-824384824, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAE6BBeBpa17UzHAxndbuLI-QFtqAdQFks5qz8qTgaJpZM4KXNrQ
.


Mutation Bugs in havoc stage ( find_diffs )

https://github.com/IntelLabs/kafl.fuzzer/blob/4d9404ddd7b0760d419b4ff476b63d7e1b8c6201/kafl_fuzzer/common/util.py#L89
I found dead code that return always
first_diff = 1
last_diff = min(len(data_a), len(data_b)) -1

so this code cannot find different segment payload A and payload B.
i think it need to patch havoc stage like this.
this bugs exist from kAFL 0.2v to kAFL 0.7v
def find_diffs(data_a, data_b):
    first_diff = 0
    last_diff = 0
    for i in range(min(len(data_a), len(data_b))):
        if data_a[i] != data_b[i]:
            if first_diff == 0:
                first_diff = i
            last_diff = i

Thanks again for the bug report !
Fixed by https://github.com/IntelLabs/kafl.fuzzer/pull/73


Sample Config Leap Seconds needs update

Describe the bug
The sample configuration's CFE_MISSION_TIME_DEF_LEAPS was set to 32 and should currently be 37
To Reproduce
Steps to reproduce the behavior:
Visual inspection
Expected behavior
CFE_MISSION_TIME_DEF_LEAPS should be 37
Code snips
https://github.com/nasa/cFE/blob/e0d1ce883476931c86adc00adac49edea1a30d5a/cmake/sample_defs/sample_mission_cfg.h#L176
System observed on:
N/A
Additional context
#187
Reporter Info
Alex Campbell NASA GSFC

Edit: Added Leap second NIST table to "addditional context" section


NixOS: libgmp fails to load due to UnsatisfiedLinkError

When using fast-rsa in the nixos environment, I can't load gmp unless I specify the .so lib path explicitly in jna.library.path. Here's the stacktrace I see:
Java::JavaLang::UnsatisfiedLinkError (Unable to load library 'gmp': Native library (linux-x86-64/libgmp.so) not found in resource path ([file:/tmp/jetty-<IP_ADDRESS>-8080-.dir/webapp/WEB-INF/lib/jruby-core-1.7.22.jar, file:/tmp/jetty-<IP_ADDRESS>-8080-.dir/webapp/WEB-INF/lib/jruby-stdlib-1.7.22.jar, file:/tmp/jetty-<IP_ADDRESS>-8080-.dir/webapp/WEB-INF/lib/.null.jar, file:/tmp/jetty-<IP_ADDRESS>-8080-.dir/webapp/WEB-INF/lib/jruby-rack-1.1.19.jar])):
com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:271)
com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:398)
com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:337)
com.squareup.jnagmp.LibGmp.load(LibGmp.java:69)
com.squareup.jnagmp.LibGmp.loadLibGmp(LibGmp.java:65)
com.squareup.jnagmp.LibGmp.(LibGmp.java:52)
com.squareup.jnagmp.GmpInteger$MpzMemory.(GmpInteger.java:40)
com.squareup.jnagmp.GmpInteger.(GmpInteger.java:49)
com.squareup.crypto.rsa.NativeRSACoreEngine.init(NativeRSACoreEngine.java:71)
com.squareup.crypto.rsa.NativeRSAEngine.init(NativeRSAEngine.java:34)
org.bouncycastle.crypto.encodings.PKCS1Encoding.init(Unknown Source)
org.bouncycastle.jcajce.provider.asymmetric.rsa.DigestSignatureSpi.engineInitSign(Unknown Source)
java.security.Signature$Delegate.engineInitSign(Signature.java:1174)
java.security.Signature.initSign(Signature.java:527)
org.jruby.ext.openssl.PKey.sign(PKey.java:207)
etc...

this looks very much like a warbler war file :)
what is the file:/tmp/jetty-<IP_ADDRESS>-8080-.dir/webapp/WEB-INF/lib/.null.jar
? not sure if this is something jetty just adds. assume it warbled war-file
has it as well
before I start digging into warbler I would appreciate more info how you
created the warfile or some minimal sample project.

@crufia Out of curiosity, what is the explicit jna.library.path you did set to make this work?
@mkristian My first thinking when I saw this issue was that it might be something that jnr/jnr-ffi#50 might solve, since NixOS (a Debian-derivative) does a lot of clever tricks to ensure isolation and immutability, and futzing with the load paths might be one of them

@rtyler not sure since the linux-x86-64/libgmp.so is part of transitive jar from the gem
there is some other load issue here.
but yes, it would be nice to add this jnr-ffi jar to war but that is not possible with jnr-ffi :(

Thanks guys. I'll ping you privately about warbler and how I create the war file - fwiw, I made a mistake when formatting the issue. I didn't realize that angle brackets would disappear entirely. I had a bunch of (lol)s here, except, well...

the load library code looks into the wrong classloader. workaround for the current gem:
java.lang.Thread.currentThread.context_class_loader = JRuby.runtime.jruby_class_loader

I confirmed that the issue is isolated to warbler (or at least has nothing to do with nix). The workaround (specifying the classloader before requiring fast-rsa-engine) didn't work - same issue.

that is what I feared that the classloader change does not stick.
diff --git a/config/application.rb b/config/application.rb
index 9fe070d..673bc94 100644
--- a/config/application.rb
+++ b/config/application.rb
@@ -41,3 +41,9 @@ module MyApp
   end
 end
+
+java.lang.Thread.currentThread.context_class_loader = JRuby.runtime.jruby_class_loader
+
+require 'fast-rsa-engine'
+private_key = OpenSSL::PKey::RSA.new(File.read(File.expand_path('../foo.pem', __FILE__)))
+p private_key.sign(OpenSSL::Digest.new('sha512'), 'hello world')

where I immediately load the native library by using the respective code.

will prepare a workaround for the gem but want to wait for a feedback from square/jna-gmp

@crufia this is the hack I prepared. https://github.com/lookout/fast-rsa-engine/commit/d5e8e127b6e4e923e1a1b26100e0ca8c9861c874
could you try it on your side if this helps to make sure it works if I release a new gem

the actual fix would be a new release with: https://github.com/square/jna-gmp/pull/4


Implement Bot API v3.4

A new update has been released: https://core.telegram.org/bots/api#october-11-2017
Implement the changes in v2 and v3 branches.

Implemented everything in v2 and v3, closing this issue


[$1000]  Chat inconsistency between User A and User B after a Request money

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

user A login
user A creates a money request for user B
user B login
user B pays request money from user A
user A writes a message in the IOU thread with user B, click send
user B cannot see the message
user B must exit the chat and enter again to be able to see the message sent by user A

Expected Result:
both users should be able to see chat messages the moment they are sent
Actual Result:
both users cannot see the chat messages at the time they are sent, one of the users must exit the chat to update
Workaround:
Can the user still use Expensify without this being fixed? Have you informed them of the workaround?
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: 1.3.49-0
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/43996225/2c1da535-cfd9-401e-8f5f-6e1d9a32683e
https://github.com/Expensify/App/assets/43996225/703cebfe-09c2-4c6d-b691-d411ee12332b
https://github.com/Expensify/App/assets/43996225/ec0e4c91-36ea-4001-a036-01c9dde1a3eb
Expensify/Expensify Issue URL:
Issue reported by: @eusalazar
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1690939247360729
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~0106ee6101f89ce228
Upwork Job ID:<PHONE_NUMBER>954178560
Last Price Increase: 2023-08-23

I haven't gotten to testing this behavior. I'll work on it this afternoon.

Commenting to get Melvin to chill out with the Overdue label.

Hey, I'm Piotr from Callstack - expert contributor group - I would like to help resolve this issue

Update: I was able to reproduce the issue. Seems like it could be backend issue, I've checked in websocket and there were no events to add new report.

@eVoloshchak do you agree with @Piotrfj that this could be a backend issue? If so, I'll make this internal.

@johncschuster, yes, looks like this could be a backend issue, I also can't see any events.
This isn't 100% reproducible though, it didn't work for me initially, but was working properly after I refreshed the browser  for user B

Got it. Thanks for calling out the difficulty of reproducing this!

@mountiny @luacmartins what are your thoughts on this one? FWIW I'll be ooo for the next 10 days so if this is urgent we should reassign, or otherwise downgrade it to weekly.

Can we consistently reproduce this issue? It seems like we had some issue in the past, which makes me think that's related to pusher being flaky and we could just close the issue. However, if we can consistently reproduce then I agree this is internal and has some urgency to it.

I get the sense that we cannot reliably reproduce this issue but I could be wrong. @Piotrfj, are you able to clarify the reproduction steps to consistently reproduce this?

@johncschuster I also wasn't able to reproduce it every time, but it occurred in more than 50% trials.

can we confirm that this is indeed 50%? It seems like this is just pusher being flaky at times, but I wouldn't expect it to be that high though.

I'm not sure how I would confirm it's above or below 50% of the time. Is there a way to tell that Pusher is being flaky when we are testing this behavior? Like, how do we know if we can attribute the behavior to Pusher vs something else?

@Piotrfj do you get the pusher events with the new comments? I took a look at the backend code and I couldn't find any logic that filters out events. So if you're not getting the pusher update, that means something must be happening on pusher's end.

Ping @Piotrfj ?

@luacmartins @flodnv Here is an example of websocket actions comparing IOU thread and normal thread
https://github.com/Expensify/App/assets/41384305/07ff3ed7-54c4-45f0-929e-1d0eec43d2ff
https://github.com/Expensify/App/assets/41384305/a51ac63e-fdef-4f3f-aca5-adffc97aa45f
As you can see in IOU thread frontend does not receive the 'reportComment' event.

@Piotrfj I've found the WS console to be quite unreliable for inspecting these events. Could you try logging pushJSON here? Do you still see the same results?

(Bumping to make Melvin happy since Carlos' comment didn't count)

@luacmartins After debugging in proposed approach I've received this output:

First screen is presenting IOU thread, second one normal 1v1 thread.

That response does indicate that we're missing the message in the update. We gotta investigate this internally then.

No updates, and probably none for a while.
On Tue, Sep 19, 2023 at 5:56 PM melvin-bot[bot] @.***>
wrote:

@flodnv https://github.com/flodnv, @johncschuster
https://github.com/johncschuster, @eVoloshchak
https://github.com/eVoloshchak Eep! 4 days overdue now. Issues have
feelings too...
—
Reply to this email directly, view it on GitHub
https://github.com/Expensify/App/issues/24042#issuecomment-1725189942,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA7Y2OPG5QTPP55LF6KC2UTX3FT4TANCNFSM6AAAAAA3A7FCFE
.
You are receiving this because you were mentioned.Message ID:
@.***>

In that case, do we want to downgrade the label?

I can confirm I was able to reproduce this

Nice one, @flodnv!

No update here

No updates, still focusing on higher priority strategic projects.

Thanks for the update, @flodnv! Should we downgrade the label on this one?

I did that but you undid that last week 😄

Still no update for now.

Sorry for the delay. Turns out user B will never receive the pusher update, does not matter how many message there are in the IOU thread, the backend was never sending it, so I fixed that 🙃 PR is in review...

I will put this on hold for internal issue https://github.com/Expensify/Expensify/issues/326093


updates files in the destination dir. if needed

Dear @BioOmics ,
would it be possible or useful to add an option that updates files in the destination directories only if changes have occurred? For example, if the data in the database has been updated or modified, then download and update only those files, not the others - similar to how rsync uses the --update option.
thanks
nic

Dear @nicolo-tellini
Thank you very much for your suggestion! iSeq may test this feature in the near future. However, I believe it might be a challenging task to implement, as it’s difficult to monitor high-throughput sequencing data for modifications in real-time. Additionally, the demand for repeatedly downloading updated high-throughput sequencing data might be relatively limited among our user base.
Therefore, this functionality may not be included in future updates—unless, of course, our testing goes exceptionally well.
Best regards

Hi @BioOmics ,
OK :) thanks a lot for your comment, I close the issue.
have a good day


[Question] Common procedures for dealing with crashes from docker containers in rkt

Hey guys, I was just wondering about some common procedures for troubleshooting core dumps (w/o going full-fledged debugger on it) that are produced by docker containers that are run with rkt? I tried fiddling with capabilities and such because they're the only real knobs I know I can turn. Or if you know the answer to this problem, that'd be cool too.
Essentially, I'm running rkt from 4.14.96-coreos and trying to do the following but with rkt.
$ docker run -i microsoft/powershell
Here's what the output looks like in rkt:
$ rkt --insecure-options=image run --interactive=true docker://microsoft/powershell
This results in pwsh segfaulting. Here's what it looks like in case it's necessary.
# rkt --insecure-options=image run --interactive=true docker://microsoft/powershell
Downloading sha256:9ff103f7a78 [=============================] 82.9 MB / 82.9 MB
Downloading sha256:c7051e06956 [=============================]     517 B / 517 B
Downloading sha256:3c3721ea51c [=============================] 57.8 MB / 57.8 MB
Downloading sha256:705054bc3f5 [=============================]     849 B / 849 B
Downloading sha256:7308e914506 [=============================]     164 B / 164 B
Downloading sha256:38e2e6cd562 [=============================] 32.3 MB / 32.3 MB

Then the related logs:
kernel: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
kernel: IPv6: ADDRCONF(NETDEV_UP): vethdf4a03fd: link is not ready
kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethdf4a03fd: link becomes ready
kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
systemd-networkd[576]: vethdf4a03fd: Gained carrier
systemd-timesyncd[668]: Network configuration changed, trying to establish connection.
systemd-timesyncd[668]: Synchronized to time server <IP_ADDRESS>:123 (1.coreos.pool.ntp.org).
systemd-udevd[24309]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
systemd-udevd[24309]: Could not generate persistent MAC address for vethdf4a03fd: No such file or directory
systemd-timesyncd[668]: Network configuration changed, trying to establish connection.
systemd-timesyncd[668]: Synchronized to time server <IP_ADDRESS>:123 (1.coreos.pool.ntp.org).
systemd-machined[1461]: New machine rkt-a76f50ab-276b-42ea-b635-b55c9333819e.
systemd[1]: Started Container rkt-a76f50ab-276b-42ea-b635-b55c9333819e.
systemd[1]: Started Process Core Dump (PID 24337/UID 0).
systemd-coredump[24338]: Process 24327 (pwsh) of user 0 dumped core.
systemd[1]: Stopped Container rkt-a76f50ab-276b-42ea-b635-b55c9333819e.
systemd-machined[1461]: Machine rkt-a76f50ab-276b-42ea-b635-b55c9333819e terminated.

I'd essentially like to be able to distinguish what Docker is doing different from rkt that could result in pwsh crashing (probably the other way, but I can't strace docker like I can rkt+systemd...), but every fucking Docker internals document talks about namespaces like they're magical or something when really I'm interested in how it's running things differently to determine whether this issue could be related to systemd-nspawn or something else that's missing.

This happens because seccomp is blocking a syscall used by powershell (a quick strace seems to point to membarrier()).
You can disable seccomp with --insecure-options=seccomp or preferably follow the seccomp guide to adjust the whitelist.

Awesome, thanks you.
That's so weird because membarrier(2) doesn't seem to error out in my strace log, nonetheless that solved it...or at least got me to a point that I can figure out the next step. Thanks!
$ strace -vf -o >( grep 'syscall_324\|membarrier' ) rkt --insecure-options=seccomp run --interactive=true --inherit-env docker://microsoft/powershell
12343 syscall_324(0, 0, 0x1, 0x7f570e168320, 0x1, 0x1) = 0x19
strace: Exit of unknown pid 11748 seen
Failed to initialize CoreCLR, HRESULT: 0x80004005
$

(Closing this as it's been resolved)


chore: add util to map datastore schema to generic

Description of changes:
Forms will use GenericDataSchema to interface with customer data models.
This PR -

Adds GenericDataSchema
Adds a util that converts DataStore Schema to GenericDataSchema.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Codecov Report

Merging #497 (a4b5f63) into tagged-release/forms-and-views (f54c226) will increase coverage by 0.24%.
The diff coverage is 100.00%.

@@                        Coverage Diff                         @@
##           tagged-release/forms-and-views     #497      +/-   ##
==================================================================
+ Coverage                           88.65%   88.89%   +0.24%
==================================================================
  Files                                  66       66
  Lines                                2301     2351      +50
  Branches                              608      624      +16
==================================================================
+ Hits                                 2040     2090      +50
  Misses                                257      257
  Partials                                4        4

Impacted Files
Coverage Δ

...generate-form-definition/helpers/field-type-map.ts
100.00% <ø> (ø)

.../lib/generate-form-definition/form-to-component.ts
81.48% <100.00%> (ø)

packages/codegen-ui/lib/generic-from-datastore.ts
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f54c226...a4b5f63. Read the comment docs.


Too large text on https://www.beamerbridge.com/

@olympe-silve
The text on our website looks too large:

I am not sure whether that was intentional, but at least for me it looks quite strange.
Notice that scaling the page results in a gap between the bottom of the text and the next section:

my 2cents:
I would place the logo and navigation inside of the main content container. This way on higher resolutions the beamer logo and navigation on the right would not be that far away from each other and the content.
And generally I would just go one size smaller with tailwind.

I think the first section uses a 100vh height, so this gap is more or less intentional. In my opinion we should not support any arbitrary weird scaling. Scaling is not the same as viewing the page on a bigger monitor.


deploy.prototxt question

Why does the 'Softmax' layer have a bottom linked to 'norm2' instead of 'fc9_1'?  Is this correct?

I think I see what you are doing - classifying to the 997 classes (identities) output by the 'fc9' layer.
Does this mean that the 'fc9_1' layer shown in the deploy.prototxt is not needed when running in deploy mode?

@ZoroDerVonCodier yes, softmax layer is useless in deploy.prototxt


High Nomad CPU usage when retrieving files

When retrieving files from Nomad with the new retrieve files API #159, I noticed a high CPU load of the Nomad process in my development setup (> 100%).

Before investigating further, we should collect some more information about concrete occurrences:

Does this issue happen in production as well (Linux vs macOS)? Can we reproduce it?
How often does this issue occur and do we see any side effects?


init commit of getting started guide

This PR will merge Joan's implementation guide into the development branch.

this PR has been moved to https://github.com/awslabs/aws-media-insights-engine/pull/77


Group2Invoke pass messes with static timing

With an example vadd program, group2invoke ends up significantly degrading the final latency of the design. Running this design with -p pre-opt -d group2invoke gives a top level static latency of 18, but running it with just -p pre-opt gives a top level static latency of 27. I'm not quite sure what optimizations are provided by group2invoke, but either there is a bug here or group2invoke should be turned off when using static timing.
Here's the design for reference:
import "primitives/core.futil";
component main<"toplevel"=1>(@clk clk: 1, @reset reset: 1, @go go: 1) -> (@done done: 1) {
  cells {
    std_slice_2 = std_slice(32, 3);
    std_slice_1 = std_slice(32, 3);
    std_slice_0 = std_slice(32, 3);
    std_add_1 = std_add(32);
    std_add_0 = std_add(32);
    std_lt_0 = std_lt(32);
    @external(1) mem_2 = std_mem_d1(32, 8, 3);
    @external(1) mem_1 = std_mem_d1(32, 8, 3);
    @external(1) mem_0 = std_mem_d1(32, 8, 3);
    stage_1_register_1_reg = std_reg(32);
    stage_1_register_0_reg = std_reg(32);
    stage_0_register_2_reg = std_reg(32);
    stage_0_register_1_reg = std_reg(32);
    stage_0_register_0_reg = std_reg(32);
    while_0_arg0_reg = std_reg(32);
  }
  wires {
    group bb0_0 {
      stage_0_register_2_reg.in = while_0_arg0_reg.out;
      stage_0_register_2_reg.write_en = 1'd1;
      bb0_0[done] = stage_0_register_2_reg.done;
    }
    group assign_while_0_init_0 {
      while_0_arg0_reg.in = 32'd0;
      while_0_arg0_reg.write_en = 1'd1;
      assign_while_0_init_0[done] = while_0_arg0_reg.done;
    }
    group bb0_3 {
      stage_1_register_1_reg.in = stage_0_register_2_reg.out;
      stage_1_register_1_reg.write_en = 1'd1;
      bb0_3[done] = stage_1_register_1_reg.done;
    }
    comb group bb0_6 {
      std_lt_0.left = while_0_arg0_reg.out;
      std_lt_0.right = 32'd8;
    }
    group bb0_7 {
      std_slice_2.in = while_0_arg0_reg.out;
      mem_0.addr0 = std_slice_2.out;
      stage_0_register_0_reg.in = mem_0.read_data;
      stage_0_register_0_reg.write_en = 1'd1;
      bb0_7[done] = stage_0_register_0_reg.done;
    }
    group bb0_8 {
      std_slice_1.in = while_0_arg0_reg.out;
      mem_1.addr0 = std_slice_1.out;
      stage_0_register_1_reg.in = mem_1.read_data;
      stage_0_register_1_reg.write_en = 1'd1;
      bb0_8[done] = stage_0_register_1_reg.done;
    }
    group bb0_9 {
      std_add_0.left = while_0_arg0_reg.out;
      std_add_0.right = 32'd1;
      while_0_arg0_reg.in = std_add_0.out;
      while_0_arg0_reg.write_en = 1'd1;
      bb0_9[done] = while_0_arg0_reg.done;
    }
    group bb0_10 {
      std_add_1.left = stage_0_register_0_reg.out;
      std_add_1.right = stage_0_register_1_reg.out;
      stage_1_register_0_reg.in = std_add_1.out;
      stage_1_register_0_reg.write_en = 1'd1;
      bb0_10[done] = stage_1_register_0_reg.done;
    }
    group bb0_11 {
      std_slice_0.in = stage_1_register_1_reg.out;
      mem_2.addr0 = std_slice_0.out;
      mem_2.write_data = stage_1_register_0_reg.out;
      mem_2.write_en = 1'd1;
      bb0_11[done] = mem_2.done;
    }
  }
  control {
    seq {
      assign_while_0_init_0;
      par {
        bb0_7;
        bb0_8;
        bb0_0;
        bb0_9;
      }
      par {
        bb0_7;
        bb0_8;
        bb0_0;
        bb0_9;
        bb0_10;
        bb0_3;
      }
      @bound(6) while std_lt_0.out with bb0_6 {
        par {
          bb0_7;
          bb0_8;
          bb0_0;
          bb0_9;
          bb0_10;
          bb0_3;
          bb0_11;
        }
      }
      par {
        bb0_10;
        bb0_3;
        bb0_11;
      }
      bb0_11;
    }
  }
}

This is related to #911: the problem isn't that group2invoke does anything bad, it's that remove-comb-group swoops in and converts invoke x(...) with cg to seq { cg_to_g; invoke x(...) } which increases latency for each invoke-with. Once #911 is solved, this should be fixed

Oh ok, great! I haven't looked much at the invoke syntax so I didn't know it also uses comb-groups.

Yup! https://docs.calyxir.org/lang/ref.html#invoke

@andrewb1999 if the above explanation makes sense, let's close this issue because fixing #911 will also fix this

Yep, that's fine with me. I'll go ahead and close this.


Documentation updates

Added CONTRIBUTING.md
Tweaks to README.md

Codecov Report

Merging #2 (9be9cd2) into master (4e87f53) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master        #2   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            6         6
  Lines          101       101
=========================================
  Hits           101       101

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Return a 400 when requesting an invalid feature

type: Feature request
What
When requesting polyfills it is currently possible to request a non-existent feature and receive a JS bundle in response, e.g.:
https://www.ft.com/__origami/service/polyfill/v3/polyfill.min.js?features=abcde
Details
I do not think this is desirable behaviour as it makes it possible for folks to incorrectly assume a feature exists or type in a feature name incorrectly without noticing. For example in this PR I referenced a different version of the docs (cc #1867) and because my page continued to work I assumed it was OK.
I think it may be beneficial to be stricter and respond with a 400: Bad Request when an invalid feature is requested.

I like this. It would be good for a v4 release


How to DNS Check TXT Records e.G. SPF1

⚠ Please verify that this bug has NOT been raised before.

[X] I checked and didn't find similar issue

🛡 Security Policy

[X] I agree to have read this project Security Policy

📝 Describe your problem
i want to check a spf Record which is always set as txt record.

i cant find a way how to get the infomration out of uptime Kuma

📝 Error Message(s) or Log
No response
🐻 Uptime-Kuma Version
.121.2
💻 Operating System and Arch
Ubuntu 20.02
🌐 Browser
Chrome / Edge
🐋 Docker Version
No response
🟩 NodeJS Version
No response

my bad, just unpatient! works!


Fix build failure due to specifying wrong branch name for gbenchmark

This PR fixes the build failure when enabling benchmarking (by default). Its branch name seems to have changed recently https://github.com/google/benchmark/issues/1116.
$ cmake -S .

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

CMake Deprecation Warning at build/cmake/gtest/gtest-src/googlemock/CMakeLists.txt:45 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

CMake Deprecation Warning at build/cmake/gtest/gtest-src/googletest/CMakeLists.txt:56 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/mssn/workspace/lab/hexl/build/cmake/gbenchmark/gbenchmark-download
[ 11%] Performing download step (git clone) for 'gbenchmark'
Cloning into 'gbenchmark-src'...
fatal: invalid reference: master
CMake Error at gbenchmark-download/gbenchmark-prefix/tmp/gbenchmark-gitclone.cmake:40 (message):
  Failed to checkout tag: 'master'

make[2]: *** [CMakeFiles/gbenchmark.dir/build.make:99: gbenchmark-prefix/src/gbenchmark-stamp/gbenchmark-download] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/gbenchmark.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
CMake Error at cmake/gbenchmark/CMakeLists.txt:17 (message):
  Build step for gbenchmark failed: 2

-- Configuring incomplete, errors occurred!
See also "/home/mssn/workspace/lab/hexl/build/CMakeFiles/CMakeOutput.log".
See also "/home/mssn/workspace/lab/hexl/build/CMakeFiles/CMakeError.log".

I'm sorry, I forgot to read README carefully and I found the line At this time, Intel HEXL does not accept external contributions. We encourage feedback and suggestions via issues there. I will remain open, but if you've already handled this issue, please never mind about this PR. Thanks!

Thanks for pointing this out, @musaprg. We just fixed this in v1.1.1 and on main. Hopefully we'll be able to add external contributors in the future.


spec.entrypoint is required on a wftmpl if we want to submit it

What happened:
When I try to submit a workflow template that does not define spec.entrypoint argo submit fails. It's not mandatory and the command should support it as workflows do when they call a workflow template that does not define it.
$ argo submit --from wftmpl/wt-run-get-reports-job -p p1=v1 -p p2=v2 -p p3=v3
2020/07/10 18:06:28 Failed to submit workflow: spec.entrypoint is required

What you expected to happen:
I expect the definition of entrypoint to not be mandatory if we want to submit a workflow template.
How to reproduce it (as minimally and precisely as possible):
Create https://github.com/lippertmarkus/argo/blob/a7ca46e1185145d7d7b27a802506f35d5bd8a524/examples/workflow-template/templates.yaml#L1-L15 after having removed the entrypoint and try to submit it
argo submit --from wftmpl/workflow-template-whalesay-template -p message=test
2020/07/10 18:34:08 Failed to submit workflow: spec.entrypoint is required

Anything else we need to know?:
Environment:

Argo version:

argo version                                                                                 100%    18:36:10 
argo: 2.8.2+8a151ae.dirty
  BuildDate: 2020-06-18T23:53:01Z
  GitCommit: 8a151aec6538c9442cf2380c2544ba3efb60ff60
  GitTreeState: dirty
  GitTag: 2.8.2
  GoVersion: go1.13
  Compiler: gc
  Platform: darwin/amd64

Kubernetes version :

clientVersion:
  buildDate: "2020-06-18T02:59:13Z"
  compiler: gc
  gitCommit: c96aede7b5205121079932896c4ad89bb93260af
  gitTreeState: clean
  gitVersion: v1.18.4
  goVersion: go1.14.3
  major: "1"
  minor: "18"
  platform: darwin/amd64
serverVersion:
  buildDate: "2020-03-27T21:51:36Z"
  compiler: gc
  gitCommit: af3caf6136cd355f467083651cc1010a499f59b1
  gitTreeState: clean
  gitVersion: v1.15.11-eks-af3caf
  goVersion: go1.12.17
  major: "1"
  minor: 15+
  platform: linux/amd64

Message from the maintainers:
If you are impacted by this bug please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

you can pass entrypoint as commandline option
 argo submit --from wftmpl/wt-run-get-reports-job -p p1=v1 -p p2=v2 -p p3=v3 -entrypoint <templatename>

Yeah I can but I think it's only a workaround.

Submitting workflowtemplate is basically executing workflowtemplate entire as a workflow. Entrypoint is a mandatory on workflow.
you can define entrypoint on submittable workflowtemplate or you can pass through commandline option

I'm sorry @sarabala1979 but I do not agree with the new label you assigned it. When I use a template by referencing it in a workflow I never specify the entrypoint but it's executed as expected:
templateRef:
            name: wt-sync-accounts-job
            template: sync-accounts-job

I think it should still be labeled as a bug cause you are adding a limitation that clearly should not be there.

oh wait I actually need to specify the template

Workflowtemplate will be used in two scenarios.

Referring defined Templates from workflowtemplate to workflow as templateref. In this scenario, entrypoint is optional.
a. This is mainly just reusing  template definitions with different arguments
After 2.8, you can define the entire workflow as Workflowtemplate and you can create workflow out of workflowtemplate. In this scenario, entrypoint is mandatory (you can have it in workflowtemplate or you need to pass while submit --from)

For anyone who finds this issue searching for the error like I did, albeit a different root cause:

My issue was specific to CronWorkflows
I had spec.entrypoint (incorrectly) defined
Starting a CronWorkflow manually would fail in the UI with invalid spec: spec.entrypoint is required
CronWorkflows should have entrypoint defined under spec.workflowSpec.entrypoint CronWorkflow Spec Docs
Once I updated the spec, the issue was resolved.


[dkr] add ENABLE_FAILPOINTS to all Dockerfile

Motivation
Make those docker stage identical with validator to save some build time.
Have you read the Contributing Guidelines on pull requests?
(Write your answer here.)
Test Plan
(Share your test plan here. If you changed code, please provide us with clear instructions for verifying that your changes work.)
Related PRs
(If this PR adds or changes functionality, please take some time to update the docs at https://github.com/libra/website, and link to your PR here.)

/canary

:broken_heart: Test Failed - commit-workflow

/land


Draft of a governance doc

Adapted from rasterio's, which is adapted from NumPy's.
This is a follow up on something I wrote at the start of 2023: https://github.com/shapely/shapely/discussions/1740#discussioncomment-4813526.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 87.592%

Totals

Change from base Build<PHONE_NUMBER>:
0.3%

Covered Lines:
2506

Relevant Lines:
2861

💛  - Coveralls

@caspervdw I was hoping that adapting Numpy's governance doc would be uncontroversial. And I find it to be both practical and philosophical.
As to professionalization: I don't want to take it too far! I'd like Shapely to not depart too far from its punk and DIY roots :smile:

Unrelated to this PR: it looks like we have some flaky tests.

Yep the GEOS main branch often fails because we have lots of super strict tests. I think that’s a good thing but having the red X everywhere is not.
Maybe we should allow the failures against main(and manually check them)

@sgillies  are you :+1: on merging this?


Initial Insight dashboard

This PR implements the Insight Generator dashboard as a Dash app. The app is wired into Dazzler so it benefits from the existing multi-tenancy setup at no additional dev cost.
Dashboard functionality
The dashboard renders Insight Generator reports. Each report is made up by a set of recommended settings for each KPI Insight Generator analysed as well as recent KPI evolution over time. The recommended settings come from the Results attribute of a given NGSI Insights entity whereas the KPI evolution over time still needs some clarification---see below.
To display a report, you press the "Load Entity IDs" button, then select the ID of the report you're interested in from the drop down menu on the right of the button. A group of tabs will appear on the right of the dashboard containing the report data.
There's a tab for each KPI Insight Generator analysed. The tab contains a graph plotting the KPI values over time. Below the graph are the parameter settings Insight Generator recommends to optimise the KPI. The expected KPI value after applying the recommended settings is shown below the parameter table.

Loose ends

Semantics. My interpretation of the Results attribute of the NGSI Insights entity is the following. KPI_name lists the (names of) KPIs Insight Generator analysed, KPI_name = k1, k2, .... For each k[i] there's a corresponding set of recommended settings which are basically a list of name-value pairs: (features_names[i][j], features_values[i][j]), with 0 ≤ j < len(features_names[i]. Also, for each k[i] there's a corresponding KPI_best[i] value which is what you'd get if you applied the recommended settings. Is this the right interpretation?
KPI graphs. What data should KPI graphs plot? The CSIC mockup seem to plot the KPI best value as a constant function of time---i.e. for k[i] plot the functionf(t) = KPI_best[i]. Not sure that's what they meant, so another possibility is that for each KPI listed in the Results structured value, there's a corresponding time series to fetch. This is what we do in the implementation of the demo datasource, but obviously, that's just my interpretation which could be wrong!
KPI entities. If we do want KPI time series, then we'll need to decide what NGSI entities should capture them.
Most relevant params. Not sure where that data should come from so the dashboard doesn't display them at the moment. Are the params for k[i] the same as features_names[i]?
Generate button. Not there at the moment, will need some brainstorming. For example, if the report generation takes about 30 mins, we should probably trigger it by some other means since the user isn't likely to wait that long for the dashboard to load?

Demo
Start the live simulator as explained in the README, then browse to http://localhost:8000/dazzler/demo/-/insight/.

@avillaj1988 if you could please

have a look at the demo---see instructions above
help clarify those loose ends I listed in the PR description
check out the code to see if it makes any sense---Dash is basically a Python backend for Plotly...

@avillaj1988 oh, forgot to mention. if you'd like to debug the app easily, change this line

https://github.com/c0c0n3/kitt4sme.dazzler/blob/main/dazzler/config.py#L26

to
        builder='dazzler.dash.board.insight.dash_builder')

then run Dazzler with a debugger and browse to http://localhost:8000/dazzler/demo/-/


refine doc block parsing to be more than just one big token

Will not do until doc block topology settles down


Add supported fields to Authenticate struct type

Adds DeviceName, Model, and ModelName to Authenticate struct, which are required fields for an AuthenticateRequest according to Apple's documentation

This PR was intended for a fork and was mistakenly opened here.

For what it's worth: the table-prefix thing is a feature I've thought about adding. Having a configurable/dynamic option for that might be useful. Downgrading the syntax to MySQL 5.x isn't something we'll likely support (though — I might be open to adding a mysql5 backend. Maybe. Maybe not.
As far as the actual fields — we don't need to support them because none of the storage backends use them and the Raw authenticate message is passed to any integrations (notably the web hook). As well, in mysql backend the Raw authentication message is stored on the device record. So anybody can use those fields if they wish without having to add them. Though I might be open to adding them if a storage backend (or integration) really needed them (like we do for e.g. serial number).

Added #64 :)


docs(developer-guide): add project flag to firebase commands, add pul…

…lImages to personal env example

@beeman Thanks for the PR!
I forgot that specifying the project id is only needed if you haven't already selected a firebase project via the CLI a la
$ firebase use <PROJECT ID>

So maybe we can add a not on that too?

Merging this for now and add the additional note in another PR


Unable to use default_type for NXOS netconf

Describe the bug
When specifying default_type in get_config() calls against NXOS (only 9.3.(10) tested so far), scrapli_netconf raises a CapabilityNotSupported error at base_driver.py#L530
This is because NXOS returns the string in its server capabilities: urn:ietf:params:netconf:capability:with-defaults:1.0?basic-mode=report-all
But the comparison in base_driver.py#L525 looks specifically for urn:ietf:params:netconf:capability:with-defaults:1.0 in the list of server capabilities
To Reproduce
Steps to reproduce the behavior:
NXOS_NS_FILTER = '<System xmlns="http://cisco.com/ns/yang/cisco-nx-os-device"/>'
conn = NetconfDriver(**nxos_box)
conn.open()
response = conn.get_config(filter_=NXOS_NS_FILTER, default_type="trim")

Expected behavior
Manually querying the box for ?with-defaults=trim shows that it does appear to "support" with-defaults and does indeed return only trimmed results (similarly for "explicit")... though cisco.com has little documentation or examples to support my findings :).  So I would ideally like it not throw an exception and to have the "trim" config results returned
Suggestion
Perhaps its enough to look over each server capability string in the list, looking for at least urn:ietf:params:netconf:capability:with-defaults:1.0 in one of those strings, to conclude with-defaults is supported?

👋 hey @sanjmonkey thanks for raising this! your suggestion seems reasonable! would you be interesting in throwing together a pr for it?
thanks!

closing this out, will follow up in the pr! thanks again!


Surface ruletype validation errors to user

Summary
Make the ruletype validation errors visible to the user, so they get guidance on what to fix.
Change Type
Mark the type of change your PR introduces:

[ ] Bug fix (resolves an issue without affecting existing features)
[x] Feature (adds new functionality without breaking changes)
[ ] Breaking change (may impact existing functionalities or require documentation updates)
[ ] Documentation (updates or additions to documentation)
[ ] Refactoring or test improvements (no bug fixes or new functionality)

Testing
Outline how the changes were tested, including steps to reproduce and any relevant configurations.
Attach screenshots if helpful.
Review Checklist:

[x] Reviewed my own code for quality and clarity.
[ ] Added comments to complex or tricky code sections.
[ ] Updated any affected documentation.
[ ] Included tests that validate the fix or feature.
[ ] Checked that related changes are merged.

coverage: 54.729%. remained the same
when pulling 0b07b991fe027f345808526c71f89a0cd367e2ea on eleftherias:ruletype-validation-user-visible-error
into ea52b4b617be36f9357a6aa457243cf42bdfeb6d on mindersec:main.


RuntimeWarning：程序正常配置后运行后报错

日志如下:
[root@VM-0-3-centos JustList]# python3 main.py
[saved] config -> /root/justlist/JustList/app/core/onedrive/config.json
[onedrive] checking at 2020-12-11 22:23:43
[saved] config -> /root/justlist/JustList/app/core/cloud189/config.json
[cloud189] checking at 2020-12-11 22:23:44
[cache] running
<frozen importlib._bootstrap>:228: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:228: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:228: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:228: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:228: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:228: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
Segmentation fault
[root@VM-0-3-centos JustList]# python3 main.py
[saved] config -> /root/justlist/JustList/app/core/onedrive/config.json
[onedrive] checking at 2020-12-11 22:23:57
[saved] config -> /root/justlist/JustList/app/core/cloud189/config.json
[cloud189] checking at 2020-12-11 22:23:57
[cache] running
<frozen importlib._bootstrap>:228: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:228: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:228: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:228: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:228: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
<frozen importlib._bootstrap>:228: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
Segmentation fault
[root@VM-0-3-centos JustList]# ^C
[root@VM-0-3-centos JustList]#

看上去好像是Python3.9造成的
更换至Python3.8.6程序就能正常运行

暂时还没切换至 Python 3.9 及以上版本，所有可能会有些 bug 不知道... 之后 debug 一下，感谢反馈。


Registration of schema on the Azure Schema Registry in Kafka Connect scenario

Could you tell me whether it is possible to setup the source connector in Kafka Connect scenarios so that it does not automatically register schemas on the Azure Schema Registry.
I ask since I'm currently using com.microsoft.azure.schemaregistry.kafka.avro.AvroConverter class as a value converter in the source connector. I'm using Kafka Connect to ingest data from postgres database to a Kafka topic and I'm storing schemas in Azure Schema Registry.
Automatic schema registration is not the best practice, hence I would like to avoid it.
As far as I can see auto.register.schemas option was removed from configuration of connectors so I am interested; is there any alternative way to prevent connector from automatically registering schemas?
This is the configuration of my source connector:
{
"name": "jdbc-postgresql-avro-connector-azure",
"config": {
"connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
"tasks.max": "1",
"value.converter": "com.microsoft.azure.schemaregistry.kafka.avro.AvroConverter",
"value.converter.schema.registry.url": {schemaRegistryUrl},
"value.converter.schema.group": "postgreskafkaschemagroup",
"value.converter.tenant.id": {tenantId},
"value.converter.client.id": {clientId},
"value.converter.client.secret": {clientSecret},
"connection.url": {postgresConnectionUrl},
"connection.user": {user},
"connection.password": {password},
"connection.attempts": 3,
"mode": "incrementing",
"query": "SELECT * FROM users",
"table.types": "TABLE",
"topic.prefix": "users",
"incrementing.column.name": "id"
}
}
I would appreciate any help.

You'd need to edit this line to pull the boolean from AvroConverterConfig rather than have it be hard-coded to true
https://github.com/Azure/azure-schema-registry-for-kafka/blob/master/java/avro-converter/src/main/java/com/microsoft/azure/schemaregistry/kafka/connect/avro/AvroConverter.java#L75

@OneCricketeer Have you success load the com.microsoft.azure.schemaregistry.kafka.connect.avro.AvroConverter? I am confued on loading this plugin into cluster

It's no different than other plugins mentioned in the Kafka documentation.
Build this jar using mvn package, copy it to your Connect workers under their plugin.path config directories

Thanks @OneCricketeer . Fixed it.
Do you know why the shema parma I get from fromConnectData method is always a string? Do you know examples of input sources?
Thanks in advance.

Unfortunately, I can't help without seeing your code.
If you use StringConverter, you'll get a String-type Connect Schema. Try using a structured event instead

Thanks.
I am trying to test this since our Schema stored in Azure, and trying to reading Avor data from Azure Eventhub to other Azure Events similar like distribute.
Seems we need to do the define the Schema in poll method in source connector side when insert the message into the SourceRecord Array.

The schema or its ID should be encoded within the event itself. The converter class is not indented to be used standalone, only within a Connector, such as the original question about JDBC Source. Are you trying to write your own? Perhaps you can start a new issue thread? @zhaozy93

Hi @zhaozy93 Could you please tell me how did you integrate this jar with plugin?
Im using Strimzi Kafka Connec and  im just adding this jar - https://mvnrepository.com/artifact/com.microsoft.azure/azure-schemaregistry-kafka-avro into my lib/ directory which is located in zip file downloaded from confluent
Then i tried to set
value.converter: com.microsoft.azure.schemaregistry.kafka.connect.avro.AvroConverter
but i got error message:
Invalid value com.microsoft.azure.schemaregistry.kafka.connect.avro.AvroConverter for configuration value.converter: Class com.microsoft.azure.schemaregistry.kafka.connect.avro.AvroConverter could not be found


Variance-weighted ANOVA example

Following up on #13 here's an example of a variance-weighted ANOVA that can be performed if transformations don't resolve the problems with assumption tests for the scenario 'CRD with 1 treatment':
Instead of assuming equal variance with: model <- aov(y ~ group, data = my.data)
Do variance-weighted ANOVA that does not assume equal variances. There could be option below transformation drop-down to do that, but don't allow transformation + variance-weight ANOVA.
oneway.test(y ~ group, data = my.data)
(Source: https://stat.ethz.ch/R-manual/R-devel/library/stats/html/oneway.test.html)

nope.


[garden-local] Only add loopback addresses for multi-zonal setup

How to categorize this PR?

/area testing
/area high-availability
/kind enhancement
What this PR does / why we need it:
Assuming that most developers will use the standard local-setup for testing and evaluation, it's not necessary to add additional addresses to the loopback device. This PR lets the setup only add those addresses if a multi-zonal environment is required.
Special notes for your reviewer:
/cc @ScheererJ
Release note:

Gardener only adds additional addresses to the machine's loopback device if a local multi-zonal environment is used. Earlier this step was performed with all `make kind-*up` commands.

/retest-required

@shafeeqes Can we merge this? It only concerns the local setup

@shafeeqes Can we merge this? It only concerns the local setup

Sure.
/milestone v1.63


Team update - Jun 08, 2022

This is a @Quansight-Labs/czi-a11y-grant team sync 🎉🎉🎉! This is a way for the Team Members to provide status reports on what they've been up to this week and request help and attention for things they are working on. This issue will be closed at the end of the day.
Copy and paste the template below, and answer questions as you wish!

Response Template
**Thanks I'd like to give 🙌**
- So-and-so helped me out a lot with foo...

**Updates from last week :heavy_check_mark:**
- I worked towards deliverable: <link-to-deliverable>

**Challenges I faced and things I'd like assistance with 🙏**
- I had a hard time figuring out ...

**My availability for next week**
- I'll be off on foo day...

**Important items for discussion 💬**
- I have a question about goal <link-to-goal>
- Can @foo give a comment on issue #NN ?
- I opened #NN for discussion, please chime in
- I need to discuss foo

:mag: Needs Triage
The Needs Triage issues require an initial assessment and labeling.
No issues need triage! 🎉
:dart: Project boards
Please make sure the boards reflect the current status of the project.
Jupyter a11y grant project

Thanks I'd like to give 🙌

Thanks to @tonyfast for the continued Jupyter expertise! I am so grateful I can always ask you where to look for info!

Updates from last week :heavy_check_mark:

quansight-labs/accessibility #3 accessibility statement has been updated. I think it's ready for merge, unless I missed something.
jupyter/accessibility #95 testing script documentation has been opened.
#93 jupyter/accessibility #94 accessibility workshop to community docs has been opened.
Collected all our alt text events so we can reference them easily. Kinda of relates to #32.
#128 has a PR at #137 now. Please review since I'm giving it my best guess.

Challenges I faced and things I'd like assistance with 🙏

Does the draft accessibility statement need more changes? I think I've addressed almost all the feedback, but I'm not sure what to do to finish this.

My availability for next week

I'll be out of office on June 10.
I'll also be out a little later this month, just an early heads up.

Important items for discussion 💬

Can we all review the roadmap changes? They're short so it won't be much of your time; I want to be sure I'm representing y'all accurately.

Thanks I'd like to give 🙌

Muchas gracias for your patience, all of you.

Updates from last week :heavy_check_mark:

I was out last week, nothing done on my end
Other than the meeting we had Monday with lots of good discussion, I've given the grant no time so far this week.

Challenges I faced and things I'd like assistance with 🙏

Website is full steam ahead, but there's no assistance to solve that issue.

My availability for next week

I'll be available all next week.

Important items for discussion 💬

Next work cycle

@gabalafou no updates at the moment
@tonyfast

fixes on jupyter/accessibility #90 based on feedback
trying to use jupyterlab/extension-cookiecutter for tests and exploring for theming work


Update PHP CS Fixer name to proper one

It is already disabled? or it will be disabled?
can you do it on your own, please?
or just restart the CI job

It is already disabled? or it will be disabled?
can you do it on your own, please?
or just restart the CI job

I think Jessica meant that you need to make your current branch up-to-date with the latest master.
git remote add upstream git@github.com:sdras/awesome-actions.git
git pull upstream master
git push

@jessicalostinspace , @fabasoad ,
are you aware of this feature ? you can use it from your side ;)


Ruby parse errors throw confusing error

Summary
I ran into this fun today when I left a trailing comma on an attr_reader
If the Ruby class is invalid, the library should not fail with initializer must accept ... ArgumentErrors. Instead it should throw an error better explaining the issue.
Open question: is it valid to initialize a class for a view component without an initialize parameter? It seems like it should be 🤔 .
Other Information

@joelhawksley requested changes made 💖

Open question: is it valid to initialize a class for a view component without an initialize parameter?

What do you think about that? Sort of reminds me of stateless components in React


Add pinecone metadata string array support

Add pinecone metadata string array support
Pinecone documents support string arrays, so they shouldn't be flattened.
This fix basically removes all arrays from the metadata object which contain only strings and appends them after the remaining metadata object was flattened.
Fixes #1446

Fantastic PR, thank you!

Thanks a ton! I was actually working on this in my own personal project 🙏

Not announced but 0.0.90 is live with it :)


Private methods and properties on classes and objects

Allow defining private methods and properties on classes and objects. The syntax should work with both const and private on a property.
Syntax:
private class Person {
  private let mySecret = "..."

  private const func foo() {}
  const private func bar() {}
}

Won't be implemented.


Keep eye checkbox position static on horizontal scroll, make model and categories tree whitespace between eye checkbox and node label smalelr

closes #493
before:

after:

The PR contains breaking changes - please make sure to run extract-api and rush change. The change files should document all the API-related changes in addition to visual component changes.

Will this PR also take care of the eye alignment issue in Spatial Containment widget?

@wilmaier No, because Spatial Containment tree uses its own css styles.


Aggregated Codecov Reports

This PR makes the following changes to AdvantEDGE:

Updated entrypoint.sh files for core and sandbox pods to execute services based on codecov mode flag and generate timestamp-based codecov output files.
Updated meepctl test to generate aggregated codecov reports from the timestamp-based codecov reports.

Please rebase to the tip of develop for merge.


How to improve the quality of released dataset

I observe that there are many low-quality pairs in the dataset that is published, such as pairs with significant differences between the initial and final frames, or where the initial frame is the beginning of a video (with no substantive content). These instances are likely due to training noise. Are there any further filtering methods available?
Additionally, each video in ChangeIt contains multiple shots. In this case, how can we avoid the model from treating the contents of different shots as collected into one pair? Are there more granular annotations available in ChangeIt (such as distinguishing each shot), or do the models provided by ChangeIt have this capability? Alternatively, can the dataset be pre-clipped using other methods?
Looking forward to your response, thank you very much.

We did not use any additional filtering. But using any such filtering could improve the results.
As for the shots: We did not handle the shots in any special way. Beware that in many videos, any single shot may not contain objects in all states. For example, this is the case if multiple cameras are used to film the action and the resulting video is composed of interleaved shots from different cameras.
ChangeIt dataset is provided without any manual annotation (except for the small test set). Use any method for clipping or extracting frames as you wish :)

Thanks a lot!