Remove version: GitHub.ClassroomAssistant version 2.0.3

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

URL: https://github.com/education/classroom-assistant/releases/download/v2.0.3/Classroom.Assistant-2.0.3.Setup.exe
Response Error: The remote server returned an error: (404) Not Found.
(Automated message - build 912)


Pipfile.lock 'forgets' about editable dependencies if they are not top-level deps

I have a dependency which I maintain myself, and installed with pip install -e.
The generated Pipfile shows the dependency and correctly points to https://github.com..., however Pipfile.lock references the Pypi version of the dependency rather than the one I want.
I believe this is because there is another package which depends on the Pypi version of the managed dependency, and pipenv is getting mixed up.
Expected result
If I have a Pipfile like this:
[packages]
graphql-core = {git = "https://github.com/xxxxxxx/graphql-core.git", ref = "6358ad32479"}

then I can run pipenv lock to generate a Pipfile.lock which contains:
"default": {
        "graphql-core": {
            "git": "https://github.com/xxxxxxx/graphql-core.git",
            "ref": "6358ad32479"
        },
}

Actual result
However, if I have another dependency which also lists graphql-core as one of its dependencies, and only specifies a version number, e.g:
[packages]
graphene = "<2.1,>=2.0" # has graphql-core="==2.0" as one of its dependencies
graphql-core = {git = "https://github.com/xxxxxxx/graphql-core.git", ref = "6358ad32479"}

then my Pipfile.lock will look like this - i.e. it has forgotten that this should be an "editable" dependency:
"default": {
         "graphql-core": {
             "hashes": [
                 "sha256:4830699be53f9154273fa15726fc8b0c90bc22bbb8fc7c932586503b3cb9330e",
                 "sha256:539355351343dede3ecb771e0d273a1b72405cb6d64f45bb8f92ecc4d7109af0"
             ],
             "version": "==2.0"
         }.
         "graphene": { ... }
}

how did they ever become editable dependencies if they weren't specified in the pipfile?

all editable dependencies should be specified in the Pipfile.

Sorry, I should have been more clear. graphql-core IS a top-level dependency and it is listed in the pipfile (as an editable dep). But, it is also listed as a dependency inside another package.


.dll from inference libs not transferred into plugin builds when including anira as subdirectory

Hi,
When cloning anira as a subfolder and adding it with add_directory for a specific target, the .dll files associated with torch, onnx, and tensorflow are not transferred into the build artifacts. If think this has to do with PARENT_SCOPE not being set somewhere for these libs but I'm not sure. I'm using Ninja with MSVC provided from VS2022.
Thanks in advance.

I managed to make it work by copying the lines:
file(GLOB_RECURSE INFERENCE_ENGINE_DLLS_ONNX "${ANIRA_ONNXRUNTIME_SHARED_LIB_PATH}/*.dll")
list(APPEND NECESSARY_DLLS ${INFERENCE_ENGINE_DLLS_ONNX})

file(GLOB_RECURSE INFERENCE_ENGINE_DLLS_TFLITE "${ANIRA_TENSORFLOWLITE_SHARED_LIB_PATH}/*.dll")
list(APPEND NECESSARY_DLLS ${INFERENCE_ENGINE_DLLS_TFLITE})

file(GLOB_RECURSE INFERENCE_ENGINE_DLLS_LIBTORCH "${ANIRA_LIBTORCH_SHARED_LIB_PATH}*.dll")
list(APPEND NECESSARY_DLLS ${INFERENCE_ENGINE_DLLS_LIBTORCH})

in the parent CMakeLists.txt file of my plugin. The variable NECESSARY_DLLS only contains the anira dll when passed to a parent folder.

We have updated the handling of shared libraries. It is now possible to access the required shared libraries through the ANIRA_SHARED_LIBS_WIN variable, which replaces the previously used NECESSARY_DLLS.
A demonstration how to access them from a external project (anira as a submodule) can be found in the nn-inference-template plugin.


Refactor OCR Result Processing: Simplified and Detailed Output Support

Description:
This PR refactors the code responsible for handling OCR-recognized results. The main changes include:

Optimization: Reduced redundant code by centralizing common operations.
Functionality: Added support for two output modes:

Detailed Output: Returns OCR text, confidence level, and bounding box coordinates (position and size).
Simplified Output: Returns only the recognized OCR text.

Filtering by Confidence: Results are filtered based on a configurable confidence threshold, ensuring only high-confidence results are processed.

Changes:

Simplified the code by avoiding repeated calls to result.confidence() and result.boundingBox().
Introduced a conditional structure to handle detailed and simplified outputs more efficiently.
Preserved existing functionality while improving clarity and maintainability.

This is great, Thanks!
I will update the version on main and make a PyPi release next.

Btw: The tests currently don't work on the GitHub Actions runner but did locally.


Clock Gating pass

This PR creates a clock gating pass.
It finds any time there is a mux/register where the output of the register feeds into the input of the mux, and replaces it with a specialized register that has verilog amenable to inferring clock gating.
Depends on #857

https://travis-ci.org/github/rdaly525/coreir/builds/669261090 Is passing, but not sure why it isnt reporting back to github


tf.nn.depthwise_conv2d issue with float16 => nan output

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
TensorFlow installed from (source or binary): binary (using docker image tensorflow/tensorflow:1.12.0-gpu)
TensorFlow version (use command below):  ('v1.12.0-0-ga6d8ffae09', '1.12.0')
Python version: 2.7.12
CUDA/cuDNN version: cuda 9.0, cudnn 7.2.1
GPU model and memory: nvidia Tesla V100-DGXS-16GB

Describe the current behavior
tf.nn.depthwise_conv2d_native returns nan when used with float16. Multiple call to same sess.run(op) return different results.
Describe the expected behavior
No nan in the output. Similar value when running same op multiple times.
Code to reproduce the issue
import numpy as np
import tensorflow as tf

with tf.Session() as sess:
    tf_input1 = tf.ones(shape=[1, 22, 28, 28], dtype=tf.float16) / 5.0
    tf_input2 = tf.ones(shape=[7, 7, 22, 1], dtype=tf.float16) / 10.0

    mat = tf.random.uniform( [4096,4096], minval=0, maxval=None, dtype=tf.float16, seed=0, name=None)
    res_mat = tf.matmul(mat, mat)
    #res_mat = tf.no_op()  # un-comment this line and the issue dissapears

    sess.run(res_mat)
    res = tf.nn.depthwise_conv2d_native(tf_input1, tf_input2, strides=[1, 1, 1, 1], padding="SAME", data_format='NCHW')
    v1 = sess.run(res)
    sess.run(res_mat)

    v2 = sess.run(res)
    if np.isnan(v2).any():
       print("issue: nan detected")
    np.testing.assert_equal(v1, v2)

Other info / logs
Issue occurs only on nvidia hardward that support float16 operation (on cpu no issue)
replacing res_mat with tf.no_op() => v1 and v2 are equal without nan
Using float32 => no nan
In my understanding the float16 version of the op does not reinitialized some buffer content.

kjhl

@jvishnuvardhan, I'm not familiar with the inner workings of this function. Can you find someone more knowledgeable to check it out, please?

similar, I cannot really comment on this. Please reroute.

@laurent-george This bug is not related tensorflow. Could you ask it in nvidia community forum? Thanks!
I am closing this as it is better to ask elsewhere and also it is not related to tensorflow. Thanks!

Assigning to @chsigg, as I am not familiar with the depthwise conv code.

@lgeo3 Your reproducer seems to work fine with 1.14 so I suspect this has already been fixed.  I'll speculatively close this issue for now, but please reopen if there is something more to do here.


Call for creating an image from a disk

Probably the function fits better in a different scope? On the other hand the path '/global/images' matches in project scope intrinsically.
Fixes #<issue_number_goes_here> (it's a good idea to open an issue first for discussion)
It's probably quicker to start a discussion from this PR?

[(*)] Tests and linter pass
no new issues introduced, already on master: 6 problems (6 errors, 0 warnings)
[ ] Code coverage does not decrease (if any source code was changed)
no tests have yet been added
[*] Appropriate docs were updated (if necessary)

CLA was signed

Thanks for the PR. I think it should definitely be scoped where you put it (on Project), although I could see it being helpful on Disk as well. What do you think?

Thanks for the quick reply and your suggestions.
Having the call as well on Disk as well sounds good. On the other hand the disk relates to a Zone, which contradicts the 'global' concept. Thus, maybe better leave it as is, to avoid ambiguity?

Thus, maybe better leave it as is, to avoid ambiguity?

👍 SGTM

Looks like node4 is a bit flaky on CI for 'Compute' test case?
I did not investigate further but noticed the node failed before this patch (same timeout reported)

Yeah, don't worry about the CI. I have a commit coming soon. Before I send it, we have to figure out this CLA thing. Can you try commenting "I signed it!" to see if that'll wake up the CLA bot?

I signed it!

thanks for the hint, this re-invoked the bot after adding my github name (which was optional and therefore left blank)

So, I made another change and moved the new method from Project to Compute. (Sorry, it's been a while since I've worked with this API, so I've been a little slow to recall our conventions).
Most of our methods create or return a custom class, such as Disk. We generally expose create{Object}, get{Object}s, and then the custom class has methods to modify it: {Object}.delete(), {Object}.setMetadata(), etc.
What we should really do is create the Image class that has methods that match the upstream API, like image.deprecate(), image.getMetadata(), image.setLabels(), etc.
I'm okay with not implementing those for the time being. However, before I merge, would you be interested in creating them? image.js file would basically be a clone of any of our other files, such as project.js. Then the accessor methods, like compute.getImages(), and compute.image() would be clones of compute.getFirewalls() and compute.firewall() for example.
Let me know what you think!

I signed it!

Creating the Image class makes totally sense. I will look into it

Hey @cbastuck, thanks for doing all of this! Sorry I couldn't review sooner. I think it looks good. There might be a few trivial, convention things I'll take care of. I'll try to do this within the next day or so, and we can release right after.

Hi @stephenplusplus sounds good. Do I have to do something regarding the CLA? Maybe re-authoring all commits with the email address I signed the CLA with? When signing I added my github username which links to the 'authored' address<EMAIL_ADDRESS>Probably this is not resolved?

Actually, I think everything is fine with the bot. It has a known bug where it gets confused when a PR has two authors. Since you were fine before I pushed my commit, I believe we're good. (@alexander-fenster can you confirm?)

Could you please add tests to improve the coverage?

As for the bot, everything is OK as everyone have signed the CLA and as long as all parties are OK with the changes from all parties we are good to go.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

So if all authors sign off the PR we can merge.

I will add tests

I was completely blocked be the daily work and did not find the time to increase the test coverage as promised, sorry.

No problem!


Re: stable evaluation of expit

Based on  #3386, special.expit is implemented like
if x < 0
    a = exp(x)
    a / (1 + a)
else
    1 / (1 + exp(-x))

However, in other languages/frameworks (such as Julia, R), it is simply implemented as
1 / (1 + exp(-x))
I don't think it has numerical instability at x < 0.
According to IEEE, exp(800) would emit Inf, which is OK as 1 / (1 + exp(800) = 1 / Inf = 0. In such case the other brance doesn't provide with extra precisions. exp(-800) / 1 + exp(-800) = 0

That PR flipped the inequality on the first line; prior to that PR, the implementation failed for large x. Maybe you're right, and the first branch of the conditional could just be removed.

in other languages/frameworks (such as Julia, R), it is simply implemented as

By the way, in general it's not cool to read R code and implement it in scipy; it's against licensing terms.

nvm, in c exp(800) overflows and triggers an error.

it seems to work OK in https://github.com/scipy/scipy/pull/6146 and passes tests

Sry, I was confused about c & c++ exception... There is no exception in C.


Writing sbt.version vs scripted filtering

steps

Write a scripted test
Run sbt in that scripted test

problem
sbt will write out sbt.version under project/build.properties which acts as a test filter.
expectation
The, possibly discouraged, practice of running sbt in a scripted test shouldn't implicitly alter the test setup.
notes
sbt version: 1.2.8
Possible solutions:
One possible solution is just before writing the file sbt can detect it's running a scripted test and skip writing.   😕
Or maybe the filtering shouldn't be based on sbt.version in project/build.properties but on something else (let's say, temporarily, sbt.filter.version).  In addition, the filter version could be a bit more flexible than a flat version, like maybe ^1.1, ~1.2, or >=1.0.0, <1.1.0.  See:

https://docs.npmjs.com/misc/semver#caret-ranges-123-025-004
https://docs.npmjs.com/misc/semver#tilde-ranges-123-12-1
https://docs.npmjs.com/misc/semver#ranges

This solution I'd say should include giving a deprecation warning about the old use of sbt.version for filtering, and maybe that warning could also educate the sbt user that you can temporarily change the sbt version with the system property sbt.version: sbt -Dsbt.version=1.2.1.

Would it be possible to prompt the user whether to create build.properties?
No sbt build detected in directory $PATH. Would you like to create one? (y/N):

I have several times accidentally started sbt in the wrong subdirectories (non-scripted) of my workspace and I have to delete the created build.properties.

Yeah, maybe.  I say maybe because we've had some problems in the past with trying to inform/interact with the user and also trying to detect interactive mode.
But that chance would probably be here: https://github.com/sbt/sbt/blob/v1.2.8/main/src/main/scala/sbt/Main.scala#L914

2. Run sbt in that scripted test

When you say "run in that scripted test" do you  mean running in the directory, without first copying it to /tmp?

That's right, people do that.  I'm fairly sure I've done it too, and had to clean up sbt's littering.

Then sbt works as currently designed, so I don't think this is a bug (let's define this as unintended behavior), and I also disagree with the expectation. It's a minor inconvenience of an otherwise good behavior, and I think we should keep it stupid.

I think you'd have a leg to stand on if it were documented both that:

running sbt in a scripted test is (strongly?) discouraged, and that
setting sbt.version in scripted tests filters.


Feat data item customization for v2 data items

Closes ##3427
Summary
New dropdown for V2 data items in dashboard editor
Change List (commits, features, bugs, etc)
DataSeriesFormItemModal
Acceptance Test (how to verify the PR)
when data item has downsample methods the downsample drop down should be shown when editing
Regression Test (how to make sure this PR doesn't break old functionality)

tests here

Things to look for during review

[ ] Make sure all references to iot or bx class prefix is using the prefix variable
[ ] (React) All major areas have a data-testid attribute. New test ids should have test written to ensure they are not changed or removed.
[ ] UI should be checked in RTL mode to ensure the proper handling of layout and text.
[ ] All strings should be translatable.
[ ] The code should pass a11y scans (The storybook a11y knob should show no violations). New components should have a11y test written.
[ ] Unit test should be written and should have a coverage of 90% or higher in all areas.
[ ] All components should be passing visual regression test. For new styles or components either a visual regression test should be written for all permutations or the base image updated.
[ ] Changes or new components should either write new or update existing documentation.
[ ] PR should link and close out an existing issue

I think you may need an update from next. I am unable to push to your branch but I was opening up a PR to fix these snapshots that you could then merge in and it looks like they have already been updated in next.

I tried to link it with monitor UI and seems be no cards are throwing following issues.

Uncaught TypeError: cardRef is not a function
    at ref (Card.js?c82c:439:1)

@kevinsperrine I see that you have changed some card level codes. Maybe did that caused these issues?

Errors reported stem from linking project locally with mismatched yarn versions.


Single RAW Image to Separate Bands (R,G,B,NIR)

Hi,
I would like separate bands from a single RAW image.
#########################

import rawpy
import imageio
path = 'image.nef'
with rawpy.imread(path) as raw:
rgb = raw.postprocess()
imageio.imsave('default.tiff', rgb)

##########################
with that code I can get RGB very well. However, I couldn't separate the Red, Green, Blue and NIR as one by one. Could anyone have any advice?

I would advise you to check out some numpy tutorials. rgb is a 3D numpy array where one dimension is the color channel and based on that you can easily select a single channel.


Bad error message on deallocation when the size is wrong

This code is wrong because ptr has type *mut u8 so the deallocation happens at the wrong size:
#![feature(allocator_api, slice_ptr_get)]

use std::alloc::{Allocator, Layout, System};

fn main() { unsafe {
    let ptr = System.allocate(Layout::new::<i32>()).unwrap().as_mut_ptr();
    ptr.write(5);
    let x = Box::from_raw_in(ptr, System);
    drop(x);
} }

However, the error message does not make this very clear:
error: Undefined Behavior: no item granting write access for deallocation to tag <2885> at alloc1396+0x1 found in borrow stack
  --> /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/unix/alloc.rs:42:9
   |
42 |         libc::free(ptr as *mut libc::c_void)
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no item granting write access for deallocation to tag <2885> at alloc1396+0x1 found in borrow stack
   |
   = help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
   = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information

it is true that the pointer provenance also makes this wrong, but maybe we could check the allocation size first and give the more clear error message.

Actually, we can't really give a better error than this -- free does not take a size parameter that could be used for a check!

What if Miri provided its own std::alloc::System implementation? That way it could verify allocation sizes better.

We don't really have infrastructure to do that -- and also it is somewhat questionable since System is guaranteed to use the system allocator, so you can e.g. use libc::malloc directly to allocate and later use System to free.
Miri does provide its own implementation for Global and checks everything precisely there.


Add eslint-plugin-import, eslint-plugin-node, etc

No real reason this isn't currently included, would be nice to have some of the rules from this package.

What about -node and -unicorn? I know that -node and the -import plugins overlaps in few places, but still - some things works better on one than on the other. As about the unicorn it's fantastic as a whole, but there the rules are more stylistic?

eslint-plugin-node sounds like a good idea, I'm unfamiliar with eslint-plugin-unicorn so will need to check it out.


Port range support

Hi,
Love your work!
was testing your zero-copy supported forked of the realm, but it seems to have performance issues. Fortunately, midori has no issues with performance so far.
any plans on supporting port range forwarding?

Thanks :)
The zero-copy implementation of midori and realm is the same. Did you enable zero-copy with -z or zero_copy=true ?

any plans on supporting port range forwarding?

Not yet. The feature is rarely necessary, and easily abused. I recommend users to write multiple configurations, or write a script to generate the config file.


bump to v0.5.2

release 0.5.1 first

Codecov Report

Merging #254 (2f6ac12) into main (44b8d0b) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #254   +/-   ##
=======================================
  Coverage   69.28%   69.28%
=======================================
  Files         109      109
  Lines        6225     6225
  Branches      496      496
=======================================
  Hits         4313     4313
  Misses       1793     1793
  Partials      119      119

Impacted Files
Coverage Δ

pyEX/_version.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 44b8d0b...2f6ac12. Read the comment docs.

bump straight to 0.6.0


Document addition: node..connect will not add duplicate connections

Calling connect multiple times with the same destination object will not add the relationship more that once. Though this is usually what you want, it might not always be.

Thanks I've updated the docs


Zoom level reset when (auto)saving tasks.json

You lose the current zoom level when editing and saving tasks.json. I expect the current zoom level to be kept as it was before saving.
Repro steps:

open tasks.json
change the zoom level
make any random edit
manually save the file or autosave kicks in
OUTCOME: zoom level is reset

VS Code version: 0.10.6

Indeed, I can reproduce this.

I think I fixed this, @weinand did you try with latest?

@bpasero I saw this with 0.10.6 but I just verified that it no longer occurs in the latest.


Use new endpoints to get and list references

Remove the GetRefs and ListRefs methods in favor of ListMatchingRefs, which performs both functions. The GetRef method remains the same, but now returns a standard 404 error if the reference does not exist.
Closes #1512.

Thank you, @wesleimp!
Merging.


Meeting: 2024-06-03

General information

See our Steno Epi group website for more details, like how-to docs and description of roles
SDCA Common Docs website for general practices, onboarding, help, etc.
Newcomer? Read this section.

Meeting details

Location/Zoom: Skillerum 1+2 (A201-255+A201-257). Physical attendance is encouraged but not required. Recurring Zoom link: https://aarhusuniversity.zoom.us/j/65703475926 (passcode: 935178).
Time: 13:00-15:00

Agenda

Lightning talk by @danielibsen (he has to leave early): Introduction to scoping review on open collaboration and experiences with working reproducibly and using GitHub
New project board explanation, walkthrough, and question period (by Luke)
(~30 min) 3-4 lightning talks (~5-10 min) by:

@Aastedet: Update on osdc package and experiences making an R package
Luke: Documentation-first culture: Examples from the Seedcase Project

Minutes

NOTES

AU Grant writing workshop (course) Fall 2024:
https://au.phd-courses.dk/CourseCatalog/ShowCourse/1481?returnUrl=https%3A%2F%2Fau.phd-courses.dk%2FCourseCatalog%2FShowCourses%2F%3Fquery%3DGrant%26showOpen%3Dtrue%26sortOrder%3DStartDate
ERC:
https://erc.europa.eu/apply-grant/starting-grant
Register with AU Health (before 06-06):
https://health.medarbejdere.au.dk/en/research-support/applications-for-funding/erc-starting-grant
DDEA (deadline 11-09):
https://ddeacademy.dk/grants/


Bump Gateway API v1.1

Trying to redo - https://github.com/knative-extensions/net-gateway-api/pull/729

/test integration-tests-contour_net-gateway-api_main

/assign @izabelacg
This is ready - but the diff is too large so a bunch of checks fail - we can ignore those. For me the main thing is the integration tests pass.

/lgtm

Overriding broken jobs - diff is to large and reviewdog breaks
/override "style / Golang / Lint"
/override "style / Golang / Do Not Submit"
/override "style / Golang / Boilerplate Check (go)"
/override "style / Golang / Boilerplate Check (sh)"
/override "style / suggester / shell"
/override "codecov/patch"

/override "patch"

/override "style / Golang / Lint"
/override "style / Golang / Do Not Submit"
/override "style / Golang / Boilerplate Check (go)"
/override "style / Golang / Boilerplate Check (sh)"
/override "style / suggester / shell"


Varying time tests for logger

Actually, when we are at time 0, why do we print 01:00:00.000, @kshxtij?

localtime vs gmtime semantics

localtime vs gmtime semantics

but we don't want our tests to depend on the timezone, do we?

I see what you mean, but we'd want our logs to display timestamps based on the local timezone

I see what you mean, but we'd want our logs to display timestamps based on the local timezone

So I've tried to fix this put std::chrono::parse seems to not exist yet so this is a future problem. A solution would be to figure out what "00:00:00.000" on 1970-01-01 gets parsed to in local time and then add an offset to that. Not to big an issue, though.

@kshxtij merge whenever you want


Why "y is not a function"?

Describe the bug
JSV is failing (again).

To Reproduce

Run react-cra with published packages

Expected behavior
JSV works

This is a duplicate of https://github.com/stoplightio/elements/issues/1323


openssl genpkey with RSA-PSS does not accept -pkeyopt rsa_keygen_pubexp:65537

As of the current master, the following command fails:
# openssl genpkey -algorithm rsa-pss -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 -out key.pem
genpkey: Error setting rsa_keygen_pubexp:65537 parameter:
000003FF8C0F7720:error:03000093:digital envelope routines:default_fixup_args:command not supported:crypto/evp/ctrl_params_translate.c:578:[action:2, state:4] name=rsa_keygen_pubexp, value=65537

Note that it only fails when generating an RSA-PSS key, but it works for a regular RSA key.
Looking at the translations in crypto/evp/ctrl_params_translate.c  I see the following:
    { SET, EVP_PKEY_RSA, 0, EVP_PKEY_OP_KEYGEN,
      EVP_PKEY_CTRL_RSA_KEYGEN_PUBEXP, "rsa_keygen_pubexp", NULL,
      OSSL_PKEY_PARAM_RSA_E, OSSL_PARAM_UNSIGNED_INTEGER, NULL },

So this translation is only valid for EVP_PKEY_RSA, but not for EVP_PKEY_RSA_PSS.
I would assume this should be changed to:
    { SET, EVP_PKEY_RSA, EVP_PKEY_RSA_PSS, EVP_PKEY_OP_KEYGEN,
      EVP_PKEY_CTRL_RSA_KEYGEN_PUBEXP, "rsa_keygen_pubexp", NULL,
      OSSL_PKEY_PARAM_RSA_E, OSSL_PARAM_UNSIGNED_INTEGER, NULL },

The same is probably true for
    { SET, EVP_PKEY_RSA, 0, EVP_PKEY_OP_KEYGEN,
      EVP_PKEY_CTRL_RSA_KEYGEN_PRIMES, "rsa_keygen_primes", NULL,
      OSSL_PKEY_PARAM_RSA_PRIMES, OSSL_PARAM_UNSIGNED_INTEGER, NULL },

and
    { SET, EVP_PKEY_RSA, 0, EVP_PKEY_OP_TYPE_CRYPT,
      EVP_PKEY_CTRL_RSA_IMPLICIT_REJECTION, NULL,
      "rsa_pkcs1_implicit_rejection",
      OSSL_ASYM_CIPHER_PARAM_IMPLICIT_REJECTION, OSSL_PARAM_UNSIGNED_INTEGER,
      NULL },

Or are there any reasons why those control commands are only valid for regular RSA, but not for RSA-PSS?
Looking at the default provider implementation for RSA-PSS I would say it also accepts those params just like regular RSA.
I am happy to submit a pull request if you confirm that those 3 should also be used for EVP_PKEY_RSA_PSS.

Additionally, crypto/evp/ctrl_params_translate.c also contains functions like
static int get_rsa_payload_n(enum state state,
                             const struct translation_st *translation,
                             struct translation_ctx_st *ctx)
{
    const BIGNUM *bn = NULL;

    if (EVP_PKEY_get_base_id(ctx->p2) != EVP_PKEY_RSA)
        return 0;
    bn = RSA_get0_n(EVP_PKEY_get0_RSA(ctx->p2));

    return get_payload_bn(state, translation, ctx, bn);
}

which are also just checking for EVP_PKEY_RSA. I wonder if it should also allow EVP_PKEY_RSA_PSS here?
Not sure where exactly these functions are used, but if they are called on a RSA-PSS key, I would assume they will fail.

I would assume this should be changed to:
{ SET, EVP_PKEY_RSA, EVP_PKEY_RSA_PSS, EVP_PKEY_OP_KEYGEN,
  EVP_PKEY_CTRL_RSA_KEYGEN_PUBEXP, "rsa_keygen_pubexp", NULL,
  OSSL_PKEY_PARAM_RSA_E, OSSL_PARAM_UNSIGNED_INTEGER, NULL },

Yes.

The same is probably true for
{ SET, EVP_PKEY_RSA, 0, EVP_PKEY_OP_KEYGEN,
  EVP_PKEY_CTRL_RSA_KEYGEN_PRIMES, "rsa_keygen_primes", NULL,
  OSSL_PKEY_PARAM_RSA_PRIMES, OSSL_PARAM_UNSIGNED_INTEGER, NULL },

Yes.

and
{ SET, EVP_PKEY_RSA, 0, EVP_PKEY_OP_TYPE_CRYPT,
  EVP_PKEY_CTRL_RSA_IMPLICIT_REJECTION, NULL,
  "rsa_pkcs1_implicit_rejection",
  OSSL_ASYM_CIPHER_PARAM_IMPLICIT_REJECTION, OSSL_PARAM_UNSIGNED_INTEGER,
  NULL },

No. This one is not relevant to PSS.

I am happy to submit a pull request if you confirm that those 3 should also be used for EVP_PKEY_RSA_PSS.

Yes please!

which are also just checking for EVP_PKEY_RSA. I wonder if it should also allow EVP_PKEY_RSA_PSS here?

Yes, probably they should.

Not sure where exactly these functions are used, but if they are called on a RSA-PSS key, I would assume they will fail.

Used for example when calling EVP_PKEY_get_bn_param() when supplying a legacy (i.e. not provider based) key and getting the param (in this case) OSSL_PKEY_PARAM_RSA_N.

I am happy to submit a pull request if you confirm that those 3 should also be used for EVP_PKEY_RSA_PSS.

Yes please!

See https://github.com/openssl/openssl/pull/21818

Not sure where exactly these functions are used, but if they are called on a RSA-PSS key, I would assume they will fail.

Used for example when calling EVP_PKEY_get_bn_param() when supplying a legacy (i.e. not provider based) key and getting the param (in this case) OSSL_PKEY_PARAM_RSA_N.

Do we have any tests that work with legacy keys where this could be tested?

A grep for cases where EVP_PKEY_RSA is used but not EVP_PKEY_RSA_PSS revealed the following places where handling of RSA_PSS might also be missing:
In crypto/evp/p_lib.c:
int EVP_PKEY_can_sign(const EVP_PKEY *pkey)
{
    if (pkey->keymgmt == NULL) {
        switch (EVP_PKEY_get_base_id(pkey)) {
        case EVP_PKEY_RSA:   <--- shouldn't RSA_PSS keys also return 1 here, as RSA-PSS keys can sign?
            return 1;
# ifndef OPENSSL_NO_DSA
        case EVP_PKEY_DSA:
            return 1;
# endif
   ...

and
static void detect_foreign_key(EVP_PKEY *pkey)
{
    switch (pkey->type) {
    case EVP_PKEY_RSA:
        pkey->foreign = pkey->pkey.rsa != NULL
                        && ossl_rsa_is_foreign(pkey->pkey.rsa);
        break;
   ....

This is called from EVP_PKEY_assign() and I guess this should also work for a pkey object that has been initialized as RSA-PSS.

A grep for cases where EVP_PKEY_RSA is used but not EVP_PKEY_RSA_PSS revealed the following places where handling of RSA_PSS might also be missing:

Yes...those look missing too.
Probably very few people are using RSA_PSS with an ENGINE so it hasn't really impacted anyone.

Should I fix those, too, in a 3rd patch in PR https://github.com/openssl/openssl/pull/21818 or do you want me to open a new PR ?

This one is solved by PR https://github.com/openssl/openssl/pull/21818 and PR https://github.com/openssl/openssl/pull/21819 and can be closed.


Deleted Musicplayer

Changes to a cask
Deleted an existing cask

[x] Commit message includes cask’s name (and new version, if applicable).
[x] brew cask audit --download musicplayer is error-free.
[x] brew cask style --fix musicplayer left no offenses.

For future reference, it seems to be dead.


docs(auth-kit): Update docs for the new SafeAuthPack

This PR updates the docs for the auth-kit:

https://github.com/safe-global/safe-docs/issues/262
https://github.com/safe-global/safe-docs/issues/263

@tanay1337 thank you for your review! I took all the suggestions and adapted the PR to match the current Safe docs structure. We can continue here:
https://github.com/safe-global/safe-docs/pull/289


Fix/hotifx prod performance

Fresh off the presses - need to merge this to fix that high octane bug in that one instance. I promise this is good to go no need to check.

seems legit

seems legit

Dabby Duck is the hero we deserve.


Example 3 for dynamic membership groups is incorrect

Example 3 claims to illustrate how to select users without any assigned plans. If that is indeed the intent, the query is wrong. (It should probably say assignedPlan.servicePlanId -eq null, but someone from the dynamic membership team should confirm.)

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 5f7a4070-1be8-5f1f-0092-68fb6b9e2305
Version Independent ID: cd34312f-b507-affe-555e-1e0c63e7d494
Content: Rules for dynamically populated groups membership - Microsoft Entra ID
Content Source: docs/identity/users/groups-dynamic-membership.md
Service: entra-id
Sub-service: users
GitHub Login: @barclayn
Microsoft Alias: barclayn

@psignoret
Thanks for your feedback! We will investigate and update as appropriate.

@psignoret
Thank you for bringing this to our attention! We've initiated a pull request (PR) to address this issue. Once the author reviews the changes, they will be merged, and the updates should go live shortly. Your diligence is much appreciated!


Unable to differentiate between persistence failures and serialization issues

In our application we use proto serializer for serializing domain events. What we have observed is the plugin/akka persistence invokes onPersistFailure() instead of onPersistRejected() for serialization issues. I believe this is because of the following code in SerializationProxy.serializeAtomicWrite()
if (xs.exists(.isFailure)) xs.filter(.isFailure).head.asInstanceOf[Try[Iterable[SerializationResult]]] // SI-8566
Is this something that can be fixed or do we have any alternative way to differentiate the two failures?
We are using version 2.2.19 of this plugin and version 2.4.4 of akka persistence

I have created a pull request for this issue #52

Hi, thanks for your contribution! I'm kinda busy right now so contributions from the community is very much appreciated in helping the plugin getting better and better! I've merged the PR and will release it shortly. Thanks again!


Main menu - Remove options button and panel

Remove options button and panel due to empty content.

Fixed


Resources: Spelling fix for responses.js

I did a few searches and couldn't find any other uses in the repository, so I think we're safe to change this. E6n will most likely be a near-future use once I get the triggers working for that.

Thanks for fixing this NEAR DISASTER


[STC] STC loss ascends while training

Hello, I'm training ASR model with STC Loss and letter-to-word encoder like below.
But when I progress training, STC Loss ascended and became 'Inf' after 12000 step.
Is there any miss in my implementation?
Any help would be appreciated.
Thank you.
training args:

num_gpu : 4
audio_length_per_gpu : 160s
lr : 0.0001~0.001
use FullyShardedDataParallel, mix_precision=off

#   max_word_length : 10
#   n_letter_symbols : 69 (blank + pad + korean)
#   n_word_symbols : 12158 (blank + korean_morph 97% in corpus)

#   blank_idx=0, p0=0.05, plast=0.15, thalf=16000

self.criterion = STC(
    blank_idx=self.cfg.blank_idx,
    p0=self.cfg.p0,
    plast=self.cfg.plast,
    thalf=self.cfg.thalf, reduction="mean"
)

#   model_output : Tensor[batch_size, max_frame_length, n_letter_symbols*max_word_length]
#   self.l2w_matrix : Tensor[n_letter_symbols*max_word_length, n_word_symbols]

word_level_output = model_output @ self.l2w_matrix

#   word_level_output : Tensor[batch_size, max_frame_length, n_word_symbols]

word_level_output =  F.log_softmax(word_level_output.transpose(1, 0), dim=-1)

loss = self.criterion(word_level_output, word_labels)

Hi,
Could you give a few details about the dataset you are using and how partial labels are created. Also, Have you tried CTC with the same training setup?
As a sanity chec,k you can output Tensor[batch_size, max_frame_length, n_word_symbols] from the model directly. This will make sure there are no errors with l2w_matrix creation.
Just to make sure have you made sure word_labels are 1-indexed to account for blank symbol ?

Hello. @vineelpratap
I'm using mixed korean asr dataset(num_audio=3180000) from aihub.
How I get l2w_matrix is like below
Now I'm trying to train with ctc(gtn_application). Thank you for your advice.
    def get_l2w_matrix(self):
        # letters = {'BLANK':0, 'PAD':1, 'letter1':2, .....}
        # morphs = {'BLANK':0, 'word1':1, .....}

        with open(self.config.letter_dict_path, 'r', encoding='utf8') as j:
            letters = json.load(j)
        with open(self.config.word_dict_path, 'r', encoding='utf8') as j:
            words= json.load(j)

        E_matrix = torch.zeros(self.config.max_word_length, len(letters ), len(words)).bool()

        # set one hot vector
        for word, word_idx in words.items():
            if word == 'BLANK':
                E_matrix[0, letters['BLANK'], word_idx] = True
            else:
                for letter_idx, letter in enumerate(word2letter(word)):
                    E_matrix[letter_idx, letters[letter], word_idx] = True

        # padding
        padding_location = ~torch.any(E_matrix, 1)
        E_matrix[:, letters['PAD'], :][padding_location] = True

        torch.save(E_matrix, self.config.e_matrix_path)

        return E_matrix.half().view(-1, E_matrix.shape[-1])

When I changed  thalf 16000 -> 8000, and model to output word level directly, STC loss increases slightly after 8k step.
More details about by model is,

I input normalized wav directly to wav2vec2.0 like model(WavLM)
change total stride of wav feature extractor 320 -> 1280 (80ms per output frame)
number of word per second in audio data is 0.5~6.5 (much blanks in audio)


Investigate how to communicate and work with PyPi (and later other packages registered for Python)

Part of https://github.com/John15321/rust-pip/issues/1
Investigate how to communicate with PyPi in order to get the necessary information about packages, send requests for downloading them etc
Please create an .md file describing your findings so that other after reading it can understand what you discovered
preferably name the file PyPi_communication.md or something like that

I will be working on it


⚠ Bofrost.at has degraded performance

In c42e56b, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 2207 ms

Resolved: Bofrost.at performance has improved in 5e7a7d9.


cl12 cl20 cl21 cl22 bruteforce macro_unary float tests can't fail

(migrated from gitlab)
The TestFloat() function in macro_unary.c has an exit block that overwrites any incoming errors. Failing test cases will be logged but the overall test reports PASSED if the cleanup code completes without error. Test cases affected: isfinite, isinf, isnan, isnormal, signbit, not

Fixed on all branches, closing.


Will it work from localhost?

Testing it on my pi, it logs that the message was sent, but doesn't show up in the ifttt activity log

Hi, can you show your code example? Which version of nodejs do you use?

They came through eventually, must have gotten queued by ifttt.


Uploader

Describe the bug
On the Chrome browser, uploader does not upload the file with same name again, in the same folder, even if the contents of file or size is different and configured to over-write existing file.
To Reproduce
go to following link in chrome
https://google.github.io/mesop/components/uploader/
Upload an image file.... SUCCESS
rename the file something else and chooses another image with the previous file name you uploaded.
Try uploading again......DOES NOT upload
Try same in Safari it works
Expected behavior
It should upload the file as expected....
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop System Info

OS: [e.g. iOS]
Browser Chrome, works with safari
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Great catch. I think this should be easy to fix. I'll try to get a fix for this some time this week.

Thanks a lot, I really appreciate that. It will make my users happy.
Best Regards


Beta/1.6.0

What
Release:

Early drop-out from content-type check for large files and known file types.
Fix for OOM error.
Nomad healthcheck.

How to review
Sanity check.
Who can review
Anyone.

grrr.


ci: add canary deploy

this change will publish canary versions automatically when things get merged on the main branch.
e.g.
 - @mux-elements/mux-audio => 0.5.6-canary.0+df61ba9
 - @mux-elements/mux-player-react => 0.1.0-canary.0+df61ba9
 - @mux-elements/mux-player => 0.1.0-canary.0+df61ba9
 - @mux-elements/mux-video-react => 0.4.12-canary.0+df61ba9
 - @mux-elements/mux-video => 0.6.2-canary.0+df61ba9
 - @mux-elements/playback-core => 0.6.1-canary.0+df61ba9

Lerna does make things simpler if it already has an option for it.
Also, I think we need to add the registry-url like in https://github.com/muxinc/elements/blob/main/.github/workflows/cd.yml#L25-L26


Featured Event Toggle Not Working

In the Event List block the Display Featured Events Only checkbox does not properly save its state.  It appears the error is on line 127 of the block's form.php - it's setting the value of the checkbox to $eventListTitle rather than 1.  This causes intval() in the block's save function to return a 0, which is then stored.  Replacing $eventListTitle with 1 in quotes on line 127 of form.php fixed the error on my site.

This has been fixed in GitHub and will be released in 9.0.2


Dev Container Features: Taking User Input in PostCreateCommand

We're hosting our own internal dev container images and features, and it works great (good job!). However, there are some custom features we're making that require user input, and we're seeing some odd behavior.
First things first, is taking user input during the PostCreateCommand a supported operation? Is there any reason our bash scripts can't perform read during that step?
The behavior that we're seeing is when two separate features perform read in their PostCreateCommand script. The first feature works great and the user input is successfully processed. However, the second feature's read will always immediately exit when the user starts typing. This happens even if we flush the input buffer ahead of time.
Any ideas?

Thanks for opening this @fightingsleep.
My colleague @joshspicer just spun up a great example that runs an interactive script in Codespaces on startup: https://github.com/codspace/interactive-postattach.

Note that in Codespaces, postAttachCommands _from Features  fire non-interactively. We're working on bringing this functionality to Feature-contributed postAttachCommands, but for now you'll need to do something similar to my demo above for interactivity. (NOTE: the dev container extension may behave differently, I'm not too sure 😄 )

That's a fair point, I see what you're saying 🤔.
How I would do this is use the postAttachCommand, and at start up check for some state in the container (a config file written, or even a marker file), exiting if it already has the info it needs and prompting interactively otherwise.  I'm curious if that would solve your issue, or if the same problem still persists.
FWIW with secrets, we do have some incoming secrets work to provide 'first class support'. Clients (i.e Codespaces, Dev Containers extension) will be able to hook into to securely prompt for secrets.  That's not fully here yet but something that I think future readers of this issue could find useful :D

@joshspicer I'll give postAttachCommand and marker files a shot for now. It does feel a bit more like a workaround than a solution though. I really think it would be useful to have the ability to prompt the user for input during the one-time setup phase (without having to manually perform this check every time you spin up / attach to the container).
I'll let you and your team make that decision though 😀
Looking forward to that first class secret support! That will be super helpful

That's good feedback, @chrmarti is it intended that several postCreateCommands can work interactively in the Dev Containers extension?  This definitely isn't supported in Codespaces, but perhaps is something we could look to rework if there is an alignment issue.


Create ArrayReverse.py

I have created an python code to Reverse an Array in place.
Please review and merge it as an Hacktoberfest'22 Contribution.
Thanks for your Support.

What a great code 😄 !
Thank you for contributing! 🚀

Thanks Man 😊


FFG-SWRPG-Dice-Roller Domain has expired

The domain containing all the assets for this API addon has expired, I can't find any contact details for the team building it though. An alternative source should be found for the images.

I can confirm it looks like galacticcampaigns that hosts the images has gone down. All the dice images aren't working for me either


Added support to execute Robot scipt from mac OS

Signed-off-by: tarukumar<EMAIL_ADDRESS>Added support to execute the "run_reboot_test.sh" script to support MAC OS(Darwin). Added another block for mac os. Earlier script was not supported on mac OS due to below reason:

kernel name for mac os different
mktemp in mac os doesn't support "-p".

Verification for mac was done on my PC and for linux i used Linux VM

Verified by ci job 201. There are 2 failed TCs which is not caused by PR


fixed import errors with !pip installs, updated torch to 1.11

I was able to keep the imports pretty lean (one more was needed) and I want to keep the old version of torch just for reproducibility's sake.  Can you let me know if the notebook runs for you?


Feature Request - Set default line width

Please add a new feature to set the default line width for line graphs. I find the default to be too wide and I never mix widths on a graph so a single setting for all traces would be useful.

Added to V1.0.51.
lineWidth: 10.0

all great.


Calendar is just showing up Purple.

Not sure what I'm Doing wrong?
and i have Calendar configured and working.
{
module:"MMM-CalendarExtMinimonth",
position:"top_right",
defaults: {
calendars: ["Shared"],
source: "CALEXT",
},
},

Hmmm. Can you share your calendar with me to inspect<EMAIL_ADDRESS>

I cant share that one. But let me look at creating another one. What is your thought on needing to see the calendar?

Well, I have nothing to say before looking inside the real data.(and configuration)
Maybe it's normal because each day might have really enough events to be displayed as purple.

Ya this is a brand new calendar and was showing up purple before I got it configured.

There was a bug when it is used with CALEXT together. (The reason was old deprecated momentJS function was used.) I've fixed. update and try again.

Yep thanks looks like it is working. I want to thank you on resolving this issue.


Improving documentation of dirmult_lme

Hi @aryankeluskar Congratulations on completing the dirmult_lme function! This PR exposes your work in the documentation of scikit-bio, such that users will see it. Like this:

@mortonjt I also cleaned up the docstring a little bit to make some terms more accurate or linking to correct sources. However, there are a few things I am unsure:

There are multiple print calls in the code. Are these intentional?
When I ran the doctest manually, it threw a ton of warnings like: "ConvergenceWarning: Maximum Likelihood optimization failed to converge. Check mle_retvals". It might reduce user experience. I wonder if it is an issue of the toy dataset?
Wrapper parameter inconsistency: In the current function, method is a parameter of MixedLM.fit. re_formula and vc_formula are parameters of MixedLM.from_formula (PS: they were not in the docstring; I added them). fit_kwargs is for MixedLM.fit. **kwargs are for MixedLM.from_formula. This might be confusing.

How about renaming **kwargs as model_kwargs, merging method into fit_kwargs, and merging re_formula and vc_formula into model_kwargs? Such that we have -3 extra parameters.

Interface inconsistency. In the relevant function dirmult_ttest, the parameters are table, grouping, treatment, reference... But in the current function, the parameters are formula, data, metadata, groups... I think that a consistent interface will be helpful. But I understand that lme calls statsmodels therefore the interface may have to be different.

Please complete the following checklist:

[x] I have read the contribution guidelines.

[ ] I have documented all public-facing changes in the changelog.

[ ] This pull request includes code, documentation, or other content derived from external source(s). If this is the case, ensure the external source's license is compatible with scikit-bio's license. Include the license in the licenses directory and add a comment in the code giving proper attribution. Ensure any other requirements set forth by the license and/or author are satisfied.

It is your responsibility to disclose code, documentation, or other content derived from external source(s). If you have questions about whether something can be included in the project or how to give proper attribution, include those questions in your pull request and a reviewer will assist you.

[x] This pull request does not include code, documentation, or other content derived from external source(s).

Note: This document may also be helpful to see some of the things code reviewers will be verifying when reviewing your pull request.

Hi @mortonjt Thanks for reviewing! I have addressed your comments and edits. Can you please also take a look at my comments on top of this PR? Thanks

@qiyunzhu Thank you very much for the kind words! I think I have an answer to the ConvergenceWarning. It indeed is stemming from the toy dataset, since using the sample datasets (such as dietox from R) provided by statsmodels didn't throw these warnings.

Regarding the print calls, I believe they are intentional. All of them are elaborating in case of an error and will not be invoked in most cases. First set of them are used to print the indices of data and metadata if they don't match from 2413-2414. Second set print the exact error message in case it is beyond our handling from 2174-2179. The latter exists in case the model returns a value other than a valid number or NaN, which is very unlikely.

@aryankeluskar Thanks for the explanation on print. My understanding of the typical practices of developing a library is that it shouldn't print anything at all. The library means to be programmatically called by other code, instead of interacting with an end user. Any feedback should be delivered in the form of exception or warning. But I would like to consult with @mortonjt on this too.

Thanks for catching this. Yes we should have warnings thrown
https://github.com/scikit-bio/scikit-bio/blob/3697ec490031e69e573361e68830a900ae664738/skbio/stats/composition.py#L2179-L2184

@aryankeluskar and I discussed today. I had better understanding of the rationale behind this function. The additional edits should address all issues. Most importantly, the modifications ensure the greatest possible consistency between the interfaces of dirmult_lme and dirmult_ttest. Also, the original unit tests were very slow due to 128 draws. They are accelerated with fewer draws in this modification. Finally, the drawing process has been optimized to avoid format-conversion overheads.

Let me merge this because otherwise the unit test is lengthy and unreadable.


Fix typo in Arabic language

fixing a small typo.

Q
A

Is bugfix?
❌

New feature?
❌

Breaks BC?
❌

Tests pass?
✔

Fixed issues

Thanks!


Change ARStep to reverse AR parameters

This fixes #440 by choosing to reverse the AR parameters, whilst maintaining everything else as is.
To cover this bug I've also added a unit test that would have picked up swapping the AR parameters in an AR(2) process by comparing a long single draw to its known theoretical stationary distribution.
Closes #440

@seabbs had a good shout about not doing the reverse at run time. I've added a rev_damp_priors field to AR. The only real gotcha now is that you have to condition on rev_damp_prior parameter if you want to do conditional AR things now, but I think this is an ok price for not doing a reverse call every time we call the model.

Nice change. I think we should pause on this until #438 is in (ideally today) as that changes the names and structure of a lot of the AR model.

Nice change. I think we should pause on this until #438 is in (ideally today) as that changes the names and structure of a lot of the AR model.

Right. But I'm going to merge this into some other branches because this is the actual likely problem which makes #436 hard to resolve (I think!).

ah okay. Shall we just merge this in then and I will deal with it in #438

I'm quite unclear why this would have a performance impact?

It shouldn't in general (right?) but in the replication notebook we effectively had strong priors on the AR parameters in the wrong order (which strongly effects the prior variance of the process etc).

This is blocked by #438

This is unblocked now but because of all the annoying changes from me we might want to close and start again

Agreed!


Building tensorflow from the sourcefile

Please go to Stack Overflow for help and support:
https://stackoverflow.com/questions/tagged/tensorflow
If you open a GitHub issue, here is our policy:

It must be a bug or a feature request.
The form below must be filled out.
It shouldn't be a TensorBoard issue. Those go here.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Redhat

TensorFlow installed from (source or binary):
source

TensorFlow version (use command below):
1.0

Python version:
2.7.14

Bazel version (if compiling from source):

GCC/Compiler version (if compiling from source):
4.8.5

CUDA/cuDNN version:
8/5

GPU model and memory:

Exact command to reproduce:

You can collect some of this information using our environment capture script:
https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh
You can obtain the TensorFlow version with
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
Describe the problem
Describe the problem clearly here. Be sure to convey here why it's a bug in TensorFlow or a feature request.
Getting the following error:
ERROR: /home/amalik/tensorflow/tensorflow/tools/test/BUILD:81:1: Traceback (most recent call last):
File "/home/amalik/tensorflow/tensorflow/tools/test/BUILD", line 81
tf_cc_logged_benchmark(name = "cast_op_benchmark", target...")
File "/home/amalik/tensorflow/tensorflow/tools/test/performance.bzl", line 23, in tf_cc_logged_benchmark
list((set(tags) + set(["benchmark-tes..."])))
File "/home/amalik/tensorflow/tensorflow/tools/test/performance.bzl", line 23, in list
set(tags)
The set constructor for depsets is deprecated and will be removed. Please use the depset constructor instead. You can temporarily enable the deprecated set constructor by passing the flag --incompatible_disallow_set_constructor=false.
ERROR: package contains errors: tensorflow/tools/test.
ERROR: /home/amalik/tensorflow/tensorflow/tools/test/BUILD:86:1: Traceback (most recent call last):
File "/home/amalik/tensorflow/tensorflow/tools/test/BUILD", line 86
tf_py_logged_benchmark(name = "rnn_op_benchmark", target ...")
File "/home/amalik/tensorflow/tensorflow/tools/test/performance.bzl", line 52, in tf_py_logged_benchmark
tf_cc_logged_benchmark(name = name, target = target, benchm..., <2 more arguments>)
File "/home/amalik/tensorflow/tensorflow/tools/test/performance.bzl", line 23, in tf_cc_logged_benchmark
list((set(tags) + set(["benchmark-tes..."])))
File "/home/amalik/tensorflow/tensorflow/tools/test/performance.bzl", line 23, in list
set(tags)
The set constructor for depsets is deprecated and will be removed. Please use the depset constructor instead. You can temporarily enable the deprecated set constructor by passing the flag --incompatible_disallow_set_constructor=false.
ERROR: /home/amalik/tensorflow/tensorflow/core/kernels/BUILD:58:14: Traceback (most recent call last):
File "/home/amalik/tensorflow/tensorflow/core/kernels/BUILD", line 53
config_setting(name = "xsmm_backward", values = {..."})
File "/home/amalik/tensorflow/tensorflow/core/kernels/BUILD", line 58, in config_setting
{"define": "tensorflow_xsmm=1", "define": "tensorflow_xsmm_backward=1"}
Duplicated key "define" when creating dictionary.
ERROR: package contains errors: tensorflow/core/kernels.
ERROR: error loading package 'tensorflow/core/kernels': Package 'tensorflow/core/kernels' contains errors.
Note: Tried other plateforms but could get any reply. I am not sure its a bug or else. I apologize if I am using the wrong forum. I am trying to install tensorflow from source so I can use MPI
Source code / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.

What's your bazel version? That's very important

It is 0.7.0. Thanks

Which Tensorflow version should be best for MPI? I am using 1.0.

It's too new. You should either upgrade your TF or downgrade your bazel.  You can get the matching bazel version in tensorflow/tools/ci_build/install/install_bazel.sh.
Please close this issue, as this is not a bug of tensorflow.

Thanks

@gunan any idea?
@abidmalik1967 what is the bazel version you are using?


Keep column order

It seems like kysely-codegen sort columns by alphabetically. Why? We have more then 50 columns, and the order is important for readability. Please consider to keep order from INFORMATION_SCHEMA.

It seems like this line is the cause of problem. https://github.com/koskimas/kysely/blob/23f004d4ebf55352b6d024b3702843dee6b41bb7/src/dialect/mysql/mysql-introspector.ts#L48

I found an issue https://github.com/koskimas/kysely/issues/146

Well found!
However, kysely-codegen also includes code that sorts the tables and columns (and enums I believe) alphabetically. Why? Two reasons:

This makes the internal tests consistent—they won't fail due to insertion order or upserts, as long as the data is the same
The best default for readability (if you don't have your own specific readability rules) is to order them alphabetically

With that said, it may be a good idea to keep the sorting opt-in or opt-out, so this may be reopened in the future.

Note that orderBy('column_name') and orderBy('ordinal_position') are both consistent, so tests will never failed as long as the data is the same.

Of course, if the ordering is part of the tests, they will never fail. My gripe was that the tests kept failing as I were writing and changing them, but as long as the ordering is stable, I guess this may not be a problem.
Regardless, I have not considered column order to matter by design, so I have not taken it into account, but since others may find importance in column order, I should probably make this configurable.


Ads do not show in Maui after last Maui update

Since the last update of Visual Studio to version 17.6.2 an app I am developing stubbornly refuses to show AdMob ads.
I have set an event handler for the ad loading on android, and checked that the ad s do not load at all.
The ad I am trying to load is the standard Google banner test ad ca-app-pub-3940256099942544/6300978111.

Me also same problem, after updating vs2022 to 17.6.2 ads not showing.
[reenguardfinde] Accessing hidden method Lsun/misc/Unsafe;->compareAndSwapObject(Ljava/lang/Object;JLjava/lang/Object;Ljava/lang/Object;)Z (greylist, linking, allowed)
[Ads] Ad failed to load : 3

Same issue here

I'm going to close this issue ad it's not an issue but just in case someone else sees this:
Ad failed to load : 3 means that everything works fine just there are no ads to show.
Exactly the error is: "ad request was successful but there are no ads to be served."


Hello, I would like to ask, when you are training the model, do you only use the first round of dialogue from the ultrachat_200k?

def load_and_process_data_ultrachat(dataset_name, split):     try:         dataset = load_dataset(dataset_name, split=split)         reformatted_data = [{             'generated': [message['messages'][0], {"role": "assistant", "content": ""}],              'real': [message['messages'][0], message['messages'][1]]         } for message in dataset]         return reformatted_data     except Exception as e:         logging.error(f"Error loading or processing dataset: {e}")         return []

Yes. Only the first round of real dialogue dataset is sampled from ultrachat 200k.


Support DI registration methods

Via an option on the MapperAttribute, extension methods to the IServiceCollection should be generated to add a single mapper by its name or all mappers at once to the service collection. By default these should be added as singletons, but an overload with a service lifetime should be available.
Example of how the generated code could look like:
namespace Microsoft.Extensions.DependencyInjection;

public static class MapperlyExtensions
{
    public static IServiceCollection AddMyMapper(this IServiceCollection services)
        => services.AddMyMapper(ServiceLifetime.Singleton);

    public static IServiceCollection AddMyMapper(this IServiceCollection services, ServiceLifetime lifetime)
        => services.Add<MyMapper>(lifetime);

    public static IServiceCollection AddMappers()
        => services.AddMyMapper();

    public static IServiceCollection AddMappers(ServiceLifetime lifetime)
        => services.AddMyMapper(lifetime);
}

tbd: currently, Mapperly only generates implementations for user defined partial methods and Mapperly-internal private methods. This would generate public accessible code, without the user defining it. Does that have any implications on usability?

I can see you want to go with source code generation consistently for everything, makes sense. In this case how would you feel about following approach. This would be analogical to the mappers, and would address your concern about generating public code w/o user awareness.
// mapperly abstractions - probably better name
public class AddMappersAttribute : Attribute
{
  public AddMappersAttribute(ServiceLifetime lifetime, params Type[] mappers)...
}

// user code
public static partial class MyComponentExtensions
{
  public static IServiceCollection AddMyComponent(this IServiceCollection services) => services
    .AddTransient<IFoo, Foo>()
    .AddTransientMappersNameDoesntMatter()
    .AddSigneletonMappers();

  [AddMappers(ServiceLifetime.Transient, typeof(CarMapper), typeof(CustomerMapper))]
  private static partial int AddTransientMappersNameDoesntMatter(this IServiceCollection services);

  [AddMappers(ServiceLifetime.Singleton, typeof(ShopMapper), typeof(InvoiceMapper))]
  private static partial int AddSingletonMappers(this IServiceCollection services);
}

this would generate code similar to this one
public static partial class MyComponentExtensions
{
  private static partial int AddTransientMappersNameDoesntMatter(this IServiceCollection services) => services
    .AddTransient<CarMapper>()
    .AddTransient<CustomerMapper>();

  private static partial int AddSingletonMappers(this IServiceCollection services) => services
    .AddSingleton<ShopMapper>()
    .AddSingleton<InvoiceMapper>();
}

It would be nice to have the possibility of registering mappers with the DI container extended with some form of abstractions to allow taking dependencies on some known contract instead of the individual mapper implementations. It should be possible to source generate interface implementations, the MVVM toolkit already does something like this.
It could look something like this:

there is an interface in the included abstractions having source and destination type parameters and a single mapper "Map" method.
user defined partial methods would define which Interfaces the mapper would implement. This could be marked by an attribute on  the method to expose it through an interface (not sure if this could already also enforce the naming of the method to match)
registrations are done against all the implemented interfaces, respecting the lifetime chosen

On a second thought, I think Mapperly should keep the principle of only generating implementations for user defined partial methods and Mapperly-Private methods. This keeps the features discoverable and less magical to the user. Closing this for now.

Oh man! I just found this cool library. Thank's for building it <3
Sadly DI support was rejected too soon, I guess.
I'm comming from AutoMapper (just as everybody, I guess).
There I could create some generic methods mapping just anything really.
A very simplified example:
class EndpointBase<TRequest,TCommand>(IMapper mapper)
{
   public TCommand CreateCommand(TRequest req) =>
       mapper.Map<TRequest, TCommand>(req)
}

I think this could be achieved by this lib, too.
Let's have an interface for mappers:
public interface IMapper { }
public interface IMapper<TSource, TTarget> : IMapper
{
   TTarget Map<TSource, TTarget>(TSource source);
}

This interface could be added by the source gen to each mapper class. The Map method could just call the user defined MapCarToCarDto function ;)
For DI to work, there should be a AddMappers extension method.
Maybe the user can decide on which extension class he would like the method to be generated.
static partial class ServiceExtensions
{
   [MapperExtensionMethod(ServiceScope = Scoped)]
   public static partial IServiceCollection AddMyMappers(this IServiceCollection services);
}

The source gen would then create a method like this containing all the mappers:
public static partial IServiceCollection AddMyMappers(this IServiceCollection services)
{
   services.AddScoped<IMapper<Car, CarDto>, CarMapper>();
   services.AddScoped<IMapper<Cat, CatDto>, OtherMapper>();
   services.AddScoped<IMapper<Dog, DogDto>, OtherMapper>();
   return services;
}

This would enable users of the lib to register mappers with DI and consume them via the IMapper<TSource, TTarget> interface.
I know there are some edge cases, where other method signatures are supported for the mapping methods.
Like the one with an existing target object.
There could be more interfaces that encapulate them as well, like ITargetMapper or the IMapper interface defines another method like Map(TSource source, TTarget target) and the source gen generates a method which throws an NotImplementedException for all missing mapping methods.
@latonz What do you think? Should I prepare a PR?

Thanks for your feedback and willingness to prepare a PR! 😊 New contributors are always welcome.
I'm not really sure if I understand your comment correctly. If not, could you provide an example of what the generated code would look like for the feature you are requesting?
As far as I understand your comment there are two requests:

an alternative configuration API surface via method invocations instead of attributes and method signatures
generating dependency injection service registration code

My two cents:

Source generators have a hard time using regular c# statements as configuration API (the source generator only has access to the semantic model and to the syntax, executing code at compile time is currently not in the scope of source generators...). Furthermore source generators need to know exactly what to code to generate at compile time. This means Mapperly needs to know all types at compile time, which can be mapped by the application at runtime. Mapperly achieves this by a combination of attributes and method signatures which are both well accessible in the semantic model. An approach with method calls to configure the mappings would either need isolation and compilation of the given calls or a complex source analysis. Both would probably result in an explosion of the complexity of Mapperly‘s source code. If you have a good idea on how to implement such a configuration API in a .NET source generator I would be very excited to hear about it 😊  See also https://github.com/riok/mapperly/discussions/1365#discussioncomment-9914655.
The complexity of the service collection registration for a Mapperly mapper should be quite trivial and no different than for any other class. It should be as simple as services.AddSingleton<MyMapper>(). If one really wants to abstract this away with a source generator it should even work with other source generators (e.g. AutoRegisterInject, I never tried it though).

Before you start working on an implementation, we should discuss this matter further until we have a concrete plan of what should be implemented and how.


feat: Add transform functions for Collector Metric Exporter

Summary
This PR addresses Issue #1109.
Currently, the Collector Exporter only sends traces to the collector. In Python, Java, and Go, for example, metric exporting already exists for the collector. This PR adds that feature. I created two versions, one for Node and the other for browser. The node version uses gRPC, and the browser uses Beacon or XHR. It's up to date with all previous collector exporter commits.
What this PR does
This is the 5th part of this long PR.
This PR creates the metric versions of transform functions, transforming objects to valid formats for the collector.
These are more straightforward and are essentially direct copies of the trace versions of the functions.

toCollectorLabels
groupMetricsByResourceAndLibrary
toCollectorExportMetricServiceRequest
toCollectorResourceMetrics
toCollectorInstrumentationLibraryMetrics

These functions are more complicated, where I had to make decisions for converting types (e.g. determining the appropriate type and temporality for a given MetricKind)

toCollectorType
toCollectorTemporality (source for how I did it)
toCollectorMetricDescriptor
toSingularPoint
toHistogramPoint

One particular question I had was regarding LastValueAggregator. This was the one of the two aggregators (along with ExactAggregator) which uses instantaneous temporality. I saw this was recently removed. Does that mean that the Temporality is now never instantaneous?
Next Steps
The next diffs will be as follows:

E2E Metric exporter for Browser with tests
E2E Metric exporter for Node with tests

Testing
I added detailed unit tests. Additionally, I E2E tested my code. However, histogram aggregators aren't currently supported, so I could only create a unit test for it.

@obecny Some of the functions in transform.ts are general purpose (e.g. toCollectorResource, toCollectorAttributes, which are used for both span + metrics). Others are span functions only. Finally, the ones I added are metric functions. I'm worried that splitting the file up into 3 files will be confusing. What do you think?

@obecny Some of the functions in transform.ts are general purpose (e.g. toCollectorResource, toCollectorAttributes, which are used for both span + metrics). Others are span functions only. Finally, the ones I added are metric functions. I'm worried that splitting the file up into 3 files will be confusing. What do you think?

I would split all of those that has anything to do with metrics, if they need to use the for example toCollectorAttributes or toCollectorResource than you can import  it. The reason is that this file may soon become bigger, but also this file is used by web and by node. The web might not need metrics in most cases so I would like to avoid importing functions that are not needed on web. Also testing file will become bigger, so I wouldn't worry of having more smaller files then one big. And finally we already have exporter for metrics and for spans, so if something is not needed for span exporter but only in metric exporter I would keep it separately and vice versa. WDYT ?

@obecny Ahh ok makes sense, thanks! Should the files be transform.ts and transformMetrics.ts?

@obecny Ahh ok makes sense, thanks! Should the files be transform.ts and transformMetrics.ts?

sounds good :)

One question, do you have maybe a working example similar to this one -> https://github.com/open-telemetry/opentelemetry-js/tree/master/examples/collector-exporter-node with a docker etc. so it will be possible to see metrics ?

@obecny Yeah, I have a working example that I've been testing my work on. I didn't use docker though, so I have to set that up. I just have a config.yaml file for my collector. Once this is fully submitted, I can make a PR uploading my example and documentation on how to run it.


Update conda-forge to more closley mirror gunicorn's

Apologies for the noise. This is an accidental change, I had intended to push this up to my fork. I'm working on trying to enable Python 3.11 builds (unsuccesfully).

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin please rerender

Hi! This is the friendly automated conda-forge-linting service.
I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.
Here's what I've got...
For recipe:

Selectors are suggested to take a <two spaces>#<one space>[<expression>] form. See lines [27, 28]
requirements: host: openssl=1.1.1 must contain a space between the name and the pin, i.e. openssl =1.1.1

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Please make sure to re-render after adding cross-compilation pieces


Get all labels from mx.io.ImageRecordIter

I would like to get all the labels from this object
test = mx.io.ImageRecordIter(
    path_imgrec = "chestxray/test.rec",
    data_shape = (3, HEIGHT, WIDTH),
    path_imglist = "chestxray/test.lst",
    label_width = 15,
    batch_size = BATCHSIZE,
    shuffle = False,
    rand_mirror = 0, #flip horizontally
    preprocess_threads = 24
)

I'm doing:
for test_batch in test:
    print(type(test_batch))
    d, l = test_batch.data, test_batch.label
    la = l[0].asnumpy()
    print(la.shape)

However, I don't get the final batch correctly.
Is there a function that loads all the labels directly to memory?

We manage and answer user questions/how to's on https://discuss.mxnet.io/, please post this question there. I will close this issue.


Support parallel functions in MSVC

This PR enables building netCDF with parallel I/O support under MSVC. This only requires minor changes as HDF5-1.10.5 supports Microsoft MPI by default with HDF5_ENABLE_PARALLEL. All tests are passing on my end except tst_parallel5 - I'll look into it.
The change to FIND_PACKAGE(HDF5) is minor to correct a typo, NO_MODULES should be NO_MODULE|CONFIG (synonyms). While this fixes HDF5 linkage on my system (I'm using the CMake Package from HDF Group), I think more extensive changes may be needed to make this more generic. You might consider integrating the changes in vcpkg, see for example microsoft/vcpkg#6771.
This should also fix the incomplete library exports discussed in #554 and netcdf-fortran#48, at least when parallel I/O is enabled.
Supporting Info
Configuration Summary
Test Results

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

All tests are passing, I just had to run them through mpiexec. Updated test results below. I also confirmed that netcdf-fortran now links correctly when parallel I/O is enabled.
Supporting Info
Test Results

This is great!
But I didn't know anyone used Windows for MPI programming. Is this for programmer convenience (That is, to allow Windows developers to debug MPI programs on their native platform)?
Or is there a use-case where Windows systems are being used to run MPI programs?

I am using Windows to run MPI enabled programs for atmospheric dispersion modeling, albeit usually on a single workstation. With HDF5 it's easier to use MPI-I/O and collective writes as opposed to trying to make it work with a thread-safe build. I don't think Windows HPC clusters are common but they are used in some engineering applications.
It's also for convenience since I prefer Windows/MSVC for development. I'm working on a new high-level, header-only C++ interface for netCDF, in the spirit of BlueBrain/HighFive: see jbuonagurio/ncpp. It's very much a work-in-progress but I'd like to support parallel functions.

Thanks, this is fantastic! I'll echo @edhartnett, I wasn't aware of any Windows-based MPI usage for netCDF, but having support for it is great! I'll get this reviewed (As well as the other outstanding PR's). Thanks!

Awesome! Thanks for your contribution. I especially welcome help with parallel I/O. It's a complex area of functionality, and we would very much welcome any extension or addition to the netCDF parallel testing programs.
Keep on netCDFing!

Thanks again!


chore(main): release core-js 1.5.11-experimental

:robot: I have created a release beep boop
1.5.11-experimental (2024-11-12)
🧹 Chore

release main (49fe368)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/open-socket/js-sdk/releases/tag/core-js-v1.5.11-experimental :sunflower:


Update Get-SPAuthenticationRealm.md

Format example

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

sharepoint/sharepoint-ps/sharepoint-server/Get-SPAuthenticationRealm.md
:white_check_mark:Succeeded
View (sharepoint-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


We need an icon.

ASAP

I decided to use this icon temporary.
If you have any idea of icon, please tell me.
This icon is applied to only windows currently.

It works all supported platforms.


Add settings option for peer editing.

Based on https://github.com/Sikorkaio/android-app/issues/8#issuecomment-359731371
Instead of manually asking users to mess with a file in the system is there any way to make this edit from inside the app? If yes it can be put in "Advanced Settings".
This will probably not be needed in a mainnet deployment but for the testnet, messing with the peers is unfortunately required.

[x] Import from peer list or json URL
[x] Import from local peer list or json
[x] Validate entries
[x] Option to replace previous entries or append
[x] Ability to remove specific entries
[x] View peer list
[x] Restart light client on peer import

Should be complete, maybe it will need some fine tuning with error messages.


Additional bag methods

Why plurals value is dependent on region & language only ?
Uncommon but it is valid to have any configs, like mnc/mcc, density, orientation ...

Why plurals value is dependent on region & language only ? Uncommon but it is valid to have any configs, like mnc/mcc, density, orientation ...

I believe I added that line because the result was not really what you would expect without it. I will have to do some more testing and take a look at the code you sent me.


Providing inputs to nix build

Thanks for working on cargo2nix :heart:
I set up a minimal project to demonstrate my issue here.
I have a build.rs file that calls zig through std::process::Command.
When calling nix build I'm currently hitting:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', build.rs:7:14

It can't find zig, I know how to provide zig for a nix devShell inside a flake but I don't know how to provide it for nix build.
I'm thinking I need to set buildInputs somewhere but I can't figure out the specifics.

See the overrides example in Readme.  You can just use buildInputs.  Here is an example of overriding on the workspace root crate:  https://github.com/cargo2nix/cargo2nix/issues/245
When you add zig to the build inputs, it will be on the path, and your build.rs should succeed.

Thanks :)
I changed
rustPkgs = pkgs.rustBuilder.makePackageSet {
          rustVersion = "1.60.0";
          packageFun = import ./Cargo.nix;
        };

to
rustPkgs = pkgs.rustBuilder.makePackageSet {
          rustVersion = "1.60.0";
          packageFun = import ./Cargo.nix;
          packageOverrides = pkgs: pkgs.rustBuilder.overrides.all ++ [
            (pkgs.rustBuilder.rustLib.makeOverride {
              name = "cargo2nix-deps";
              overrideAttrs = drv: {
                buildInputs = drv.buildInputs or [ ] ++ [
                  pkgs.zig
                ];
              };
            })
          ];
        };

and nix build works now.
I have a rust project with multiple crates(workspace members) that use zig, is there a way to make zig available for all crates in the workspace so I don't have to specify overrides for each specific crate? Perhaps it is not recommended to specify buildInputs in an overly broad manner :thinking:

works

:tada:

make zig available for all crates in the workspace

#251 will ease the pain.  For now just copy & paste.  It's cruft, but the kind that goes away easily later because it's all in one place.
Closing and picking up remaining concern in #251


Type validation for table ranges/pivot keys in operations

Recently, we have encountered a few situations in which users used integers of the wrong type when specifying table ranges and pivot keys in operations. This led to unexpected results, since in YT integers of different types are compared according to their order in the C++ value type enum (e.g. any int64 value is smaller than any uint64 value).
Let's think about validating the specified values against the table/chunk schema, or maybe even cast them automatically, if possible.

We have a corresponding internal issue – YT-19730.
The plan there is the same as you suggest:

At least fire operation's alert when pivot keys do not match table's schema. However, this requires that the table schema have all the relevant columns.
Better yet, use a different comparator for pivot keys.

We overlooked the fact that the same problem affects table ranges, great that you noticed that.


Use valid serviceAccountName in example

BRASS-530

The error, for reference:
Error: Deployment.apps "testrbac-pingauthorize" is invalid: spec.template.spec.serviceAccountName: Invalid value: "custom-serviceAccount": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is 'a-z0-9?(.a-z0-9?)*')


[msal-browser] Fix encoding issues

This PR fixes a couple issues

Cookies were not encoded before being saved, which resulted in cookie key:value errors when setting b64 values with trailing = characters.
State was being double encoded in some cases on the server-side, adding an additional decodeURIComponent should fix those issues.

Looks good, but if state is (arbitrarily?) being double-encoded on the backend... shouldn't we look to get that fixed rather than double decode as well? Seems odd. What if they end up triple encoding something someday?

Agree with this. Should we maybe check for a certain property to know we're done decoding?

Looks good, but if state is (arbitrarily?) being double-encoded on the backend... shouldn't we look to get that fixed rather than double decode as well? Seems odd. What if they end up triple encoding something someday?

Yes, we should get this fixed on the server, but for now, I think it is fine to assume things will at most be encoded twice.

Coverage decreased (-0.06%) to 79.218% when pulling 3b880f38e4bd0f0a5fc38f85819aa2d905ef39cd on fix-url-encoding-msal-browser into 532a9a4b4c7e1dcc4017fb6da441426ae1241282 on dev.


This is not async

If you pass a long enough array. It will block the event-loop.

hi, mind giving an example? Pretty sure the underlying concept behind this module isn't much different than others used for similar purposes, callback shenanigans aside

Easy to see when: map([...Array(5000).keys()], (v, cb) => cb(null, v*10), () => {}) will produce a Max call stack size exceeded.
If the iterate function is takes long, it would block the event-loop for long time.

thanks for the input. I made the tasks run in parallel and did a few tweaks so it should work now:thumbs_up:


Show when pods are loading or fail to load

We need to update the UI to provide some indication of when pods are loading or when they fail to load.

Depends on #59, otherwise ready for review.

@rudigiesler ready for review

:+1:


Disconnect HttpConnection on HttpStatusException and UnsupportedMimeTypeException

Too many open files error could happen here in some circumstances

We had some problems where we parse hundreds of URLs in a row and many of them returned a 404 which caused a too many open files error on some older Samsung Android devices.

Hi, thanks for submitting this PR. As far as I can tell, these .disconnects are redundant, because there already is a .disconnect in the IOException catch block (which catches the throws on 759 and 769) on line 799: https://github.com/jhy/jsoup/blob/7de614ffd01161d8718d7458bab9d9690ad2334f/src/main/java/org/jsoup/helper/HttpConnection.java#L796
For sanity I traced execution and verified that the existing disconnect is indeed already called when those exceptions are throw. fd46489cd718ddea7b28f89953265e7ce4ec8372
I have noticed on some Android implementations with okhttp, .disconnect is "advisory" and goes into a different thread to actually reap the connection keepalive. I wonder if that is the root of the problem you are seeing.
Let me know if I'm missing something here or if you have another way to validate this change.

Ups, I didn't realize that.
I'll see if I can find some way do validate it and if so I'll create another pull request.
Thanks anyway for looking into it!


Help on "Message: chrome not reachable"

Hi @umihico,
I am using your selenium-docker solution to run headless Chrome browser in AWS Lambda for my project. It is a great help already :)
I need a bit of support and your expert advice on this setup if you dont mind. I'll give a bit of context what I am doing so that its easy for you to give advice. -
I created this headless browser lambda and I am using it to render some images from a URL and then save these images as screenshots in S3. This whole process runs for a big number of URLs (like a stream of URLs to this lambda as input) via AWS Step Functions.
Although its working but there are always about 50-60 URLs (total URLs 3000) which runs into these following messages -

Message: chrome not reachable
Message: unknown error: unhandled inspector error: {"code":-32000,"message":"Unable to capture screenshot"}
Message: unknown error: cannot activate web view

I am trying to understand the problems as to why it happens only for few URLs and not for all. Your help will help me understand more.
Hope to hear back from you. Thank you.
This is my Dockerfile -
#based on https://github.com/umihico/docker-selenium-lambda
FROM public.ecr.aws/lambda/python:3.8 as build

# download chrome binaries, unzip them desired location
RUN yum install -y unzip && \
    curl -Lo "/tmp/chromedriver.zip" "https://chromedriver.storage.googleapis.com/98.0.4758.48/chromedriver_linux64.zip" && \
    curl -Lo "/tmp/chrome-linux.zip" "https://www.googleapis.com/download/storage/v1/b/chromium-browser-snapshots/o/Linux_x64%2F950363%2Fchrome-linux.zip?alt=media" && \
    unzip /tmp/chromedriver.zip -d /opt/ && \
    unzip /tmp/chrome-linux.zip -d /opt/

# as of feb 2023 python3.8 was stable
FROM public.ecr.aws/lambda/python:3.8

# install necessary packages for running chrome
RUN yum install atk cups-libs gtk3 libXcomposite alsa-lib \
    libXcursor libXdamage libXext libXi libXrandr libXScrnSaver \
    libXtst pango at-spi2-atk libXt xorg-x11-server-Xvfb \
    xorg-x11-xauth dbus-glib dbus-glib-devel -y

# install selenium
RUN pip install selenium

COPY --from=build /opt/chrome-linux /opt/chrome
COPY --from=build /opt/chromedriver /opt/

#COPY lambda_function.py requirements.txt ./
# copy requirements
COPY . requirements.txt ./

COPY mraid.js ./tmp/

# install requirements
RUN python3.8 -m pip install -r requirements.txt -t .

# Command can be overwritten by providing a different command in the template directly.
# entry point
CMD ["lambda_function.lambda_handler"]

This is more like help and not an issue. Please suggest a better way to ask this help if this is no the right place. THank you.

Why not re-enqueue these that are failing?,
Are always the same bunch that are failing?
@ahirnish

input is different each time - its not fixed. No of affected inputs are in the range 50-100 everytime. Message #2 can be more application specific but #1 and #3 are more tool specific. @x-N0

What I'd do is to flip the versions to see if that gets the issue resolved.

mehmetturkmen@192 docker-selenium-lambda-memo % docker run -p 8080:8080 --platform=linux/amd64 my-selenium-lambda
26 Jun 2023 07:26:04,904 [INFO] (rapid) exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)
26 Jun 2023 07:26:13,082 [INFO] (rapid) extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory
26 Jun 2023 07:26:13,083 [INFO] (rapid) Configuring and starting Operator Domain
26 Jun 2023 07:26:13,084 [INFO] (rapid) Starting runtime domain
26 Jun 2023 07:26:13,088 [WARNING] (rapid) Cannot list external agents error=open /opt/extensions: no such file or directory
START RequestId: 0c4ae207-b6d5-425e-8a02-ca28fb990da5 Version: $LATEST
26 Jun 2023 07:26:13,094 [INFO] (rapid) Starting runtime without AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN , Expected?: false
Chromedriver sürümü: None
Chromedriver yolu: /opt/chromedriver
[ERROR] WebDriverException: Message: unknown error: Chrome failed to start: crashed.
(chrome not reachable)
(The process started from chrome location /opt/chrome/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x0040006d74e3
#1 0x004000406c76
#2 0x00400042fd78
#3 0x00400042c029
#4 0x00400046accc
#5 0x00400046a47f
#6 0x004000461de3
#7 0x0040004372dd
#8 0x00400043834e
#9 0x0040006973e4
#10 0x00400069b3d7
#11 0x0040006a5b20
#12 0x00400069c023
#13 0x00400066a1aa
#14 0x0040006c06b8
#15 0x0040006c0847
#16 0x0040006d0243
#17 0x004002aaf44b start_thread
  File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriv  File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/chromium/webdr  File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/remote/webdriv  File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/remote/webdriv  File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/remote/webdriv  File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/remote/errorha    raise exception_class(message, screen, stacktrace)
END RequestId: 1d423694-c9a9-41c7-a660-4badff838b6a
REPORT RequestId: 1d423694-c9a9-41c7-a660-4badff838b6a	Init Duration: 2.28 ms	Duration: 16456.85 ms	Billed Duration: 16457 ms	Memory Size: 3008 MB	Max Memory Used: 3008 MB
I'm getting this error right now, do you have any suggestions for a solution?
@ahirnish @umihico

@ahirnish @Turkmen1Mehmet
The shown code above and my code is too different.
If my original code works fine locally, you can modify step by step to convert my file to your file. Then, you can tell us which added line start causing bugs, instead of just pasting entire file.

@ahirnish @umihico
First of all, I made a small change in the code. First I run the code piece by piece. WebDriver.init()
mehmetturkmen@192 docker-selenium-lambda-memo % docker run -p 8080:8080 --platform=linux/amd64 my-selenium-lambda
26 Jun 2023 10:47:27,361 [INFO] (rapid) exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)
26 Jun 2023 10:47:38,253 [INFO] (rapid) extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory
26 Jun 2023 10:47:38,255 [INFO] (rapid) Configuring and starting Operator Domain
26 Jun 2023 10:47:38,257 [INFO] (rapid) Starting runtime domain
START RequestId: c3d5437a-a8af-484c-9473-16eda043a525 Version: $LATEST
26 Jun 2023 10:47:38,261 [WARNING] (rapid) Cannot list external agents error=open /opt/extensions: no such file or directory
26 Jun 2023 10:47:38,263 [INFO] (rapid) Starting runtime without AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN , Expected?: false
[ERROR] TypeError: WebDriver.init() got multiple values for argument 'option    chrome = webdriver.Chrome("/opt/chromedriver",options=options)
END RequestId: 495b85d4-3693-4dd9-826c-e4207ceb8b64
REPORT RequestId: 495b85d4-3693-4dd9-826c-e4207ceb8b64	Init Duration: 1.77 ms	Duration: 1566.42 ms	Billed Duration: 1567 ms	Memory Size: 3008 MB	Max Memory Used: 3008 MB
^C26 Jun 2023 10:56:08,387 [INFO] (rapid) Received signal signal=interrupt
26 Jun 2023 10:56:08,387 [INFO] (rapid) Shutting down...
26 Jun 2023 10:56:08,390 [WARNING] (rapid) Reset initiated: SandboxTerminated
26 Jun 2023 10:56:08,391 [INFO] (rapid) Sending SIGKILL to runtime-1(17).
26 Jun 2023 10:56:08,413 [INFO] (rapid) Stopping runtime domain
26 Jun 2023 10:56:08,415 [INFO] (rapid) Waiting for runtime domain processes termination
26 Jun 2023 10:56:08,416 [INFO] (rapid) Stopping operator domain
26 Jun 2023 10:56:08,416 [INFO] (rapid) Starting runtime domain
I also updated the code
This is the error I got this time
mehmetturkmen@192 docker-selenium-lambda-memo % docker run -p 8080:8080 --platform=linux/amd64 my-selenium-lambda
26 Jun 2023 11:07:18,953 [INFO] (rapid) exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)
26 Jun 2023 11:07:24,248 [INFO] (rapid) extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory
26 Jun 2023 11:07:24,251 [INFO] (rapid) Configuring and starting Operator Domain
26 Jun 2023 11:07:24,251 [INFO] (rapid) Starting runtime domain
26 Jun 2023 11:07:24,258 [WARNING] (rapid) Cannot list external agents error=open /opt/extensions: no such file or directory
START RequestId: 6637e50c-74dc-4eb4-9484-986025c641a9 Version: $LATEST
26 Jun 2023 11:07:24,266 [INFO] (rapid) Starting runtime without AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN , Expected?: false
[ERROR] WebDriverException: Message: unknown error: Chrome failed to start: crashed.
(chrome not reachable)
(The process started from chrome location /opt/chrome/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x0040006d74e3
#1 0x004000406c76
#2 0x00400042fd78
#3 0x00400042c029
#4 0x00400046accc
#5 0x00400046a47f
#6 0x004000461de3
#7 0x0040004372dd
#8 0x00400043834e
#9 0x0040006973e4
#10 0x00400069b3d7
#11 0x0040006a5b20
#12 0x00400069c023
#13 0x00400066a1aa
#14 0x0040006c06b8
#15 0x0040006c0847
#16 0x0040006d0243
#17 0x004002aaf44b start_thread
  File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriv  File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/chromium/webdr  File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/remote/webdriv  File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/remote/webdriv  File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/remote/webdriv  File "/var/lang/lib/python3.10/site-packages/selenium/webdriver/remote/errorha    raise exception_class(message, screen, stacktrace)
END RequestId: 7ab9eb86-0a3f-42a9-8bfc-c4cd9fc5acc7
REPORT RequestId: 7ab9eb86-0a3f-42a9-8bfc-c4cd9fc5acc7	Init Duration: 4.71 ms	Duration: 17264.78 ms	Billed Duration: 17265 ms	Memory Size: 3008 MB	Max Memory Used: 3008 MB
Current main.py code
from selenium import webdriver
from tempfile import mkdtemp
from selenium.webdriver.common.by import By
import os
import sys
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.color import Color
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.chrome.options import Options
import re
import requests
import time
from bs4 import BeautifulSoup
import sqlite3
import urllib.request
import zipfile

def handler(event=None, context=None):
    chromedriver_path = "/opt/chromedriver"
    agent = {"User-Agent": 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36'}

    op = webdriver.ChromeOptions()
    op.binary_location = '/opt/chrome/chrome'
    op.add_argument('--headless')
    op.add_argument('--no-sandbox')
    op.add_argument("--disable-gpu")
    op.add_argument("--window-size=1280x1696")
    op.add_argument("--single-process")
    op.add_argument("--disable-dev-shm-usage")
    op.add_argument("--disable-dev-tools")
    op.add_argument("--no-zygote")
    op.add_argument(f"--user-data-dir={mkdtemp()}")
    op.add_argument(f"--data-path={mkdtemp()}")
    op.add_argument(f"--disk-cache-dir={mkdtemp()}")
    op.add_argument("--remote-debugging-port=9222")
    op.add_argument(f"--executable-path={chromedriver_path}")

    browser = webdriver.Chrome(options=op)
    browser.get("https://www.zara.com/tr/")
    return browser.find_element(by=By.XPATH, value="//html").text

@Turkmen1Mehmet
In comparison my original code, the diff is more than 10 lines. Please try between them, one line by line. You'll notice inserting specific line cause the error, then please share the detail in another issue.