[ENH] Extending pyjanitor to support other data-handling frameworks

Potential candidates include (but would be in no way limited to):

xarray
vaex
Dask

The idea here is that data cleaning and higher-level functions to manipulate data structures are pretty universal endeavors in any kind of analysis. A clean API could be very beneficial for many packages.
XArray has a pandas-flavor-like accessor interface already (and is used by packages such as hvPlot):
http://xarray.pydata.org/en/stable/generated/xarray.register_dataset_accessor.html
This is probably needing of a little more discussion especially as related to DRY and modularizability of pyjanitor.

This is definitely a great idea, but we'd really have to see if all the functions would be applicable to parallel computing situations. That's the reason not every Pandas function is implemented in Dask dataframes. For example, median is difficult to compute when split across machines.
Nothing immediately comes to mind as being problematic in pyjanitor, but the functions still should be considered how to be performed in a distributed manner.

Very much agreed for the parallel stuff.

This is how hvplot modifies Dask dataframes:
https://github.com/pyviz/hvplot/blob/master/hvplot/dask.py
Looks like it’s as simple as a monkey patch to add the .hvplot attribute, as far as I can tell.


apis_core.history leads to broken loaddata

When trying to load fixtures, there is an error:
django.db.utils.IntegrityError: Problem installing fixture 'sample_project.json': Could not load sample_project.Person(pk=1): NOT NULL constraint failed: sample_project_versionperson.id

removing this method:
https://github.com/acdh-oeaw/apis-core-rdf/blob/9fd8080598bd89b00220b7406c678f04e1cca5b4/apis_core/history/models.py#L19
makes the error go away

Error is coming from adding the professions m2m field to track via simple history. I realized that Profession has been removed from being tracked, but this change hasnt been migrated. Can you try to do makemigrations and migrate and then export a new fixture? I dont think it will fix the problem, but is nonetheless worth a shot before changing the m2m behavior of the history module.


IAMv2 API Docs: apply-rules

As an administrator of a Chef Automate install using IAMv2 and configuring my projects and rules via the API, I'd like to have a reference to the apply-rules endpoint so that I can ensure that any changes that I make take effect in my system.
The API documentation for IAMv2 does not currently include any reference to the apis/iam/v2beta/apply-rules

@susanev add this item to the iam v2 ga card

Example: https://gist.github.com/sdelano/ef3dd71e7445d28617924d8c8e2914da#applying-rules

@mjingle will be completing this task for us starting the week of 11/18


Pre-rendering on pages with 301 stopped working on<EMAIL_ADDRESS>

Hi all,
we are using pptr for prerendering our SPA. One of the URLs which needs to be prerendered is example.de which should end with 301 with a header location pointing to example.com.
We have a working solution below with<EMAIL_ADDRESS>but if we update to >=1.7.0 it stopped working because it is returning the fake status code.
Steps to reproduce
Tell us about your environment:

Puppeteer version: 1.6.2 is ok, any newer is affected
Platform / OS version: MacOS or Docker (node:11.8)
URLs (if applicable): -
Node.js version: 11.8

What steps will reproduce the problem?
The code working on 1.6.2
await page.setRequestInterception(true);
const FAKE_HTTP_CODE = 399;

let isRedirect = false;
page.on('request', request => {
    if (isRedirect) {
    	// detect redirect and make sure that chrome will not process next request properly by sending fake status code 399
        return request.respond({status: FAKE_HTTP_CODE});
    }

    return request.continue({url: request.url()});
});

page.on('response', response => {
    if (response.request().resourceType() === 'document' && response.status() === 301) {
        isRedirect = true;
    }
});

Response before update:
< HTTP/1.1 301 Moved Permanently
< location: https://www.erento.com/

Response after update:
< HTTP/1.1 399 unknown

The unit test which is not working anymore:
test('301 response code', async () => {
    const response = await Browser.get('https://www.erento.de', {'Accept-Language': 'de-DE'});
    expect(response.statusCode).toBe(301);
    expect(response.headers.location).toBe('https://www.erento.com/');
});

It is important for crawlers to return 301 if necessary though. Is there any way in new pptr versions to not follow redirects but to return the redirect response?

I am facing similar kind of issue with<EMAIL_ADDRESS>where its not able to detect the URL moved from HTTP to HTTPS.
When I load on headful chrome the URL loads fine and I see 301 Moved Permanently in the response however in the headless chrome it just throws Navigation Timeout Exceeded: 30000ms exceeded.

this is a dupe of #4030, check out this comment
LT;DR: the request.continue({url}) doesn't redirect to a given URL - instead, it loads data from the given url as if it was served from the original one.
In order to do a real redirect, use request.respond.

@aslushnikov that is actually what I was already trying but it doesn't help. Please look into a following example:
const puppeteer = require('puppeteer');

(async() => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();

    await page.setRequestInterception(true);

    let isRedirectStatus = false;
    page.on('request', (request) => {
        if (isRedirectStatus !== false) {
            return request.respond({
                status: isRedirectStatus,
                headers: {
                    location: 'https://www.erento.com'
                },
            });
        }

        return request.continue({url: request.url()});
    });

    page.on('response', (response) => {
        if (response.request().resourceType() === 'document') {
            const status = response.status();
            if ([301, 302].includes(status)) {
                isRedirectStatus = status;
            }
        }
    });

    await page.goto('http://www.erento.de');
    await browser.close();
})();

it ends with ERR_TOO_MANY_REDIRECTS and the expected behaviour would be to return the object in .respond() method (in example the url is hardcoded just for simplicity)

@rokerkony What your code does is redirecting all requests to 'https://www.erento.com' once there's at least one redirect response from server. Given that http://www.erento.de immediately redirects to HTTPS, all your future requests are redirected to the same URL and you naturally get ERR_TOO_MANY_REDIRECTS.
Are you trying to redirect just the navigation request? In this case, use the request.isNavigationRequest() and check for its target URL before issuing a redirect.

@aslushnikov What I'm trying to accomplish is to get the same behavior as in version 1.6.2 and lower, I hope it will be still possible somehow.
We are using pptr for our prerendering and we would love to return a response with 301 and location header when google goes to e.g.: erento.de. When we would return a content from followed redirects it would be treated as a duplicate content instead of redirect.
I hope it is well demostrated by a first post in this issue where we used fake http status 399 and pptr returned 301 with location header (see test for it).

@rokerkony so what does Browser.get do under the hood? I'm still not sure what's the end goal here - can you please elaborate more? Maybe I'd be able to suggest something.


flutter stop not hooked up for iOS Simulator

The message says the application was stopped but it wasn't :(

Could this be causing https://github.com/flutter/flutter/issues/3360 ?

This was fixed a while ago. Closing.


Automated cherry pick of #7812

Cherry pick of #7812 on release-5.34.
/cc  @michelengelen

@michelengelen Are you able to resolve the build failure, or do we need help from Devops?

@amyblais since those errors are occurring due to missing/failing access key assignments I think we need help from the devops team here!

@metanerd @cpanato Can you help take a look at the build failure?

@michelengelen Can you please create this cherry pick either manually or upstream? Currently automatic cherry-picks fail. The reason for this is described here with further info: https://mattermost.atlassian.net/browse/MM-34620

Hi @metanerd ... I did pick the commit manually in this commit on branch release-5.34.
Hence I am closing this PR!
Thanks for the help and clarification! 🙇🏻♂ 🎉


use the editdistance when there is no gcc

i would like to know is there is a way to use this python package if I do not have the gcc on the server?
I have to make sure that the code should be compiled somewhere else, and on the server, I can directly run it or use it.

Here is the way I figured out:

Download the package editdistance-master from github and unpackage it.

cd editdistance-master and Run python setup.py build on another machine. On this machine, you should have the c++ compiler and this machine have the same specs as you server (such as same 64 bits).

Then you will have a new folder called build. Under this folder, you will have lib.<plat>/ and/or temp.<plat>/. Under the folder lib.<plat>/, you will see the folder editdistance. This is where all the compiled files are placed. And there is a _init_py.

Copy this to the place where your calling python program is located. And import the package as import editdistance.

Then you can use it.


Adds VS Code extension information to docs. Closes: #5796

Closes: #5796

Merged manually, thanks for the addition!


Update strict tests for Todo App - Step 5

Describe the Issue
The code doesn't pass if you open the curly braces.
Affected Page
https://forum.freecodecamp.org/t/learn-localstorage-by-building-a-todo-app-step-5/666974
Your code
const taskData = [];
let currentTask = {

};

Expected behavior
I expected to pass, I was assuming code would be put into this object at a later time.
this is the expected solution:
const taskData = [];
let currentTask = {};

Screenshots
No response
System

Device: [e.g. iPhone 6, Laptop]
OS: [e.g. iOS 14, Windows 10, Ubuntu 20.04]
Browser: [e.g. Chrome, Safari]
Version: [e.g. 22]

Additional context
No response

Note to interested contributors
The tests can be rewritten to better use the existing assert methods from chai.
https://www.chaijs.com/api/assert/
Something like this should work. Please test everything on your end too, but this should cut down on the amount of regex tests used and better utilize the other assert methods
--hints--
You should have a const variable called taskData.
assert.match(code, /const\s+taskData\s*=/);

You should assign an array to your taskData constant.
assert.isArray(taskData);

Your taskData array should be empty.
assert.isEmpty(taskData);

You should use let to create a currentTask variable.
assert.match(code, /let\s+currentTask/);

You should assign an object to your currentTask variable.
assert.isObject(currentTask);

Your currentTask object should be empty.
assert.isEmpty(currentTask);


Row counts incorrect in ICU module

The row counts are incorrect in the ICU module for MIMIC-IV.
e.g. inputevents lists 7,643,978, but the details pane of the BigQuery table shows 9,460,658. The numbers listed are all from MIMIC-III and copy pasted into MIMIC-IV.

fixed


Add Grunt task runner

Hey @mravinale could you take a look? This only adds the Gruntfile definition with two simple tasks: watch & lint. I had to fix some issues due to jshint suggestions but don't worry: the changes are in a different commit.

:+1:


Using latest strimzi

Signed-off-by: Matthias Wessendorf<EMAIL_ADDRESS>Fixes #
Proposed Changes

updating used kafka and its operator for e2e tests

/assign
/lgtm


bug: Stage crash

Describe the bug
When ever I enter a stage and attempt to open the chat it closes the stage window and refuses to open it again even to leave the stage no matter if you are a speaker or a listener.
To Reproduce
Steps to reproduce the behavior:

Go to a Stage
Click on the chat icon
Attempt to scroll up or just stay there
Closes stage screen
Can't open it again

Expected behavior
One this is replicated will have to leave stage by notification bar or force closing the app
Screenshots
If applicable, add screenshots to help explain your problem. (There is none)
Device Information (please complete the following information):

Device: Google Pixel 7a
OS: Android 14
Discord Version 243.16
Bunny Version 22925c0-main

Checklists

[ ✅ I understand that this is a bug report and not a place to request new features or plugins
[✅] I understand that this is no place to report bugs produced by a plugin
[✅] I can reproduce this bug in Bunny's safe mode AND unable to reproduce it in vanilla/unmodded Discord

fyi adding emojis into a markdown checklist isn't how they work, either add an x so they render correctly or click them in the rendered comment to check them

fyi adding emojis into a markdown checklist isn't how they work, either add an x so they render correctly or click them in the rendered comment to check them

Sorry this was my first bug report in a while and clicking on it did nothing for me but thank you for telling me for next time


[SPARK-15167][SQL] Expose catalog implementation in SparkSession

What changes were proposed in this pull request?
Right now there's no way to check whether a given SparkSession has Hive support. You can do spark.conf.get("spark.sql.catalogImplementation") but that's supposed to be hidden from the user.
After this patch, the user can run the following to find out whether this SparkSession has Hive support.
scala> spark.catalog.implementation
res0: String = in-memory

How was this patch tested?
Jenkins.

@rxin do you think this is a good idea?

Test build #57935 has started for PR 12942 at commit 23d82c5.

Test build #57935 has finished for PR 12942 at commit 23d82c5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57935/
Test PASSed.

hm probably not this way because this wouldn't make sense once we make the catalog mountable. we might want to have some other way to tell though.

alright, for now the user will have to check the secret conf value.


Haskell provider doesn't work on M1 macs

Running Nixpackgs on the examples/haskell-stack directory fails with error
Unable to find installation URLs for OS key: linux-aarch64-tinfo6

It seems that the Haskell toolchain cannot find binaries for the aarch64 architecture on Linux

That's really weird, as Stack should be building the toolchain from source (that's why it takes so long).

Something somewhere is missing or not support on linux aarch machines. Not exactly sure why. I can take a deeper look into it later/tomorrow but wanted to create the issue in case anyone else had ideas.

Did you find out anything?

I can take a look into this if no one is working on this?

https://github.com/docker-library/docs/pull/2101/ seems like arm64 is not supported on haskell docker images yet.
https://github.com/commercialhaskell/stack/issues/2103 This also seems to be an upstream stack issue to provide support for ARM64.


Deploy Smart Contract using Foundry Fails with Deserialization Error: invalid length 1...

Description
Description
Back in January 2023, I successfully used forge create to deploy a smart contract on Hedera.
See closed issue #834
Fast forward to October 2023 and I am running to the below error.
What I did:
When i run the following command
forge create --rpc-url <your_rpc_url> --private-key <your_private_key> contracts/TodoList.sol:TodoList
I received the following error:
Error:
Deserialization Error: invalid length 1, expected a (both 0x-prefixed or not) hex string or byte array containing 32 bytes at line 1 column 1266. Response: {"timestamp":"0x65299eb0","difficulty":"0x0","extraData":"0x","gasLimit":"0xe4e1c0","baseFeePerGas":"0x1aebc97c400","gasUsed":"0x0","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","miner":"0x0000000000000000000000000000000000000000","mixHash":"0x0000000000000000000000000000000000000000000000000000000000000000","nonce":"0x0000000000000000","receiptsRoot":"0x0000000000000000000000000000000000000000000000000000000000000000","sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","size":"0x59a","stateRoot":"0x0000000000000000000000000000000000000000000000000000000000000000","totalDifficulty":"0x0","transactions":[],"transactionsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","uncles":[],"withdrawals":[],"withdrawalsRoot":"0x0","number":"0x320a0f","hash":"0xb734b7645ec3a5f3e5d51a15b4da8235f56486994c13f6b213be4eaa4b779cc4","parentHash":"0x563b39bf894273da3cd678b9c2952abb3561bb42ecf7750df127f47e3702ba84"}

What I expected to happen:
My Contract to be deployed successfully and a similar message to the one below:
[⠰] Compiling...
No files changed, compilation skipped
Deployer: 0x4DdC478A1F35Bbf266D2F5767AE399f4d6E83857
Deployed to: 0x47c81E6e3f2880a42149574f08DCaBffCa10e164
Transaction hash: 0xfec3c5a6d726a4f94a57011c7c5f6fe3f54dad70700652570cda626f58a81dca

Steps to reproduce
Steps to reproduce

Unzip the project below and open it in vscode
hedera_foundry_fork_testVendingMachine.zip
edit the .env file
Open a terminal in the project's dir
run forge create --rpc-url https://testnet.hashio.io/api --private-key <HEX_Encoded_Private_Key> contracts/TodoList.sol:TodoList

Additional context
No response
Hedera network
testnet
Version
latest
Operating system
macOS

This should be already fixed here: #1797

In addition, while using foundry for smart contract development, because it uses ethers-rs, you will also likely face this problem: #1800 at least until we reintroduce the v parameter in the receipts.

Hello @Neurone and @a-ridley ,
I started to triage this issue last friday and was able to figure out the format issue with the withdrawalsRoot field that should be fixed after #1797  but does not seem to be fixed,


I have a question about the prompt when generating a chest image.

Thanks for the great research!!
“Brain imaging generation with latent diffusion models.” According to the paper, when the diffusion model generates brain images,
it receives conditioning variables(age, sex, ventricular volume, brain volume) and generates them.
On the other hand, when generating a chest image, how do you convert prompts into conditioning variables?
Looking at inference.json, it seems to use pre-trained CLIPTokenizer and CLIPTextModel.
When generating a chest image, if I enter the prompt as, for example, "Big right-sided pleural effusion",
will the pre-trained CLIPTokenizer and CLIPTextModel convert the prompt into conditioning variables?
Is it correct to use conditioning variables?
If it is correct to use conditioning variables, can I know what conditioning variables are used when generating chest images?
Thank you.

Hi there,
the tokenizer will not convert the prompt into human-interpretable conditioning variables (e.g. age, volume) - just into an embedding space that is meaningful to the model itself, but not to you! So the only way to control the generation in a human-interpretable way is using text inputs.


TCP traffic routing based on client ip address

Describe the feature request
HTTP traffic can be routed based on client ip address using headers field in Virtual Service, otherwise there is no way to route TCP traffic based on client ip address even if proxy protocol is enabled on external LB. I researched how to route TCP traffic with Envoy Filter for several days but I couldn't resolve it. This feature would be useful for many people so that I request the feature that enable TCP traffic routing based on client ip address.
Describe alternatives you've considered
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[X] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Additional context

I think this can be done with destinationSubnets? https://istio.io/latest/docs/reference/config/networking/virtual-service/#L4MatchAttributes

In the past I have done this with ServiceEntry addresses.
See https://github.com/istio/istio/blob/master/samples/external/github.yaml for an example.

How can I apply a condition statement for client ip address? In case of HTTP, it works with the below manifest.
`
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: app-vservice
namespace: test
spec:
hosts:

"app-service"
http:
match:

headers:
x-forwarded-for:
exact: <IP_ADDRESS>
route:
destination:
host: app-service
subset: v2

route:

destination:
host: app-service
subset: v1
`

I think this can be done with destinationSubnets? https://istio.io/latest/docs/reference/config/networking/virtual-service/#L4MatchAttributes

not des but src


how to stop konga service use command line

HI,
just want to know how to stop konga use a linux command line ,thanks

You can kill the process or stop the container if you're using docker


.Net8 OpenApi Enums

Hi,
is there a problem with enums as endpoint parameters in .Net8; They are now exportet as strings in the specification-file.
if i rollback to .net6 they are exportet correctly as enums.
Thanks.

https://github.com/RicoSuter/NSwag/issues/4727


Change libnice to v 0.1.7

libnice 0.1.13 was causing connection problems in some cases - We'll downgrade to 0.1.7 as it has proven to work without problems until we further investigate the issue

Just a note that I also struggled finding a version of libnice that behaved well. The older versions were missing some important bugfixes, and the 0.1.13 release had the connectivity issues you observed.
I eventually settled on using their master branch as of June 23rd, 2016 (commit 58fdd9b5f7e04e5f77bbc50ba2ac24f592a10b84) and this has worked without issue for me.
I've also inquired about why they haven't made a new release in forever, and weren't interested in cutting a new point release at the time.


[iOS,v10] Remove source if view is removed

iOS, v10:

Remove source if view is removed and we created the source
add getZoom to mapView

Thank you @mfazekas ! Was just running in that issue.


Xiaomi series evaluated as rooted

I ran the 'RootBeer Sample' app downloaded by google play on some xiaomi devices. and I found a problem that the devices are evaluated as rooted.
Xiaomi Mi max2 - busybox binary
Hongmi(Redmi) Note3 Pro - busybox binary, dangerous props
Xiaomi Mi Note - busybox binary, su binary, root native

Xiaomi Mi A1 - not rooted
Huawei honor 6x - not rooted

I'm not sure whether the devices are actually rooted or not. But as far as I know the devices are not rooted because I got the devices from where some kind of public institutions.
Is that issues for xiaomi devices? Or is that rooted actually?
How can I deal with it?

@HanHyeoksu What ROMs do those Xiaomi devices run?

@TheComputerGuy96 I don’t know exactly it’s ROM name or not but I remember that ROM name was MIUI. Is there some relationship between ROM and rooting?

@HanHyeoksu The busybox binary can sometimes be a false positive when it comes to detecting root, as some manufacturers leave it in their production builds. This is why we have an option to ignore this when checking for root when using this library.
With your Xiaomi Mi Note, the su binary was found which is usually pretty clear indication that rooting is present so might be worth checking that out if you weren't expecting that.

@stealthcopter Thanks. You're right. I replaced isRooted with isRootedWithoutBusyBoxCheck in my app.
So, you mean that Xiaomi Mi Note is likely to be rooted because su binary was found. right?
Then what about Redmi Note3 pro? 'Dangerous props' was found in it. It means that rootbeer found the 'ro.debuggable=1' and 'ro.secure=0' in the android phone and I've got know the fact that the props is found then it's likely to be rooted.

@HanHyeoksu Yes, exactly!

@stealthcopter Thanks a lot!


Make tests "pass" on NetBSD

A bit belatedly PRing this. On NetBSD / Pkgsrc we are using this patch for the pkgsrc-wip spotifyd package.
I'd held off PRing it because I was unsure about the gid bit, but I might as well PR it as it at least opens it up for discussion.

For the nobody uid it's the same as OpenBSD. For the gid, however, it's
a bit different. You can't guarantee the gid is going to be the same
"max value" as per the uid for the nobody user. On my two amd64 NetBSD 8
systems the nobody user is listed as:
nobody:*:32767:39:Unprivileged user:/nonexistent:/sbin/nologin
But I don't think that 39 is going to be guaranteed everywhere.
Therefore simplest solution seems to be to skip this test on NetBSD.

Thanks!


perf(media): optimize result type check for json deserialize

This patch optimizes type checking for the JSON media handler. It also
removes 'utf-8' from several locations across the code base because it is
unnecessary and also incurrs a slight performance penalty (~40 ns under py36).

As pointed out in a comment on uri.pyx, the proposed changes in Cython code are actually a slight de-optimization.

No worries, I'll revert my itchy trigger finger. 😛


option to obtain authcode via https://github.com/inAudible-NG/tables

I added a check if git is installed, if no authcode is provided by the user the script will present the user with
[AAXtoMP3]$ ./AAXtoMP3 SOME_FILE.aax
ERROR Missing authcode
This script can try to obtain the authcode by itself.
Do you want the script to try this? [Y/N]: y
Trying to obtaining checksum: SOME_CHECKSUM
Trying to obtaining authcode: SOME_AUTHCODE

if the user answers no the script will just like before spit out the usage info and exit

Though easy to use and the recommended approach to getting the authcode off an .aax file, I've explicitly avoided linking the inAudible-NG/tables project to this one. I believe very firmly that the most important aspect of this script is that it is only for Audible books that the user explicitly owns. That project makes it possible to crack the encoding of an Audible book and thus piracy becomes much more tenable.
Thank you for your contribution, it is for the above reason that I am denying it.

Yeah i see where you are coming from, seems about right. Anyways, great script, thanks for that


Create an inspirational post about very practical Vendir usecase

Working on it, I'll released it very (very) soon 😸

https://twitter.com/rastadidi/status/1411864443567312907
https://dev.to/adriens/deploy-neo4j-s-apoc-plugin-with-code-thanks-to-carvel-vendir-33l4


Add linter package - base

This PR adds the linter package golangci-lint with some initial linters enabled and adapted for. Also included is a GitHub action to run them automatically.
Motivation for this is to allow adherence to established standards for a critical library.
This PR is a smaller, first step than originally #1 was planned.

A year has passed for this reduced pull request.
What is your plan with this repository?

Years of no update, no response, issues piling up - I suspect this repository is abandoned.

Hello and thanks for reporting. I can confirm that this repository is not maintained anymore, it was part of a MsC thesis project but it was not correctly marked as "unmaintained". Kudelski Security does not use this library in any commercial offering or services, it was just meant for testing and fun. I have now updated the README file to clarify this, and I will now archive this repository to avoid further issues. The reason why we never responded before is that nobody actively monitored these issues, apologies for that!


Docker image imported by ctr gets unexpected "docker.io/" repo prefix

Description
When I load a Docker image tar via ctr import, the image ref unexpectedly has a "docker.io/" prefix on the repo. I would expect that only to be true for images pulled from the Docker container registry.
If it's relevant, I'm using image tars produced by Bazel using rules_docker.
Steps to reproduce the issue:

Run ctr import bar.tar for an image with repo "bazel/src/main/java/foo" and tag "bar"
Run ctr image ls

Describe the results you received:
Displayed image ref is "docker.io/bazel/src/main/java/foo:bar"
Describe the results you expected:
Displayed image ref is "bazel/src/main/java/foo:bar"
Output of containerd --version:
containerd github.com/containerd/containerd v1.4.0-beta.1-58-g9f154d14 9f154d145e1acdc64b11aa8e0a705e85ff4d042e

Any other relevant information:
For comparison loading the image with docker load -i bar.tar gets the expected repo name.

Nevermind, this appears to be intentional. https://github.com/containerd/containerd/blob/f823c377086bccc8c35a0ddfe12a02a0c6c112af/images/archive/importer.go#L61-L63


when I use nimble example bleprph, it will assert in ble_svc_gap_init function (IDFGH-8211)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
4.1.3
Operating System used.
Windows
How did you build your project?
Command line with idf.py
If you are using Windows, please specify command line type.
No response
Development Kit.
ESP32-DevKitC-V4_
Power Supply used.
USB
What is the expected behavior?
example can run right
What is the actual behavior?
assertion "rc == 0" failed: file "/mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/bt/host/nimble/nimble/nimble/host/services/gap/src/ble_svc_gap.c", line 297, function: ble_svc_gap_init
abort() was called at PC 0x401159c3 on core 0
0x401159c3: __assert_func at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/newlib/newlib/libc/stdlib/assert.c:62 (discriminator 8)
ELF file SHA256: b6c1c8451b07a9f2
Backtrace: 0x4008ebc1:0x3ffbb590 0x4008ef65:0x3ffbb5b0 0x401159c3:0x3ffbb5d0 0x400d5f97:0x3ffbb600 0x400d5ab9:0x3ffbb620 0x400d5972:0x3ffbb640 0x400d1bf2:0x3ffbb660 0x4009029d:0x3ffbb680
0x4008ebc1: invoke_abort at /mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/esp32/panic.c:157
0x4008ef65: abort at /mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/esp32/panic.c:174
0x401159c3: __assert_func at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/newlib/newlib/libc/stdlib/assert.c:62 (discriminator 8)
0x400d5f97: ble_svc_gap_init at /mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/bt/host/nimble/nimble/nimble/host/services/gap/src/ble_svc_gap.c:297 (discriminator 1)
0x400d5ab9: gatt_svr_init at /mnt/hgfs/test/bleprph/build/../main/gatt_svr.c:196
0x400d5972: app_main at /mnt/hgfs/test/bleprph/build/../main/main.c:372 (discriminator 2)
0x400d1bf2: main_task at /mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/esp32/cpu_start.c:597 (discriminator 2)
0x4009029d: vPortTaskWrapper at /mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/freertos/port.c:143
Steps to reproduce.
1、build the nimble example,bleprph
2、flash to  esp32 devc board
3、 start and run assert
Debug Logs.
I (12) boot: ESP-IDF v4.1.3-dirty 2nd stage bootloader
I (12) boot: compile time 09:47:29
I (12) boot: chip revision: 1
I (14) boot_comm: chip revision: 1, min. bootloader chip revision: 0
I (21) boot.esp32: SPI Speed      : 40MHz
I (26) boot.esp32: SPI Mode       : DIO
I (30) boot.esp32: SPI Flash Size : 4MB
I (35) boot: Enabling RNG early entropy source...
I (40) boot: Partition Table:
I (44) boot: ## Label            Usage          Type ST Offset   Length
I (51) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (58) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (66) boot:  2 factory          factory app      00 00 00010000 00100000
I (73) boot: End of partition table
I (78) boot_comm: chip revision: 1, min. application chip revision: 0
I (85) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x0d1d0 ( 53712) map
I (114) esp_image: segment 1: paddr=0x0001d1f8 vaddr=0x3ffbdb60 size=0x02e20 ( 11808) load
I (119) esp_image: segment 2: paddr=0x00020020 vaddr=0x400d0020 size=0x5afe4 (372708) map
0x400d0020: _stext at ??:?

I (263) esp_image: segment 3: paddr=0x0007b00c vaddr=0x3ffc0980 size=0x005a4 (  1444) load
I (264) esp_image: segment 4: paddr=0x0007b5b8 vaddr=0x40080000 size=0x00404 (  1028) load
0x40080000: _WindowOverflow4 at /mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/freertos/xtensa_vectors.S:1778

I (270) esp_image: segment 5: paddr=0x0007b9c4 vaddr=0x40080404 size=0x15ed4 ( 89812) load
I (330) boot: Loaded app from partition at offset 0x10000
I (330) boot: Disabling RNG early entropy source...
I (330) cpu_start: Pro cpu up.
I (334) cpu_start: Application information:
I (339) cpu_start: Project name:     bleprph
I (344) cpu_start: App version:      1
I (348) cpu_start: Compile time:     Sep  2 2022 09:46:26
I (354) cpu_start: ELF file SHA256:  b6c1c8451b07a9f2...
I (360) cpu_start: ESP-IDF:          v4.1.3-dirty
I (365) cpu_start: Starting app cpu, entry point is 0x40081360
0x40081360: call_start_cpu1 at /mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/esp32/cpu_start.c:276

I (366) cpu_start: App cpu up.
I (376) heap_init: Initializing. RAM available for dynamic allocation:
I (383) heap_init: At 3FFAFF10 len 000000F0 (0 KiB): DRAM
I (389) heap_init: At 3FFB6388 len 00001C78 (7 KiB): DRAM
I (395) heap_init: At 3FFB9A20 len 00004108 (16 KiB): DRAM
I (401) heap_init: At 3FFBDB5C len 00000004 (0 KiB): DRAM
I (407) heap_init: At 3FFC3B68 len 0001C498 (113 KiB): DRAM
I (414) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (420) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (426) heap_init: At 400962D8 len 00009D28 (39 KiB): IRAM
I (433) cpu_start: Pro cpu start user code
I (451) spi_flash: detected chip: generic
I (451) spi_flash: flash io: dio
I (452) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (537) BTDM_INIT: BT controller compile version [5688ed5]
I (537) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (547) phy_init: phy_version 4670,719f9f6,Feb 18 2021,17:07:07
test
svr init
assertion "rc == 0" failed: file "/mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/bt/host/nimble/nimble/nimble/host/services/gap/src/ble_svc_gap.c", line 297, function: ble_svc_gap_init
abort() was called at PC 0x401159c3 on core 0
0x401159c3: __assert_func at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/newlib/newlib/libc/stdlib/assert.c:62 (discriminator 8)

ELF file SHA256: b6c1c8451b07a9f2

Backtrace: 0x4008ebc1:0x3ffbb590 0x4008ef65:0x3ffbb5b0 0x401159c3:0x3ffbb5d0 0x400d5f97:0x3ffbb600 0x400d5ab9:0x3ffbb620 0x400d5972:0x3ffbb640 0x400d1bf2:0x3ffbb660 0x4009029d:0x3ffbb680
0x4008ebc1: invoke_abort at /mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/esp32/panic.c:157

0x4008ef65: abort at /mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/esp32/panic.c:174

0x401159c3: __assert_func at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/newlib/newlib/libc/stdlib/assert.c:62 (discriminator 8)

0x400d5f97: ble_svc_gap_init at /mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/bt/host/nimble/nimble/nimble/host/services/gap/src/ble_svc_gap.c:297 (discriminator 1)

0x400d5ab9: gatt_svr_init at /mnt/hgfs/test/bleprph/build/../main/gatt_svr.c:196

0x400d5972: app_main at /mnt/hgfs/test/bleprph/build/../main/main.c:372 (discriminator 2)

0x400d1bf2: main_task at /mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/esp32/cpu_start.c:597 (discriminator 2)

0x4009029d: vPortTaskWrapper at /mnt/hgfs/Project/gate_code/esp-idf-v4.1.3/components/freertos/port.c:143

More Information.
No response

Hi @luacerbb ,
I quickly tried on mentioned tag (v4.3.1) and the application is executing normally. Attached console log for same.
Can you please ensure you have a clean build ? Please share the sdkconfig of your setup.
Also, it looks like failure happens during registration of ble_svc_gap_defs .  Kindly confirm, if there are any other modifications done, that can give us better understanding of the issue.
Thanks.
console_log.txt

Hi @luacerbb , kindly discard the above request. I will update on this asap.

Hi @luacerbb  ,      Please check if attached patch helps.
0001-Nimble-Check-stack-initialization-status-before-exec.txt

thanks,when I attach patch,it work well.


operator precedence enforcement

PREFACE: I have searched and not found an issue about considerations that may have already been made about this, if I missed the issue I'd close this one.
blogpost - reasoning behind this
in short: x = a & b + c * d && e ^ f == 7 is something nobody wants to read

And even for operators that are used together - like && and || -
it is difficult to give them a relative precedence while keeping it intuitive.
So instead of picking a side, we can just pick none:
Let them have the same precedence and make it an error to mix them without parentheses.

goal:

A carefully designed operator precedence enforces the common guideline of “use parentheses when it is not intuitive”. It is now impossible to write unclear expressions as the compiler will simply reject them.

a suggestion as to how to order the precedence is at the end of the post

from learning crystal so far I like strong typedefs and very reasonable auto-format which really helps the programmer and thus I think enforced clear operator precedence would fit in very well and keep code clean and safe
and it would definitely be better to post in a request for comment pool

In crystal, it parses as x = (a & (b + (c * d))) && ((e ^ f) == 7) so yeah I also think it's wrong!

Sure. Please write down the algorithm to implement this.

what algorithm do you mean exactly?
I am afraid this will be more than I am able to do at the moment.

I mean, you propose that the compiler enforces parentheses where it would be otherwise ambiguous for a user to understand it. For that to work, the compiler has to have a formal algorithm for this, or maybe a bunch of heuristics. We don't have time to develop this, and I personally think that if it's too confusing to read, just add parentheses, it's not that terrible. No other programming language that I know of enforces parentheses. So if you want to see this happen you'll have to write the code or the algorithm that implements this. But if you don't have time for this, I think we can close this.

this was the proposed ordering:
#The final operator precedence
If we just take the most common operators, I identify the following “categories” of operators:

Logical operators: &&, ||, !

Comparison operators: ==, !=, <, <=¸ …

Mathematical operators: binary/unary + and -, *, and /.

Bitwise operators: ~, &, |, ^, << and >>

Other unary operators like function call, array subscript or member access

It makes sense to assign them the following relative precedence:

unary operators > mathematical/bitwise operators > comparison operators > logical operators

I personally think that if it's too confusing to read, just add parentheses, it's not that terrible.

same goes for formatting code, if it is not formatted well, I just format it well
...the issue arises not only if oneself cannot read the expression but the main  reason is when someone else is reading/ debugging/ altering the code that seemed to be perfectly valid to someone else
but if there is nobody who can work this out in practice than you can probably close it, agreed

I'm pretty sure we are never going to implement this, so I'm closing this.
For example, is this confusing?
1 * 2 + 3 * 4

Maybe for some it is, if they don't know that * has a higher precedence. Do we force parentheses?
(1 * 2) + (3 * 4)

I don't know, the first one was probably easy to understand. The same probably applies to other operators. And in doubt, just use parentheses to make sure you get the correct association.

like in the p=blogpost
Then the following expressions are all well-formed:

a * b + c == foo & a
a && (!b || c)
array[a] + 32 < ~a | b

would work, it is not about enforcing brackets on every operator but mainly between classes of operators

OK, I'll reopen this. Maybe some day someone will implement this...

keep faith ;)
I'll try to come back with more details!

So I made some more examples, tried to come of with more explicit rules.
If you think this helps to specify a possible approach/ need different examples I can try to work out further information.
sorted by strength of binding: very strong to weak
1. member access and function calls
Bar.foo(3)
i = Bar.member
i = foobar[0]

2. Math und Binary ops
+ - * /         # these two rows (Math and Binary) must not mix without braces
~ ^ | & << >>   # these two rows (Math and Binary) must not mix without braces

3. comparison ops
== != <= <

4. logical ops
! `>` && ||

=
=   # is not a valid op, has to be put in braces

no mixed chaining rules
1. ^ | &    # bitwise
2. << >>    # bitshift
3. && ||    # logical ops

a ^ b & c	# all of those mixed chains of (1),(2),(3) are vorbidden
a << b >> c # all of those mixed chains of (1),(2),(3) are vorbidden
a && b || c # all of those mixed chains of (1),(2),(3) are vorbidden

a && b && c # allowed, no mixed chain

examples
!a && b         # all other combinations of !a ...xyz are vorbidden
!a || b         # all other combinations of !a ...xyz are vorbidden
a  && b || c    # vorbidden, chaining of || and &&
!a.foo()		# vorbidden, mixed ! with other than && or || without braces

~a & b          # allowed, like !a || b
~a + b          # vorbidden, mixed math and binary ops
~( a + b )      # allowed

a & b == c      <=> ( a & b ) == c

Mult* and Division/ bind stronger than Plus+ and Minus-
4 * 3 + 1 # works
4 * 3 ^ 1 # does not work, mixed binary and math

a = 1 + b = c       # does not work, assignment has no precedence as op
a = 1 + ( b = c )   # works

examples of all allowed combinations
locigal <-> comp
a && b < c      <=> ( a && b ) < c

bin_math <-> comp
a + b == c      <=> ( a + b ) == c
a << b == c     <=> ( a << b ) == c

bin_math <-> logical
a + b && c      <=> ( a + b ) && c
a << b && c     <=> ( a << b ) && c

all the following work:
because 1. member access and function calls have highest precedence.
where Bar.foo() can be one of the following three:
Bar.foo(3)
Bar.member
foobar[0]

a + Bar.foo()
a == Bar.foo()
a || Bar.foo()
a || !Bar.foo()

Just a small note: A logical operator should have higher precedence than a comparator, making 1 && b < c equal `a && (b < c). This fits with your ordering of having comparators between logical ops and math/binary ops.

thanks, I updated the notes accordingly

I would support this if it was a PR. Not sure if we'll have time to implement this for a long time though.

I would see this as a feature for the formatter because it's just a formatting issue. The parser doesn't need parenthesis to disambiguate because the precedence rules are clear, so it doesn't need to enforce it. It only helps humans to follow the code better, so that's a job for the formatter. As long as it's not too intrusive, this should be fine.


Apparently Thread#run should be called Thread#schedule.

Also, #join blocks the parent and runs the child thread, which is what I
wanted. Also apparently, no explicit is necessary.
The ruby docs on threading are really clear!

:+1: Beautiful


pull access denied

I tried to change the xserver service in my Balena project from balenablocks to balenalabs.
Is it correct that balenalabs is newer than balenablocks ? Or which one do you recommend ?
The following error occurs when trying to go from balenablocks to balenalabs version of xserver:
Before, I had the following inside the docker-compose.yml File:
xserver:
        image: balenablocks/xserver
        restart: always
        privileged: true
        labels:
            io.balena.features.dbus: "1"
        volumes:
            # when we start, a UNIX socket is mounted in this directory, for communication to the X server
            - xserver-volume:/tmp/.X11-unix

For the above, everything works and I can push to the Balena fleet without problems.
However, if I exchange the balenablocks/xserver with bh.cr/balenalabs/xserver-amd64, such as:
xserver:
        image: bh.cr/balenalabs/xserver-amd64
        restart: always
        privileged: true
        labels:
            io.balena.features.dbus: "1"
        volumes:
            # when we start, a UNIX socket is mounted in this directory, for communication to the X server
            - xserver-volume:/tmp/.X11-unix

Then I get the following error message when trying to push to the balena fleet :
[Success]        Successfully uploaded images
[Error]          Some services failed to build:
[Error]            Service: xserver
[Error]              Error: (HTTP code 404) unexpected - pull access denied for bh.cr/balenalabs/xserver-amd64, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
[Info]           Built on x64_01
[Error]          Not deploying release.
Remote build failed

I also tried image: bh.cr/balenalabs/xserver (i.e. without the -amd64). But did not help, same error !!!
Then I tried to upload my own custom Balena Block from a cloned xserver instance. It did it. It worked and under Balena Blocks I find my own custom Block:

Inside the docker-compose.yml File, I exchanged to now take my custom Block, such as:
image: bh.cr/<slug_name>/xserver

or also:
image: bh.cr/<slug_name>/xserver-adm64

It did not work !!!!!!!
Again, when trying to push my project to the fleet, I end up once more with the exact same error message:
[Success]        Successfully uploaded images
[Error]          Some services failed to build:
[Error]            Service: xserver
[Error]              Error: (HTTP code 404) unexpected - pull access denied for bh.cr/balenalabs/xserver-amd64, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
[Info]           Built on x64_01
[Error]          Not deploying release.
Remote build failed

What do I need to do ?
Which of the two xserver possibilities do you recommend ? (i.e. balenablocks/xserver or balenalabs/xserver ??)

Should be fixed, closing in favour of: https://forums.balena.io/t/xserver-image-pull-access-denied/366366

excellent, now it works ! Thank you very much


Misoptimization of wide shifts

Version
Yosys 0.37+29 (git sha1 3c3788ee2, clang 10.0.0-4ubuntu1 -fPIC -Os)
On which OS did this happen?
Linux
Reproduction Steps
Hi there!
I stumbled into an issue where a wide shift gets some misoptimization.
I could reduce it to a very simple circuit, that is packaged into this repository for helping you reproduce the issue.
Interestingly, reducing the shift operand width by 1 more will clear the bug.
Please let me know if you require any more information or help, or please let me know if I did something wrong here.
Thank you!
Flavien
Expected Behavior
no optimization
Output: 00000000
with optimization
Output: 00000000
Actual Behavior
no optimization
Output: 00000000
with optimization
Output: ffffffff

Looks like the blame is on SigSpec::as_int overflowing here
https://github.com/YosysHQ/yosys/blob/80511ced71b657e77a5389d54c6aa045c047ccaf/passes/opt/opt_expr.cc#L1306

This might be the same bug as #4010. I have some work in progress to detect and error out on as_int overflows in general together with some fixes for places where it could overflow (like this optimization) but I haven't found the time finish it.


Ignore CDL history attribute leading timestamp.

The PR changes the behaviour of iris.tests.assertCDL to ignore the leading timestamp of the CF-netCDF history attribute.
This is desirable, as when testing the CDL representation of a cube generated by functionality that appends/creates the history attribute with a CF-metadata recommended time stamped history entry, it is not possible to compare archived expected results with actual results as the time stamp will always differ.
Seeking advice: We don't seem to have test coverage of changes to testing-framwork code i.e. iris.tests.__init__ (that, or I simply can't find them). Hence no additional test cases have been bundled with this PR. What's the best working practice here? Apart from all existing tests passing ...

We don't seem to have test coverage of changes to testing-framwork code

No, we have no tests of the tests. Personally I don't much like the idea of going down that road! Where would it end? Tests of tests of tests? Tests of tests of tests of tests? :scream:

The slightly weird thing is that this PR is changing the behaviour of an Iris assertion (and hence the tests that use the assertion), but nothing in Iris actually wants this change or is even affected by it. :confused: What's to stop someone coming along and trimming this behaviour out when they notice it's not being used? That would be perfectly understandable as the tests aren't part of the public API - we have no duty to preserve their behaviour from once minute to the next.
I started typing this comment with the opinion that I didn't mind the proposed functionality, but the point about the public API has changed my mind. I'm not comfortable with the idea of treating the tests as part of the public API, so I'm :-1: on balance.


[3.2] Dynamically load libpulse.so.0 and libasound.so.1 on Linux

By generating stubs using https://github.com/hpvb/dynload-wrapper we
can dynamically load libpulse and libasound on systems where it is available.
Both are still a build-time requirement but no longer a run-time dependency.
For maintenance purposes the wrappers should not need to be re-generated
unless we want to bump pulse or asound to an incompatible version. It is
unlikely we will want to do this any time soon.
cherry-pick from 09f82fa6eaa4dac6dd1743724b82a99906e64861

Needs a rebase to fix CI, somehow it still tried to use AppVeyor.

Thanks!


[FLINK-21400] Store attempt numbers outside ExecutionGraph

Introduces a data structure to store the attempt numbers outside the ExecutionGraph. It's really just a simple Map<ExecutionVertexID, Integer> which ties a specific vertex+subtask to an attempt count.
Counts are set when an execution is registered at the EG, and retrieved when the ExecutionVertex creates a new Execution. The current attempt count is also still stored in the Execution, making the change less invasive (for example, resetForNewExecution continues to work without modifications).
One thing is that, as is, the semantics when it comes to rescaling are a bit funky.
ScaleUp:
If you begin with p=1 and an attempt count of 4, and then rescale to p=2, then what should the attempt count be for both subtasks?
In this version the attempt count for subtask 1 would be retained, while subtask 2 starts at 0.
Setting both to 0 would also make sense, but if we downscale again to p=1 then it would be nice if the attempt count had some relation to the original count.
Alternatively we could try to derive the attempt count for subtask 2 from other subtasks; in this example the obvious choice would be 2, because we're just replicating subtask 1.
ScaleDown:
The main issue arises when scaling down where the subtask with the largest index has the highest attempt count; currently this count would be lost. So you have p=2, and subtask 2 has an attempt count of 4, and now you scale down to p=1. The attempt count would now be solely determined by subtask 1, although we in essence just merged the two.
Overall, I don't think resetting attempt counts to 0 is an option, because they can be used to gauge the health of a vertex, and we'd run into collisions within metrics if we ever re-use a subtask+attempt combination.
The current approach is by far the simplest, and is the only option iff we want to adhere to these rules:

every combination of subtask + attempt count is only used once
the attempt counts for a given subtask over time always form a continuous series starting at 0

But I'm quite interested in what other people think about this.

One issues is that the semantics for the attempt numbers as a whole are not really well-defined; internally we use it to differentiate different deployments (for example, when we tried to create deterministic ExecutionAttemptIDs), but users probably think more of it like "how often did this subtask fail".
I'm not even sure how valuable the attempt number is as a whole; I'd think that users either want to identify instable TaskManagers (where the question is "how many failures occur on this TM") or operators ("how often do subtasks of this operator fail?"), but the attempt number fulfills neither because it is also incremented for subtasks did not actually cause a failure.

The semantics of the execution attempts is a very good question. I think I would keep it as you've implemented it in the PR: We keep monotonously increasing attempt counters for all ever seen Executions.

In case of a scale up, new Executions with an attempt number 0 will be inserted
In case of a scale down, some of the Executions won't be restarted and, thus, their attempt counter stays as is
Increasing the attempt counter for those Executions which will be restarted.

Concerning the implementation, have you considered not introducing the counter to the ExecutionGraph but rather maintaining it outside? What this would require is the following:

A way to give an immutable map of attempts to the EG when it is created in order to initialize the Executions.
A way to retrieve the set of last attempt counters when the EG is terminated. This value could be used to update the external attempt counter.

The PR now contains a proper implementation.


IOT improvements

Closes #2079
Closes #3333
Closes #3890
Closes #4694
Supercedes #4697

Codecov Report

Merging #4848 (3e5cc05) into master (de559e4) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff            @@
##           master    #4848    +/-   ##
========================================
  Coverage   95.14%   95.15%
========================================
  Files         594      594
  Lines       64088    64189   +101
========================================
+ Hits        60978    61076    +98
- Misses       3110     3113     +3

Flag
Coverage Δ

servertests
35.91% <32.77%> (-0.02%)
:arrow_down:

unittests
95.09% <100.00%> (+<0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

moto/iot/exceptions.py
90.24% <100.00%> (+0.50%)
:arrow_up:

moto/iot/models.py
93.03% <100.00%> (+0.76%)
:arrow_up:

moto/iot/responses.py
99.01% <100.00%> (+0.09%)
:arrow_up:

moto/iotdata/models.py
94.16% <100.00%> (+0.04%)
:arrow_up:

moto/iotdata/responses.py
100.00% <100.00%> (ø)

moto/resourcegroupstaggingapi/models.py
83.66% <0.00%> (-0.58%)
:arrow_down:

moto/route53/responses.py
97.18% <0.00%> (-0.41%)
:arrow_down:

moto/sns/models.py
96.82% <0.00%> (-0.40%)
:arrow_down:

moto/sqs/models.py
96.75% <0.00%> (-0.18%)
:arrow_down:

moto/config/models.py
99.21% <0.00%> (-0.14%)
:arrow_down:

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update de559e4...3e5cc05. Read the comment docs.


Bug: unable to build with tsc when not using @middy/core

Bug description
The code doesn't compile using tsc when using the Logger module without middy.
Expected Behavior
Should be able to use Logger without the need to import middy when its not used. This is possible by importing the submodule via import { Logger } from "@aws-lambda-powertools/logger/lib/Logger";. If this is the solution maybe the documentation should reflect that.
Current Behavior
Middy is required atleast as the dev dependency while compiling with tsc. This is not a problem when esbuild or ts-node is used.
Possible Solution
Steps to Reproduce
Steps:

npm install and import Logger using import { Logger } from "@aws-lambda-powertools/logger";
Build project using tsc
Expect this error Cannot find module '@middy/core' or its corresponding type declarations.

package.json
{
  "scripts": {
    "build": "npm i && tsc logger-demo.ts"
  },
  "dependencies": {
    "@aws-lambda-powertools/logger": "1.2.0",
    "@types/aws-lambda": "8.10.101"

  },
  "devDependencies": {
    "typescript": "4.7.4"
  }
}

logger-demo
import { Logger } from "@aws-lambda-powertools/logger/lib/Logger";

const logger = new Logger();
logger.info("Testing logger");

tsconfig.json
{
    "compilerOptions": {
        "experimentalDecorators": true,
        "target": "ES2021",
        "module": "CommonJS"
    }
}

Environment

Powertools version used: 1.2.1
Packaging format (Layers, npm): npm
AWS Lambda function runtime: node16
**Debugging logs:**n/a

Related issues, RFCs
https://github.com/awslabs/aws-lambda-powertools-typescript/issues/1068

@obiwabrakenobi, not at this time, however we're happy to review & merge any PR coming from the community to fix the issue in a more timely manner.

I'll have a look into it

Thank you!

Hi "AWS Lambda Powertools Team",
We are going to use Tracer, Logger, and Metrics for observability on our project and we are building our codebase with "tsc". Middy is still under consideration, so it would be nice to avoid redundant dependency on the extra library we don't use.
Could you please share updates related to these 3 issues #1080, #1081, #1068, and when it's planned to be fixed?

We just released version 1.5.1 that should fix the issue.
You can read more about the fix in #1225, however now you should be able to transpile your functions with tsc without having @middy/core in your codebase.
I have tested v1.5.1. in the repository provided here, and the code now compiles successfully to this:
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
const tracer_1 = require("@aws-lambda-powertools/tracer");
// import { captureLambdaHandler } from '@aws-lambda-powertools/tracer'
const tracer = new tracer_1.Tracer();

Hope this helps!


Adding application/json input handling.

References #531

Added additional unit test for json data.
PHP input is sanitized through json_decode
If there is binary or invalid data in the json input the entire payload will be discarded. Could run input through sanitize first?

Also References #400 #124

@Jean85 I added the negative test case. This was good as it identified a need to abstract the json encode stuff for future XML tests.


[WIP] Composite tuple/algo

[x] Creation of a new view to create/retrieve/list (a) compositetraintuple(s)
[ ] Modification of the view to create a new compute plan also with compositetraintuples
[ ] Creation of a new view to get a compute plan
[x] Creation of a new task corresponding to a compositetraintuple (two in/out models only, one with restricted permissions to the node):

preparation of the task
execution of the task
reporting of the task

[x] Modification of prepare task for testtuple to get two inModels in case of a compositetraintuple
Algo
[x] Create a new view to create/retrieve/list compositealgo

Associated PRs:

cli
chaincode
tools

Does assets.py was generated ? I don't see any modification on generate_assets.py 🐙

I second @Kelvin-M point: we have to re-generate assets.py.
This should include new composite models in the model list which will make the filters fail on the /model view. The reason for that is that the filter_list for model explicitely looks for model['traintuple']['outModel'] while the composite models have neither traintuple nor outModel.
I'll make a PR to update the filter_list function and the assets.

LGTM !
This is introducing a lot of code that could be factorized. This could be done later in a separate PR.


adds selector hasChildWith

a way to find an element that contains particular text. currently, this gives no result: Query.find [ tag "button", text "Click Me" ] (I believe this happens because the text node that matches the text selector is not itself a button tag? So maybe this could be solved by adding a Selector.havingChild : List Selector -> Selector?) (#32)

this adds havingChild.

I think @rtfeldman's suggestion can be implemented using hasChildWith, as can withText.
I think hasChildWith is a missing building block that will enable people to write more complex selectors on their own.

I like @rtfeldman's proposal for containing. I think that's easier to understand than hasChildWith.


Make the elements responsive

Changes demo - https://praveen-murali.neetorecord.com/watch/f5c86637-29ec-4e83-80a7-8b15d87cf9a1

Fixes #1270
Fixes #1271
Fixes #1272

Description

Made the images responsive.
Made the videos responsive.
Made the video embeds responsive.
Fixed the spacing inconsistencies in figure captions.

Checklist

[ ] I have made corresponding changes to the documentation.
[ ] I have updated the types definition of modified exports.
[x] I have verified the functionality in some of the neeto web-apps.
[ ] I have added the necessary label (patch/minor/major - If package publish
is required).

Reviewers
@AbhayVAshokan _a Please review.

@praveen-murali-ind _a Self assigning to fix one another issue.


fix(metadata-jobs): use KafkaEventConsumerFactory in metadata jobs

Remove the separately maintained KafkaEventConsumerFactory for mae-consumer
and mce-consumer. This enables the metadata consumer jobs to work with
the AWS Glue schema registry again, as was the case before (in v0.8.10).
Closes #3373
Does it make sense to remove the separate kafka configurations altogether with this PR?
Checklist

[x] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[x] Links to related issues (if applicable)
[ ] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable)

Sorry about the delay in fixing this issue. Seems like this part of code got mistakenly reverted. https://github.com/linkedin/datahub/pull/3675 Here is the PR. Feel free to close this PR!

Alright, thanks!


Create jit version of nonzero op

Coming up and related to #1283
Current run seeing the following output error after fixing issue from #1587
Run instruction: main:@711 = gpu::nonzero(main:@709,main:@710) -> int64_type, {1, 5940000}, {5940000, 1}
driver: /code/AMDMIGraphX/src/targets/gpu/device/include/migraphx/gpu/device/shape.hpp:53: migraphx::gpu::device::hip_shape<1>::hip_shape(const migraphx::shape &) [N = 1]: Assertion `std::all_of(s.lens().begin(), s.lens().end(), &is_divisor_encodable)' failed.

prefix_scan operator needs to be implemented first.

Okay I'll make that another issue then and link to this.

Moving to backlog, issue with this went away after further fixes have gone into develop


Is there a way to get the wizard_path helper to use slash instead of dot for the step path?

Right now it seems like do something like verifications_helper(:step1) returns /verifications.step1 rather than /verifications/step1, which is what I want. /verifications/step1 is the path as it appears in the browser when the controller renders the page.
Mostly interested in this because I'd like to do some integration testing with Capybara, and will occasionally have places where I'd like to test the redirect path to a Wicked Controller step. Currently I've written a small helper to output the path with a / instead of a . but wondering if there's a more canonical way of doing this.

You shouldn't be getting a dot. I remember seeing this before, but I don't remember the specific cause. I think maybe you have to explicitly pass id: to the path helper, but I don't totally remember. DId you ever get this figured out?

Any where in the docs we could make this better or clearer? I would take a doc pr.


Add a TSLint config to the default init

The default project structure should have a TSLint file. It would probably have a minimal number of rules actually set, though.

Do you have any basic TSLint config working fine with angular2 app  ?

You probably just want the output of tslint --init as a starting point and can customize it later as best practices develop.

OK I will use this default config

And should we add out-of-the-box command defined in package.json ? Or a new CLI command ng lint ? or let the user install tslint on his own ?


Issues before final exam - Abj

recovery optimization Abj
refactoring Abj
remove backend loggers Abj
move index recovery to log manager

Remove unnecessary "this"s


issues with !DISCORD_COMMAND unlink

i'm having issues with using !DISCORD_COMMAND unlink:

if the bridge is in auto mode, when I call the unlink command it disconnects and then immediately reconnects
if I first call the auto command to disable auto mode, subsequently calling unlink does nothing

For part 1, that is actually expected behaviour if there's still people on both the mumble end and the discord end. If you want to kill the bridge while people are on both sides you should do as you did in part 2 and toggle it to manual mode first.
For part 2, yeah I see the issue. Looks like when the bridge is toggled to manual it clears the Discord voice channel ID the bot is paying attention to. I think I wrote that code and I don't remember why I did, so I'm removing the line. Shouldn't be a problem anyway; I believe that value is only referenced when the link and unlink commands are parsed and when "link" is processed it pulls the channel ID from the sender.


chore(esp32c3): add core crate in main workspace

This change moves most of the driver and initialization code currently
in platforms/esp32c3-buddy into a new platforms/esp32c3/core crate,
which is part of the main workspace, with only the actual bins built for
the target binaries in their own workspace.
This allows most of the driver code to participate in stuff like cargo check and cargo fmt runs in the main workspace, which should make our
lives a bit easier. It was necessary to ensure that incompatible
features of the critical-section crate were not enabled in the main
workspace, so I did some tweaking of the Cargo features to ensure that
we weren't enabling an incompatible configuration in core crates.
I also fixed some Clippy lints that immediately showed up in VS Code once
the C3 code was moved into the main workspace.

Hmm, it looks like we can't get rid of the dependency on esp-riscv-rt in esp32c3-hal, so any code that uses the HAL will always include RISC-V assembly in its build. This means that having code that uses the HAL in the main workspace will cause build failures while trying to build for the host's target (assuming the build host is not RISC-V): https://github.com/tosc-rs/mnemos/actions/runs/5717974422/job/15492976270?pr=209
So, I guess this just straight up won't work. Oh well.

We could probably do something like this if the esp32c3-core crate depended on the esp32c3 PAC crate instead of esp32c3-hal, which would let us just use the PAC types without compiling the inline assembly in the rt crate, which would only be compiled when building for the target. I don't know if changing the dependencies like this would really be worth the effort, though.


Compress image before feed to NN if image is too large

Bug Description

I did a cat and dog image classification task, only used 100 pictures to train, and a RuntimeError appeared on a single 8GB system: $ Torch: not enough memory: you tried to allocate 0GB. The image input is 100 * 224 * 224 *3. This problem occurs. The input standard for reducing the image is 100 * 32 * 32 * 3. This problem is not the case, but the accuracy is very low for the training result. The final evaluation value of the second classification. Less than 0.6.
The specific issues are as follows:
╒==============================================╕
|               Training model 1               |
╘==============================================╛
Using TensorFlow backend.
Current Epoch:   0%|                             | 0/1 [00:00<?, ? batch/s]Exception ignored in: <bound method tqdm.__del__ of Current Epoch:   0%|                             | 0/1 [07:59<?, ? batch/s]>
Traceback (most recent call last):
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/site-packages/tqdm/_tqdm.py", line 885, in __del__
    self.close()
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/site-packages/tqdm/_tqdm.py", line 1090, in close
    self._decr_instances(self)
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/site-packages/tqdm/_tqdm.py", line 454, in _decr_instances
    cls.monitor.exit()
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/site-packages/tqdm/_monitor.py", line 52, in exit
    self.join()
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/threading.py", line 1053, in join
    raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/site-packages/autokeras/search.py", line 296, in train
    verbose=verbose).train_model(**trainer_args)
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/site-packages/autokeras/model_trainer.py", line 103, in train_model
    self._train()
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/site-packages/autokeras/model_trainer.py", line 147, in _train
    outputs = self.model(inputs)
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/site-packages/autokeras/graph.py", line 610, in forward
    temp_tensor = torch_layer(edge_input_tensor)
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/site-packages/torch/nn/modules/batchnorm.py", line 66, in forward
    exponential_average_factor, self.eps)
  File "/root/anaconda3/envs/automl-formal/lib/python3.6/site-packages/torch/nn/functional.py", line 1254, in batch_norm
    training, momentum, eps, torch.backends.cudnn.enabled
RuntimeError: $ Torch: not enough memory: you tried to allocate 0GB. Buy new RAM! at [/pytorch/aten/src/TH/THGeneral.cpp:204]

Reproducing Steps
Steps to reproduce the behavior:

Step 1: Choose 1 CPU to use (among 8 CPUs):
Step 2: Run the Cat and Dog Wars classification task, select 50 cats and 50 dogs for the classification task, enter the image parameters: 100 * 224 * 224 *3

Expected Behavior

Succeed in running the classification task without this error
Image classification accuracy increased to an acceptable greater than 80%
Setup Details
Include the details about the versions of:

OS type and version:
Python: 3.6
autokeras: 0.2.14
scikit-learn:0.19.1
numpy:1.14.5
keras:2.2.2
scipy:1.1.0
tensorflow:1.10.1
pytorch:

Additional context

@satyakesav
First, you may examine whether the data is loaded to GPU memory batch by batch, or all together.
If is loaded all together, try to change it to load batch by batch.
Second, if the image is oversize (the total size is larger than 64643), resize the image before feed into the NN, according to their original length/width ratio. Keep the size just right.
Thanks.


Do not sort overlays

Fixes https://github.com/holoviz/hvplot/issues/468
Fixes https://github.com/holoviz/hvplot/issues/469

Coverage decreased (-0.02%) to 80.545% when pulling 77164b7b74daeca8eeb0c6d9d06f73f4645675c6 on kde_sort into 5ebc40396de6f4c2b87858aefcd64c6daae22b47 on master.


GH-550: master to 2.2; fix tangles

Fixes https://github.com/spring-projects/spring-kafka/issues/550

package tangle ....listener and ....listener.config - remove config package
class tangles between ContainerProperties and the listener containers

AckMode moved to properties
Error handler setters moved from properties to containers

class tangle between listener adapter and handler adapter due to ResultHolder

Sorry - forgot about the Windows problem. You should squash the last 3 commits; retaining 2 commit.
Yes, please add the #278 comment (to the squashed second commit).
Thanks.

I also noticed that we have several not closed KafkaConsumers in some tests.
Fixing...
Thanks for confirmation!

Merged as https://github.com/spring-projects/spring-kafka/commit/b048aaa8f00045e4cc65d5d137b1aa372beca3a2 and https://github.com/spring-projects/spring-kafka/commit/9f724ea27a2b403d3dcc846ce5c7c6805cdff108


Tailer: speed-up by avoiding BAOS.toByteArray()

Coverage increased (+0.004%) to 89.721% when pulling 6a2c89c27fd19feb17c3ee76a29b830f23d02a9d on illes:tailer into 8bb1788ae75318bd5cca0c64893f616c82402399 on apache:master.

The original version is using a variable "charset". Your version ignores that variable?

No, it should not ignore it. charset is used in the overriden toString() method, see Tailer.java:517.

I think I understood the change, and my first impression is that it would do the same avoiding the toByteArray call. However, I'd still be -0 for the change (meaning if others prefer to add it, please go ahead).
My minor nit-picks is that we are overriding a method from ByteArrayOutputStream. (as a developer, I like to know whether my libraries are using lower level SDK API's if any, and how they are using). And I also think a developer could accidentally put back the toByteArray somewhere else, as it's possible to use both (we'd just have to note the way the current code is doing, but sometimes we accidentally use what we are used to).
But probably nit-picking, so as I said earlier, it's OK to merge it if others are OK :+1:

Tailer has changed recently, may you please rebase on master?

Closing, no reply since September of last year.


ExecuteHostCommand & current line buffer usage

Hello,
is it possible to use the immediate contents of the line buffer inside of ReedlineEvent::ExecuteHostCommand when keybinding is used?
Not sure if using API is appropriate for this use-case:
User:

types some command into CLI, e.g.: hello --address
presses keybinding e.g. SHIFT + '?'
keybinding:

does NOT edit the pendign line itself
executes host-command that includes the line being written, e.g. help hello --address
keeps the pending line without any change for firther editing (if applicable)

Is above possible either via keybinding declaration & it's command parameter, or other approach?
Explanation:

I am using reedline together with clap crate in REPL mode
(clap's try_parse_from() insterad of from building from env on binary execution)
Above behavior might greatly complement clap's ability to provide extensive & automatic help sub-commands that typically come "free" with its Command/Subcommand types

We've talked about something like this and would probably welcome a PR but maybe it should be a separate command like ReedlineEvent::ExecuteHostCommandReturnResult or so that we can still keep the existing functionality of ExecuteHostCommand.

Yeah it would be great to have more options to hook in there (like using an external comamnd to fill the line buffer with certain content as well). There might be two options to approach this API wise:

Yield from Reedline::read_line with more different signals and recall the same Reedline object with a specific method. (Less ownership hassle, having to take care that the editor resumes operation correctly after the external command (that might try to write to the terminal itself))
attach closures as callbacks that get the current line buffer and return a new line buffer (for the case where that is relevant).

Would be happy to help, though an attempt of PR is far beyond my current Rust skill level...
After an hour of putting together the new enum variant like ExecuteHostCommandWithBuffer(Box<dyn FnOnce(String) -> String + Send>), i crash into more errors and google mentioning needing Arc to pass closure around.
As there is no similar behavior for any of ReedlineEvent's, i dare not add tons of new "magical words" into the lib's core enum not knowing whether i go into right direction at all :)

We landed some facilities to enable that with #472. Let us know if this API helps you out in this situation. On the nushell side we use this to implement line buffer editing commands via the commandline command (https://github.com/nushell/nushell/pull/6492)


To add the the solution of "Onboard devices without Internet access for Azure connected machine agent"

Hi Team,
The Azure Arc typically used for on premise machines which under the environment of restricted internet connection.
Please consider adding the article of offline installation in command method, it would be helpful for users to get started with Azure Arc service.
Much appreciated!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6036c6de-a2e8-ffd7-50e9-44ae32bf1682
Version Independent ID: f7ab9abc-abd0-09e1-4b64-9ec304495ee5
Content: Connect hybrid machines to Azure using a deployment script - Azure Arc
Content Source: articles/azure-arc/servers/onboard-portal.md
Service: azure-arc
Sub-service: azure-arc-servers
GitHub Login: @MGoedtel
Microsoft Alias: magoedte

@tinaling - Just to make sure I understand, are you asking for supporting documentation based on the scenario of installing the Arc-enabled servers agent on hybrid machines that are in an air-gapped network (meaning, never have connection to the internet)? Can you help me better understand the scenario? Thanks.

Installing the Connected Machine agent and registering the resource in Azure using a manual method on a hybrid-machine is already covered in our documentation where its communication with the internet is restricted. It is covered in this article for both Windows/Linux.
It appears this is properly addressed and due to lack of response to my follow up question, I'm going to proceed with closing this issue. #please-close.


Fix FindPodCommandParameter method when value contains '='

when I use this method to find parameter "--feature-gates", it show in kube-apiserver pods like this :
--feature-gates=ServerSideApply=true,IPv6DualStack=true
it gets 'ServerSideApply' as --feature-gates value.
this is not right.

This seems to be causing the system tests (Deployment, Deployment (globalnet)) to fail with:

[15:15:03.043] [dir=submariner-operator; cl=cluster1; fn=kubectl]$ command kubectl --context=cluster1 get secrets -n submariner-k8s-broker -o jsonpath={.items[?(@.metadata.annotations['kubernetes.io/service-account.name']=='submariner-k8s-broker-client')].metadata.name}
Failed to find the secret's name

This seems to be causing the system tests (Deployment, Deployment (globalnet)) to fail with:

[15:15:03.043] [dir=submariner-operator; cl=cluster1; fn=kubectl]$ command kubectl --context=cluster1 get secrets -n submariner-k8s-broker -o jsonpath={.items[?(@.metadata.annotations['kubernetes.io/service-account.name']=='submariner-k8s-broker-client')].metadata.name}
Failed to find the secret's name

This PR isn't causing that - it's caused by https://github.com/submariner-io/subctl/pull/653 which removed the submariner-k8s-broker-client SA et al.

This PR isn't causing that - it's caused by submariner-io/subctl#653 which removed the submariner-k8s-broker-client SA et al. We're seeing it now after a new subctl with that change was published. I think we can just remove the verify_subm_broker_secrets() check.

I'm not so sure - I commented on https://github.com/submariner-io/subctl/pull/653.

Thanks for the PR @jinglina! :1st_place_medal:

thanks


Enforce content type headers for mock adapter

Currently it is possible to provide raw elixir data for the body parameter used by the mock adapter. This only works for some cases... It is not feasible to try and support every edge case in regard to using this feature. It would be nice if this feature went away and a policy that requires you set the content type header for a mock put in it's place. This eliminates complexity that comes via raw elixir body parameter and would make using the mock adapter a lot less confusing..
Desired behavior:
data = [%{foo: :bar}]
Tesla.Mock.mock fn
  env -> %Tesla.Env{status: 200, body: Posion.encode!(data)}}
end

The above would result in an exception or warning...
"No content type header provided to Tesla.Mock for request ..."
Likewise, passing in elixir data for the body parameter would result in exception or warning.
data = [%{foo: :bar}]
Tesla.Mock.mock fn
  env -> %Tesla.Env{status: 200, body: data}
end

"body parameter for Tesla.Mock must be a bitstring ..."

See #124  as a reference.

I know this might be a pain when mocking json, but believe me sometimes you do want to test response without content type. I think the best we can do is to add more examples to docs and maybe provide some helper functions in Tesla.Mock module that could be imported, such as already mentioned json(body) function.

I agree 👍 Docs and helper functions is the better and more flexible way to go.  Closing this, cheers!


HasDefaultValueSql and ValueGeneratedOnAdd behaviour

Windows, PostgreSQL, lastest releases.
This following behavior seems a little strange:
                // password TEXT NOT NULL

                entity.Property(e => e.Password)
                    .HasColumnName("password")
                    .ValueGeneratedOnAdd()
                    .HasDefaultValueSql("fn_create_password()");
                    .IsRequired();

If user.Password is null, this will throw a not null exception on password while adding. Is this expected?
Also, is there a way to access the user.Password reference inside .HasDefaultValueSql() ?
                    .HasDefaultValueSql(`fn_create_password(user.Password reference?)`);

Thx,

@incompletude Typical use for HasDefaultValueSql is to get a value generated when none was specified. Reading between the lines it seems like what you are attempting is:

Password property can be set or null
Either way, password is included in the insert
Something such as a trigger runs which changes the password as it is being saved
That new value is read back and the entity is updated with the new value

Is this correct? (If so, I will attempt to give some guidance, but I wanted to make sure I am addressing the questions you want answered before going into details.)

Hey @ajcvickers, thx for all your help.
I have done the following:
CREATE FUNCTION tr_criar_ou_alterar_usuario()
RETURNS TRIGGER AS $$
BEGIN

    IF NEW.senha IS NOT NULL THEN

        SELECT
            ces.fn_criar_senha(NEW.senha),
            ces.fn_criar_chave()
        INTO
            NEW.senha,
            NEW.chave;

    ELSE

        NEW.senha := OLD.senha;
        NEW.chave := OLD.chave;

    END IF;

    RETURN NEW;

END; $$
LANGUAGE 'plpgsql';

CREATE TRIGGER tr_usuarios_criar_usuario
BEFORE INSERT OR UPDATE
ON usuarios
FOR EACH ROW
EXECUTE PROCEDURE tr_criar_ou_alterar_usuario();

                entity.Property(e => e.Senha)
                    .HasColumnName("senha")
                    .IsRequired();

                entity.Property(e => e.Chave)
                    .HasColumnName("chave")
                    .IsRequired();

On create, the password is required, and the trigger fills it correctly. On update, if the property is filled, a new password and token is also set.
Is this the correct approach?

@incompletude Sorry for being slow on this. Can you post a a runnable project/solution of complete code listing that shows the behavior of PropertySaveBehavior.Ignore?

EF Team Triage: Closing this issue as the requested additional details have not been provided and we have been unable to reproduce it.
BTW this is a canned response and may have info or details that do not directly apply to this particular issue. While we'd like to spend the time to uniquely address every incoming issue, we get a lot traffic on the EF projects and that is not practical. To ensure we maximize the time we have to work on fixing bugs, implementing new features, etc. we use canned responses for common triage decisions.


Wrong label in internal/bms/lifepower4.go

Duplicate structure LFP4AlarmInfo element name "current_event_code" instead of "remaining_capacity_alarm".

Good catch!


Update iOS Sample to compile on Xcode 9.4

Been interested in using Lambda functions for a simple use case at https://www.shootmavis.com. My current expertise is iOS, so wanted to use the examples here to start creating my first few functions. But noticed the example for iOS was quite a bit out of date.
This PR contains various changes as described in detail, in the separate commits.
In summary, the changes are as follows:

Update the cocoapod file to allow pod install, including allowing frameworks and pinning the iOS version.
Update to Swift 4.1
Apply XCode recommended project file updates
Fix storyboard button actions to point to updated function names generated by Swift.

The project now compiles, and will work if the steps in the README steps are followed.

Latest commit also updates pods, deals with warnings and errors


Change layer of window via command failed

I try to toggle window from bsp to float centered via hotkey, and use next command for it
yabai -m window --toggle float;  \
yabai -m window --grid 11:16:3:2:10:7; \
yabai -m window --layer above;

First two command works great, but last with layer failed with next message:
could not change layer of window with id '45023' due to an error with the scripting-addition.

I want to make window above than others (make in main focus).  Video show how it's work now
https://github.com/koekeishiya/yabai/assets/10831688/5f8d55a1-add9-4c53-89bb-8a1f274ea262
I use this command incorrect, or can you recommend how I can make it?
Maybe good decision to always make focused windows above than others?

I found command yabai -m window --focus and it's made what I want. But still leave question why command with layer fails?

Make sure you follow the wiki on how to disable SIP and configure the scripting addition, if you want to change window layers.
https://github.com/koekeishiya/yabai/wiki/Disabling-System-Integrity-Protection
https://github.com/koekeishiya/yabai/wiki/Installing-yabai-(latest-release)#configure-scripting-addition

The Sticky windows work, but for some reason the layers command doesnt get recognized
❯ yabai -m window --layer below
unknown command '--layer' for domain 'window'
unknown command 'below' for domain 'window'
❯ yabai -m rule --add app="^Finder$" layer=below
unknown key 'layer'


About section still mentions v5

The about section still says "Quickly migrate your hooks from husky v4 to v5". I think this is also what gets set as meta description, so that is what shows up when sharing the url as well.

Thanks :) it's fixed :+1:


Version check on start and potential self-upgrade?

Since github provides a nice and clean way to get the latest version from api, maybe add a version check on update? Simply printing something along the lines of an orang text line under the puddler ascii reading something like.
Version (Running Version), latest release (Latest upstream release)
Bonus points for eventually adding U as a command that shows up in the list to update from github directly, but I am not sure that's something you can easily do, just letting users know there's a new version and making the U prompt take them to the latest release is probably good enough.

This crate seems like it would actually do exactly what you might want for updating in place, i could be wrong though, but worth a look.
https://docs.rs/self_update/latest/self_update/

Thank you for the resources you provided.
I can tell, how useful this feature could be, but there are a few things about would come from it that I really hate.

Hardcoding

I really hate to burn dependencies (f.e. repository url) like this into the code. It makes forking it a lot harder, since all of these things would have to be changed to make the code fully functional in a different repository.

Privacy / Complexity

Then there is the thing about privacy. Sure, auto-checking would be an optional setting, but firstly, we already got a few of those, and secondly, it would add another, not so insignificant, layer to the program. Also bugging github every time someone starts the script doesn't seem very effective for me. (so save the time of the latest check in the configuration?!)

I will keep this issue open, until I've made a decision.

Handling the concern of hard coding repo urls and privacy under the same idea here. How about enabling auto updates by defining the repo url in the config? combined with using timestamps as a config value for when the last check was and only checking it if its been a while since it last checked? Not that im concerned that hitting github on startup would cause enough concern to actually worry about since plenty of people use scripts for updating that hit those api's to begin with.


Provide informative error messages on web UI failures

As far as I can see, the error states returned by the cayley/http package do not propagate back to the web UI. It would be helpful if they did.

This issue was moved.


loading checkpoints that are in .ckpt format

Similar to Issue #62, where WangHexie's model directory containing model.ckpt.data-00000-of-00001, model.ckpt.index, model.ckpt.meta was not working (not Ktrain-specific issue), I was wondering if there was a way to use Ktrain for models with Tensorflow checkpoints. These checkpoints are in the format described on the official website. I'm not entirely sure how to convert from Tensorflow to Keras - similar to this issue on the main Keras GitHub.

This should be supported by default in TF2.  For instance, the following works:
from ktrain import text as txt
model = txt.text_classifier('nbsvm', train_data=trn, preproc=preproc)
model.save_weights('/tmp/saved_weights/checkpoint')
model.load_weights('/tmp//saved_weights/checkpoint')

$ ls /tmp/saved_weights
checkpoint  checkpoint.data-00000-of-00001  checkpoint.index

This should be supported by default in TF2.  For instance, the following works:
from ktrain import text as txt
model = txt.text_classifier('nbsvm', train_data=trn, preproc=preproc)
model.save_weights('/tmp/saved_weights/checkpoint')
model.load_weights('/tmp//saved_weights/checkpoint')

$ ls /tmp/saved_weights
checkpoint  checkpoint.data-00000-of-00001  checkpoint.index


plugin: Add plugin store plugin

Plugin store is a plugin that helps
install and manage headlamp plugins
from artifacthub
https://github.com/headlamp-k8s/plugins/assets/22731013/695834d7-41c1-44a8-bd1a-3d118a1bc276

Hrmm. There's a problem with the build command, which is causing the failure in CI. Which I think is unrelated to this PR.

I guess it makes sense to add artifact hub badges as well? (as app-catalog did recently)

I was thinking the "Store" part doesn't fit. Because it's not a store really. What do you think?  /cc @joaquimrocha @blixtra
Just "Plugins" or "Plugin Catalog" ?

I'm getting this error:

Maybe it's this error:
http://localhost:4466/externalproxy, no allowed proxy url match, request denied

Maybe it's this error: http://localhost:4466/externalproxy, no allowed proxy url match, request denied

Add this to your env before running backend
export HEADLAMP_CONFIG_PROXY_URLS=https://artifacthub.io/*

We have a Settings -> Plugins (that looks like the screenshot here). I'm wondering if this should be inside there or not?

Maybe it's this error: http://localhost:4466/externalproxy, no allowed proxy url match, request denied

Add this to your env before running backend export HEADLAMP_CONFIG_PROXY_URLS=https://artifacthub.io/*

I guess this needs to be a default when the app is run?
Or at least cd app && npm start should be modified so it's a default.

I guess this needs to be a default when the app is run?

Or at least cd app && npm start should be modified so it's a default.

I usually run the backend separately when developing and add the env there, I am not sure if this should be added in the app/start.js. It gets added in the app/electron/main.ts automatically from the app-build-manifest.json

I guess this needs to be a default when the app is run?

Or at least cd app && npm start should be modified so it's a default.

I usually run the backend separately when developing and add the env there, I am not sure if this should be added in the app/start.js. It gets added in the app/electron/main.ts automatically from the app-build-manifest.json

Hrmm.  I'm still getting an error with the proxy added.
ps. I made a PR for adding artifact hub proxy URL by default in the dev mode for 'make run-backend' https://github.com/headlamp-k8s/headlamp/pull/2130

I have now tried this:

Stop the backend, to simulate issues.
Try installing a plugin: it still shows the stop button briefly and proceeds to indicate it's installed 🤔

I have now tried this:

Stop the backend, to simulate issues.
Try installing a plugin: it still shows the stop button briefly and proceeds to indicate it's installed 🤔

The plugin installation isn't dependent on the backend. the backend is used only for the externalproxy to get plugin info from ArtifactHub.

The error I'm getting is coming from PluginManager.list inside InstallList.tsx So I guess it has something to do with plugins I have installed (or not installed).

Error loading Installed plugins: TypeError: Cannot read properties of undefined (reading 'list')

I have now tried this:

Stop the backend, to simulate issues.
Try installing a plugin: it still shows the stop button briefly and proceeds to indicate it's installed 🤔

The plugin installation isn't dependent on the backend. the backend is used only for the externalproxy to get plugin info from ArtifactHub.

Ah, that's right.

BTW, the Learn More doesn't seem to be opening the URL for me (running from WSL anyway).

I was thinking the "Store" part doesn't fit. Because it's not a store really. What do you think?  /cc @joaquimrocha @blixtra
Just "Plugins" or "Plugin Catalog" ?

I agree with Plugin Catalog. But we keep the store icon, which looks nice.

BTW, the Learn More doesn't seem to be opening the URL for me (running from WSL anyway).
This is working as expected for me in WSL.

I have added an user confirmation dialog to install handler in the electron app and to test it run headlamp app from the plugin_catalog_extras branch.

I have added an user confirmation dialog to install handler in the electron app and to test it run headlamp app from the plugin_catalog_extras branch.

This was done here: https://github.com/headlamp-k8s/headlamp/pull/2148

btw. The DCO check is failing, and probably those fixup commits should be applied.


About hand eval using freihand code

Hi, I followed the instruction in your EXP.md, and run Eval on Freihand using the same command as you give. But I got a json file which only contains 1980 predicted result, whereas the Freihand evaluation code says there shall be 3960 result, which confuses me.
Is the tsv format test data doesn't match the eval set used by freihand? Or it is necessary to rewrite pred.py in Freihand evaluation code to get things right?
Looking forward to your reply.

UPDATE:
I tried rewrite pred.py mentioned above, by using the ./models/fastmetro_checkpoint/FastMETRO-L-H64_freihand_state_dict.bin implemented in demo code block. However I just got bad result:
Evaluation 3D KP results:
auc=0.000, mean_kp3d_avg=147.53 cm
Evaluation 3D KP ALIGNED results:
auc=0.231, mean_kp3d_avg=4.41 cm

Evaluation 3D MESH results:
auc=0.000, mean_kp3d_avg=72.15 cm
Evaluation 3D MESH ALIGNED results:
auc=0.169, mean_kp3d_avg=5.01 cm

F-scores
<EMAIL_ADDRESS>= 0.000<EMAIL_ADDRESS>= 0.042
<EMAIL_ADDRESS>= 0.000<EMAIL_ADDRESS>= 0.230

I use the state_dict get from the readme, I don't train it again and just use it to eval, no test-time augmentation is added and the eval_batchsize is 1.
Am I doing wrong to the hyper-parameter? Or finetuning on freihand is needed? Please notify me if I'm wrong.
Looking forward to your reply.

Hello,
Our code generates pred.json file which contains 3960 prediction results as stated in FreiHAND.
We are not sure why you got a json file which only contained 1980 prediction results. freihand/test.yaml file has 3960 evaluation samples, and our code generates a prediction result for each sample.
Please check:

whether you correctly downloaded the FreiHAND dataset
len(val_loader) * batch_size == 3960

Quantitative Evaluation Results

obtained by using the released model checkpoint (FastMETRO-L-H64_freihand_state_dict.bin)

Evaluation 3D KP results:
auc=0.000, mean_kp3d_avg=71.50 cm
Evaluation 3D KP ALIGNED results:
auc=0.871, mean_kp3d_avg=0.65 cm

Evaluation 3D MESH results:
auc=0.000, mean_kp3d_avg=71.50 cm
Evaluation 3D MESH ALIGNED results:
auc=0.858, mean_kp3d_avg=0.71 cm

F-scores
<EMAIL_ADDRESS>= 0.000<EMAIL_ADDRESS>= 0.687
<EMAIL_ADDRESS>= 0.000<EMAIL_ADDRESS>= 0.983

Note that only the aligned scores are considered because FastMETRO predicts relative coordinates, as in METRO and Mesh Graphormer.
There was no test-time augmentation.

Qualitative Evaluation Results

obtained by using the released model checkpoint (FastMETRO-L-H64_freihand_state_dict.bin)

@imabackstabber
Please note that

Our code generates pred.json file which contains 3960 prediction results
There is no need to rewrite pred.py in FreiHAND Evaluation Code
The quantitative and qualitative evaluation results obtained by the pre-trained model (FastMETRO-L-H64_freihand_state_dict.bin) must be same as the above results

There was no test-time augmentation
There was no further fine-tuning

Please reopen this issue if you need more help regarding this.

UPDATE:
The 1980 result is because I use command below:
python -m torch.distributed.launch --nproc_per_node=2 \
       src/tools/run_fastmetro_handmesh.py

Note that 3960 = 1980 * 2. This will lead to a single pred.json file which only contain 1980 elements because there are actually 2 processes launched by torch.distributed.launch, and one process(i.e. the slower one) will rewrite a pred.json and override one pred.json dumped by the former one.
To correct, just use command:
python src/tools/run_fastmetro_handmesh.py

I suggest such modification could be updated in exp.md
p.s.: the former result I gained is probably due to wrong checkpoint. I re-download it, run the script, and get the same result.
Thanks for your help!

Glad to hear that you got the same result as we obtained :)
Thanks for the detailed explanation regarding this issue! We updated Experiments.md to prevent such issue; a single process would generate pred.json file.