upload_blob() method is on working with the reader stream from os.pipe()

Package Name: azure-storage-blob
Package Version: 12.9.0
Operating System: Ubuntu 18.04.4 LTS
Python Version: 3.9.10

Describe the bug
The reader stream from the os.pipe() call passed to the ContainerClient's instance upload_blob method produces an unhandled "OSError" exception.

            # If so, calculate the size using the file descriptor.
            try:
                fileno = data.fileno()
            except (AttributeError, UnsupportedOperation):
                pass
            else:
                try:
                    mode = fstat(fileno).st_mode
                    if stat.S_ISREG(mode) or stat.S_ISLNK(mode):
                        #st_size only meaningful if regular file or symlink, other types
                        # e.g. sockets may return misleading sizes like 0
                        return fstat(fileno).st_size
                except OSError:
                    # Not a valid fileno, may be possible requests returned
                    # a socket number?
                    pass

            # If the stream is seekable and tell() is implemented, calculate the stream size.
            try:
>               current_position = data.tell()
E               OSError: [Errno 29] Illegal seek
../venv/lib/python3.9/site-packages/azure/storage/blob/_shared/request_handlers.py:86: OSError

In order to avoid this issue we've applied a hacky workaround:
import os
from azure.storage.blob import ContainerClient

reader_fd, writer_fd = os.pipe()

with os.fdopen(writer_fd, 'wb') as writer:
    writer.write(b"test")

def _raise_attribute_error():
    raise AttributeError()

with os.fdopen(reader_fd, mode='rb') as reader:
    # a hack because of the azure lib implementation
    # it's only capable to catch AttributeError and not OSError
    reader.tell = _raise_attribute_error

    client = ContainerClient(
        'some_conn_str',
        'some-container'
    )
    client.upload_blob(data=reader, name='some-name')

Expected behavior
Expected the upload_blob method to work with the reader stream from os.pipe()
Additional context
We're actually running 2 threads: one for fetching data from Postgres and writing to the pipe, another one to upload data from the pipe.
With the applied workaround it works fine.

Thanks for reporting this @bbl Bogdan, let us look into this and get back to you.

HI @bbl Bogdan, thanks for investigating this and providing the steps to reproduce. I was able to reproduce this error with your example (on Linux but not Windows). I think a valid fix would be just to catch OSError in get_length() to support this scenario.
I will create a PR for this soon and try and get this into our March release of 12.10.0.


[FLINK-10436] Example config uses deprecated key jobmanager.rpc.address

What is the purpose of the change
flink-conf.yaml uses deprecated key jobmanager.rpc.address, which produces an unexpected warning.
Brief change log
Use the suggested replacement.
Verifying this change
Manually, local build and test.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive):(no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper:(no)
The S3 file system connector:(no)

Documentation

Does this pull request introduce a new feature?(no)
If yes, how is the feature documented? (no)

cc @zentol @StefanRRichter

I think this needs input from @tillrohrmann.
AFAIK this is intentionally still in there as long as the legacy mode is still supported

Let's delay this until the legacy mode was removed.

Thanks for your review! @StefanRRichter  @StephanEwen @zentol
I agree to delay this. Could I keep this PR open and ping here on the removal done?

I would close this PR until the blockers are resolved in order to reduce the noise in the PR list. Since you still have the code available it should not have any effect on you and you can link this PR from your future PR.
Furthermore, I would update this ticket to indicate that it is blocked by FLINK-10392 (or as an alternative, make it a subtask of FLINK-10392 and say that it should be addressed last)

@uce ok~


Parse CycloneDX Legal information

Description of the PR
Update CDX parser to capture legal information that is contained in the SBOM. Unit tests have been updated to test the new functionality.
Related to the open issue: https://github.com/guacsec/guac/issues/1014, and completes the update to the CycloneDX Parser.
Updates were made based on the design doc: https://docs.google.com/document/d/1NmLlU5wuP2X9CK7QCWZkkOciNn1QFLKQCFCW9CEI8HQ/edit
PR Checklist

[x] All commits have a Developer Certificate of Origin (DCO) -- they are generated using -s flag to git commit.
[x] All new changes are covered by tests
[ ] If GraphQL schema is changed, make generate has been run
[ ] If GraphQL schema is changed, GraphQL client updates/additions have been made
[ ] If OpenAPI spec is changed, make generate has been run
[ ] If collectsub protobuf has been changed, make proto has been run
[x] All CI checks are passing (tests and formatting)
[x] All dependent PRs have already been merged

Did you get small-legal-cyclonedx.json from somewhere? It seems to be not correct according to the spec.
name

"If SPDX does not define the license used, this field may be used to provide the license name"

Did you get small-legal-cyclonedx.json from somewhere? It seems to be not correct according to the spec.
name

"If SPDX does not define the license used, this field may be used to provide the license name"

oh yeah its not real. I made it up to capture multiple usecases

@jeffmendoza PTAL, I made the changes based on the comments and updated the unit tests.


Language Server - interruptions

The language server will run the parsing of the various scripts in separate threads.
This allows us to save resources by allowing those threads to be interrupted when we no longer need them (if, for example, a file is modified during parsing).
Is this a behaviour that needs to be tested? Not sure how I'd even test that.

Fred the answer is always yes. Definition of insanity and all that...

but how
I can't just start the parsing in a new thread and "hope" I interrupt the parsing before it's over.

By mocking importFile to throw the exception

By mocking importFile to throw the exception

which? one is not called during parsing, and the other is private.

(can you mock a method even if it's private? I forgot)


Can't upload some images when ngf-fix-orientation="true"

When forcing the ngf-fix-orientation attribute to be true, as it isn't triggering the necessary transforms otherwise, I keep getting the a 400 - Bad Request error, when not uploading an image that has been taken with an iOS device.
This is the plugin's response:
"Upload a valid image. The file you uploaded was either not an image or a corrupted image."
Any ideas on what might be causing this?
Thanks a lot! :+1:
Relates to #927 - "iOS - Taking a picture to upload is always coming out landscape"

@miguelcaires What is the technology on your server that returns this error?

@danialfarid It's running a Django backend.

Would it work if you just have ngf-resize="{width:100, height:100}" without ngf-fix-orientation?

Could it be this? http://stackoverflow.com/questions/7606614/django-imagefield-upload-a-valid-image-the-file-you-uploaded-was-either-not-an

Hi @danialfarid.
Sorry for the delay.
This issue was addressed in the backend.
Thanks for getting back, and congrats once again on an amazing plugin! :+1:


Clarifying AI Assistant Capabilities: Limited Code Access

I would like to request an enhancement to the AI assistant's code access functionality. Currently, the assistant indicates that it does not have full access to the code, which can cause confusion and limit the assistance provided.

Agree which is frustating

Duplicate of https://github.com/microsoft/vscode-copilot-release/issues/1083


The status.online field will be removed automatically when user was offline for a period of time

Hi, I found something interesting when using this package.
It works well when monitoring the user's status. But I found that when the user was offline for a period of time, the status.online will be remove automatically.
# when the user is online
meteor:PRIMARY> db.users.findOne({username: 'xxx'})
...
"status" : {
		"lastLogin" : {
			"date" : ISODate("2014-08-06T01:58:35.217Z"),
			"ipAddr" : "<IP_ADDRESS>",
			"userAgent" : "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36 OPR/23.0.1522.60"
		},
		"online" : true
},
...

# once the user is offline
meteor:PRIMARY> db.users.findOne({username: 'xxx'})
...
"status" : {
		"lastLogin" : {
			"date" : ISODate("2014-08-06T01:58:35.217Z"),
			"ipAddr" : "<IP_ADDRESS>",
			"userAgent" : "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36 OPR/23.0.1522.60"
		},
		"online" : false
},
...

# the user is offline for a period of time
meteor:PRIMARY> db.users.findOne({username: 'xxx'})
...
"status" : {
		"lastLogin" : {
			"date" : ISODate("2014-08-06T01:58:35.217Z"),
			"ipAddr" : "<IP_ADDRESS>",
			"userAgent" : "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36 OPR/23.0.1522.60"
		}
       # online field will be remove
},
...

Is there any way to keep the status.online field  ?

Try using v0.6.4 that I just released and let me know if that helps for you.


nfl_elo incorrect playoff game dates

I believe that the nfl_elo dataset may have the 2015 season wild card and division round playoff games on the wrong days:

According to wikipedia (and other sources that I followed up on), the wild-card games were on 1/9 and 1/10, with the divisional round on 1/16 and 1/17.
https://en.wikipedia.org/wiki/2015–16_NFL_playoffs#NFL_playoff_schedule

Thanks @AustinNess, this is fixed.


changes in nav and inserted image in dex.html | Service Page

Description
On Services page the navbar's about was not having proper link in href , corrected and
Added an image which was not displayed as it was not seen anywhere in the folder.

Closes #578

Type of PR

[ ] Bug fix
[x] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)
[Attach any relevant screenshots or videos demonstrating the changes]

Checklist

[x] I have gone through the contributing guide
[x] I have updated my branch and synced it with project main branch before making this PR
[x] I have performed a self-review of my code
[x] I have tested the changes thoroughly before submitting this pull request.
[x] I have provided relevant issue numbers, screenshots, and videos after making the changes.
[x] I have commented my code, particularly in hard-to-understand areas.

Thankyou for your contribution


Research and development

https://docs.github.com/en/rest/emojis

https://lordicon.com/

https://www.useblackbox.io/search?q=

https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types

https://css-tricks.com/parsing-markdown-into-an-automated-table-of-contents/

https://undraw.co/

https://fly.io/
https://pocketbase.io/
https://supabase.com/

https://lovelacecoding.hashnode.dev/7-resources-every-web-developer-should-know

https://codepen.io/shshaw/pen/mwKvZr/?editors=1000

https://webutility.io/markdown-beautifier

https://blog.shahednasser.com/how-to-create-a-npx-tool/
https://dev.to/ihaback/create-your-own-create-react-app-template-46ll
https://bonsaiilabs.com/create-npx-starter-command/

Something similar to postman?

Convert this to human language
https://dummyjson.com/products/1
Also how to iterate or access the data

https://devresourc.es/

https://metatags.io/


psc-ide rebuild isn't being called

Eventhough the fast-rebuild checkbox in the options is checked.
I'm running the current master.
Another thing I noticed is that you are running a load command with dependencies after each safe. That command will no longer do anything, since support for the dependencies option got removed. There is only the full load or module specific load.

Fast rebuild - this was working for me for sure. Has it actually connected to psc-ide-server?
The load command - largely this is an effect of trying to support multiple versions, since the older versions of psc-ide-server needed to have this load for a particular file. Would be good to sit down and work out a strategy here.

For the load command, allright that's totally what I intended with not making it an error. If that's a backwards compatibilty feature I never said anything.


Vertex Prediction PyTorch Experimental: Add a sample for pre-built PyTorch deployments.

samples: Add a new sample for pre-built Pytorch deployments. It's borrowed from the examples in community-content/pytorch_text_classification_using_vertex_sdk_and_gcloud.

If you are opening a PR for Community Content under the community-content folder:

[x] Make sure your main Content Directory Name is descriptive, informative, and includes some of the key products and attributes of your content, so that it is differentiable from other content
[x] The main content directory has been added to the CODEOWNERS file under the Community Content section, pointing to the author or the author's team.
[x] Passes all the required formatting and linting checks. You can locally test with these instructions.

Hi team,
Could we hold off merging this PR before we are ready? Thanks!
/cc @kthytang @samthrasher


Error types breadcrumbs

Describe the bug
when i enable typescript check in nuxt.config
typescript: {
    typeCheck: true,
 }

Reproduction

System / Nuxt Info
- nuxt 3.6.5
- OSX Ventura 13.5
- Brave (chrome)

Duplicate of: https://github.com/harlan-zw/nuxt-seo-kit/issues/40

Most of these should be fixed in the latest, can track in #40 anything missed


Why does Prometheus Operator require access to secrets?

This doc tries to explain Prometheus Operator RBAC in depth, but fails to explain why Prometheus Operator needs all access to all secrets in the cluster.
https://github.com/coreos/prometheus-operator/blob/master/Documentation/rbac.md

This is currently required, so that the Prometheus Operator can read and template secrets to build the Prometheus config, and the work being done in https://github.com/coreos/prometheus-operator/pull/2716 requires this as well as features like "additionalScrapeConfigs". There are some thoughts to push this responsibility down to the Prometheus servers in the future, but it's somewhat undecided.

Thanks


Alliance

Hello, my name is Alfonso Hernandez. I work for the Astian Foundation, a non-profit organization that offers free software and technologies.
We are the organization responsible for development and maintenance of Midori Browser, a popular browser, light, fast, secure and open source.
We are developing a service called K-LamarVPN using WireGuard, which we hope to link with Midori. We would like to establish an alliance with the project to implement it.
What do you think?

Hello Alfonso, thanks for your message.
Sounds interesting. Maybe we could schedule a call next week to discuss what you have in mind.
Would you mind sending me an email at<EMAIL_ADDRESS>with dates/times you have available?
Cheers!

ready


Chapter 6 - puzzling example for filtering

assert olderThan35.head(4) equals all.head(4)
Although understandable for programmers, I thing this is a puzzling simplification for beginners, as it only works thanks to the coincidence that the last element is filtered out. I'd rather utilize some combination of "contains" or "containsAll".

Fixed.


Make ContentLibraryRef field be pointer

This ContentLibraryRef field should be pointer, as it's optional and one of the fields in TypedLocalObjectReference is required  for third-party types.

@cyclexuxu Thanks for creating this PR. I have several changes in mind to add/update on this type. It would be nice if you can include them as well since you are touching this file already. I will send them out shortly in the following PR comment. Thanks!

@cyclexuxu Thanks for creating this PR. I have several changes in mind to add/update on this type. It would be nice if you can include them as well since you are touching this file already. I will send them out shortly in the following PR comment. Thanks!

@cyclexuxu Would you mind also updating the printcolumn of ClusterVirtualMachineImage type with the following changes, please?

Rename CONTENTSOURCENAME to ITEM-REF-NAME (as we won't have conentsource type in the new image registry service)
Rename CONTENTLIBRARYNAME to CONTENT-LIBRARY-NAME (with dashes for easier reading)
Add IMAGE-NAME with JSONPath=".status.imageName"

@cyclexuxu Would you mind also updating the printcolumn of ClusterVirtualMachineImage type with the following changes, please?

Rename CONTENTSOURCENAME to ITEM-REF-NAME (as we won't have conentsource type in the new image registry service)
Rename CONTENTLIBRARYNAME to CONTENT-LIBRARY-NAME (with dashes for easier reading)
Add IMAGE-NAME with JSONPath=".status.imageName" (this will display the vSphere image name while we encode the field name)

Hi @dilyar85 , thanks for the suggestion. I have updated the CONTENTSOURCENAME and CONTENTLIBRARYNAME as you suggested. Added the ImageName, but without the dash, as I think there already has ImageSupported, just follow the same pattern.

Hi @dilyar85 , thanks for the suggestion. I have updated the CONTENTSOURCENAME and CONTENTLIBRARYNAME as you suggested. Added the ImageName, but without the dash, as I think there already has ImageSupported, just follow the same pattern.

Yeah we should probably add a dash there too so that the column will be displayed as IMAGE-SUPPORTED (rather than the current IMAGESUPPORTED since it's not a single word). Could you please update all the multi-words columns ("OsType", "ImageSupported" and "ImageName") with a dash in between and move Image-Name to the top? Thanks.

Found that upstream uses space to separate words. e.g. NOMINATED NODE and READINESS GATES for pod.

Changed to use space to separate non single word names as we can find same cases in upstream and other repos.

IDK if there is necessarily k8s convention around the column names. As you noted, pods use spaces, eg NOMINATED NODE, while nodes use dashes, eg INTERNAL-IP,OS-IMAGE.
My preference would be for dashes since that does make rudimentary CLI parsing easier (but there are safer & more stable ways to obtain those fields).
That said, it seems we should try at least keep ourselves (mostly) internally consistent: I think we only have one spot that uses spaces (Content Library UUID), one dash usage PRIMARY-IP (likely inspired by node), and then one word for everything else.
I think it would be best if we reduce the changes in this MR to just the purpose of the title, and have a separate discussion/MR to rename what needs to be renamed here and make other printer columns consistent with whatever we choose.

IDK if there is necessarily k8s convention around the column names. As you noted, pods use spaces, eg NOMINATED NODE, while nodes use dashes, eg INTERNAL-IP,OS-IMAGE.
My preference would be for dashes since that does make rudimentary CLI parsing easier (but there are safer & more stable ways to obtain those fields).
That said, it seems we should try at least keep ourselves (mostly) internally consistent: I think we only have one spot that uses spaces (Content Library UUID), one dash usage PRIMARY-IP (likely inspired by node), and then one word for everything else.
I think it would be best if we reduce the changes in this MR to just the purpose of the title, and have a separate discussion/MR to rename what needs to be renamed here and make other printer columns consistent with whatever we choose.

Thanks. I will move the renaming part to another MR. CC @dilyar85


RTL support

Hi,
Thank you so much for this awesome framework. thank again for your efforts.
Does Tailwind support RTL ( for Arabic, Hebrew ...etc) ?

This is a duplicate of #64.


Missing comma

@paterasMSFT : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Looks good; thanks.


Drishti does not compile with linux gcc anymore.

When compiling for toolchain "gcc" on a linux box, this error is raised:
src/lib/drishti/core/Logger.cpp:52:38: error: ‘stdout_sink_mt’ is not a member of ‘spdlog::sinks’
I suspect that the GCC libraries have evolved, or perhaps an include file is missing.

I can't compile with GCC too but the error is different.
For the record GCC is not a part of CI testing (note: GCC is a default compiler, so here we use GCC too: https://github.com/elucideye/drishti#generate-and-build)
CI is tested with Clang 5.0:

https://travis-ci.org/elucideye/drishti/jobs/486975559

I have tried Clang 5.0.2, 7.0.1 and 8.0.0 without success. The error is in 'ogles_gpgpu' package.
E.g. for Clang 5.0.2:
In file included from /.../Build/ogles_gpgpu/Source/ogles_gpgpu/common/proc/multipass/gauss_opt_pass.cpp:13:
/.../clang+llvm-5.0.2-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/cmath:313:9:
  error: no member named 'signbit' in the global namespace; did you mean '__signbit'?
using ::signbit;
      ~~^


Manage automatic BLE reconnection on phone reboot

I need to reconnect an iOT BLE device when the phone restart.
Does this library manage this automatic reconnection ?

No, it does not. On iOS there's the restoreState callback which we're exposing, which might help you with restoring the connection. Android does not provide any API for such things AFAIK.


feat(k8s): collect container id

:tada: This PR is included in version 1.45.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Print correct messages in setup gpg startup

Closes #33

Merge. Tag as v0.5.1 in master branch. Test using new module mosrs-setup/0.5.1.


[BUG] Integrate CDK fix for Route53 VPC region

Describe the bug
I added a custom resource to add regional VPCs to the Route53 hosted zone because of the following CDK bug.
https://github.com/aws/aws-cdk/issues/20496
Need to get the latest version of CDK that has the fix and verify the fix.
The latest version today, 9/7/22, is 2.40.0 and we're currently using version 2.28.1.

Version 2.40.0 is from 7 days ago so it will not have the fix which was merged today.

Log showing 2.41.0 which doesn't have this fix yet.
* 46cf825739 "Cory Hall"<EMAIL_ADDRESS>2 hours ago (HEAD -> main, origin/main)
| fix(elbv2): connections not created for chained listener actions (#21939)
*   ab7668197b "mergify[bot]"<EMAIL_ADDRESS>14 hours ago
|\  chore(merge-back): 2.41.0 (#21958)
| *   dc8abdfdf4 "mergify[bot]"<EMAIL_ADDRESS>14 hours ago
| |\  Merge branch 'main' into merge-back/2.41.0
| |/
|/|
* | aedc8883bf "Joshua Weber"<EMAIL_ADDRESS>16 hours ago
| | fix(route53): vpc region in template overridden by stack region (#20530)
| *   6ad48a3193 "mergify[bot]"<EMAIL_ADDRESS>16 hours ago (tag: v2.41.0, origin/v2-release)
| |\  chore(release): 2.41.0 (#21955)
| | * 28e080aa45 "AWS CDK Team"<EMAIL_ADDRESS>17 hours ago
| |/  chore(release): 2.41.0
|/|
* | b0ba52e5a0 "Kendra Neil"<EMAIL_ADDRESS>18 hours ago

The bug fix was released in 2.42.0.
I tested with 2.42.1 and am still seeing the same problem. The Route53 code is not getting the region from vpc.env.region, but the region is not set correctly by the context provider.
Filed
https://github.com/aws/aws-cdk/issues/22178

Deprecating the legacy scheduler in favor of ParallelCluster.


Broken link: "Quickstart" link in "Learn more" item, "Getting Starded" paragraph.

Broken link: "Quickstart" link in "Learn more" item, "Getting Starded" paragraph.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2a44958f-e0d6-167a-8749-0c244cf79f09
Version Independent ID: 3aa59386-dadd-6658-f80d-bee7c7c52263
Content: Entity Framework Overview
Content Source: docs/framework/data/adonet/ef/overview.md
Product: dotnet-framework
GitHub Login: @douglaslMS
Microsoft Alias: douglasl

Thanks @MarcioAB. I've opened PR #9019 to fix this link in two topics plus some others I've found with the same problem.


extension: use bool_flag to control extension enablement

Signed-off-by: Lizan Zhou<EMAIL_ADDRESS>

This is initially intended to solve #12574, while I think it has bigger impact by introducing command line flag to control every extension enabled/disabled, so I just post this as a draft. While I think this is great to have but we should probably also discuss whether we want to do flavor that came up before. That might be also doable through flags as described https://docs.bazel.build/versions/master/skylark/config.html but I haven't digged into. @mattklein123 @htuch  cc @envoyproxy/maintainers
as a side effect this resolves #14027 but incompletely (i.e. notests).

LMK when this is ready for merge review.
/wait


How can i remove this red underline?

When I write wrong command in visual studio code, red underline occur in wrong command. I hope remove this function. What should I do?

You can disable Linting by adding "python.linting.enabled": false in your settings, or by using the Select Linter command from the the Command palette (CTRL + SHIFT + P), and then selecting Disable linting.
Please note that this repository is for feature requests and bugs, and you can ask all these questions in stack overflow.

Thank you very much karrtikr♡


fix: Pass along endpoint url when copying an AwsSession

Change _copy_aws_session to also track the endpoint URL of the original session to its clone. Need some help writing a nice test for this as my knowledge of botocore is rudimentary.
Merge Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.
General

[x] I have read the CONTRIBUTING doc
[x] I used the commit message format described in CONTRIBUTING
[x] I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

[ ] I have added tests that prove my fix is effective or that my feature works (if appropriate)
[ ] I have checked that my tests are not configured for a specific region or account (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Fixed the linting issues and copied the config to the client. I have a bunch of other tasks I should be working on and found the new triggered failures really opaque, so if someone wants to add commits to this, please do.


Hitting 'enter' in the advanced binding editor puts the focus back at the beginning of textbox

This is in Chrome -- can get tedious to manually edit the bindings file.
Edit: I just noticed the same thing happens with backspace.

It also happens if you select some text and then type a character. I see this in Prod, but I'm pretty sure this used to work. Right now, you're almost guaranteed to hit this when doing any editing in the advanced editor, so if it was always like that we would have noticed.


Progress Bar for Download

Is there a plan or work around for to allow us to see the Download progress? it would improve the user experience quiet a bit to know how its going :+1:

Closing since this is not likely to happen from the infrastructure side anytime soon.


Decision Tree Error

There's an error here.
decision-tree.js:20
root = childNode.child;
^
TypeError: Cannot read property 'child' of undefined
at Object.ID3.predict (/nodejs-decision-tree-id3-master/lib/decision-tree.js:20:23)
at Object. (/nodejs-decision-tree-id3-master/example.js:22:34)
at Module._compile (module.js:434:26)
at Object.Module._extensions..js (module.js:452:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Function.Module.runMain (module.js:475:10)
at startup (node.js:117:18)
at node.js:951:3

Hi Kenneth, can you provide steps to reproduce the issue?

Hi Ankit,
I followed your example after installing. I followed the usage to input the test data and setup for prediction and evaluation.
There I got this error during runtime.
Thanks

Hi Kenneth, I am not able to reproduce the issue following the example provided in the README. I was hoping you could provide some more information to help debug this:

Which version of the decision-tree package are you using?
Can you provide the code you are executing to help reproduce the issue?

Thanks!

Hi Ankit,
I was using the latest one, I assumed is in your master branch.
The code was the example I followed in your README.md.

I have the same issue here.
Training data:
http://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data
Testing data:
http://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.test
My program:
var DecisionTree = require('decision-tree');
var fs = require('fs');

var features = ["age", "workclass", "fnlwgt", "education", "education-num", "marital-status", "occupation", "relationship", "race", "sex", "capital-gain", "capital-loss", "hours-per-week", "native-country"];
var class_name = "result";

var columnMapping = features.concat(class_name);

function readData(file)
{
  var rows = fs.readFileSync(file).toString().split('\n').filter(function (l) { return l.trim().length > 0; }).map(function (row) {
    var d = {};
    var columns = row.split(', ');
    for (var i = 0; i < columns.length; ++i) {
      d[columnMapping[i]] = columns[i];
    }
    return d;
  });
  return rows;
}

var training_data = readData('./adult.data.txt');
var test_data = readData('./adult.test.txt');

console.log('Building tree...');

var dt = new DecisionTree(training_data, class_name, features);

console.log('Evaluating...');

var accuracy = dt.evaluate(test_data);

console.log('accuracy:');
console.log(accuracy);

console.log('Exporting...');

var treeModel = dt.toJSON();
fs.writeFileSync('./model.json', treeModel);

I got the same error as well. Here's a simplified version of what I was attempting:
https://gist.github.com/Resseguie/d145b358200c83c05615
Note it's all numeric and I didn't enter a lot of training data. Don't know if that is related.
I should also point out that I tried the example from the README and it worked fine.

I updated my gist to have random training data as well. It occasionally works and spits out an answer, but usually dies with the error on line 20.

I've just commited a fix to this, I wasn't entirely sure it was this logic was the right thing to do in ID3, but I've just looked at a few other libraries implementing this and my fix is how they handle the situation as well, see example... https://github.com/yandongliu/learningjs/blob/master/learningjs.js#L66
It comes from a scenario where testing data hasn't been seen in the training set, so returns the most dominant value up until that point.

Thanks @scrivna for contributing a bug fix to this issue! I have merged your pull request.


[storage] DB::open_as_secondary()

Motivation
So we can run read only tools while the node is still running.
Have you read the Contributing Guidelines on pull requests?
Y
Test Plan
unit test.
Related PRs

@zekun000  走好不送
/land

:broken_heart: Test Failed - commit-workflow

/land


added a new method to fetch meta-data with 'STANDARD-XML' format and …

…return a hash from a MultiXml parse

@joeheald85  I don't see how we could stream parse the XML in this format, it works in the "COMPACT" format because there is no nesting of nodes in the xml.


feat: ContainerBuilder should reject illegal container names

Description
ContainerBuilder checks names to be RFC 1035 compliant

https://github.com/stackabletech/operator-rs/issues/292
Review Checklist

[ ] Code contains useful comments
[ ] (Integration-)Test cases added (or not applicable)
[ ] Documentation added (or not applicable)
[ ] Changelog updated (or not applicable)
[ ] Cargo.toml only contains references to git tags (not specific commits or branches)

Once the review is done, comment bors r+ (or bors merge) to merge. Further information

Hm, shouldn't this be validated by the K8s apiserver anyway when actually trying to submit the pod?

@soenkeliebau what did we need this for originally?

Hm, shouldn't this be validated by the K8s apiserver anyway when actually trying to submit the pod?

My impression from the issue https://github.com/stackabletech/operator-rs/issues/292 was that validation by k8s-apiserver comes too late.

A quick test with kind (did not test any managed ones) shows that the statefulset, configmaps etc. get created:
Warning  FailedCreate      11s (x13 over 31s)  statefulset-controller  create Pod simple-zk-server-primary-0 in StatefulSet simple-zk-server-primary failed error: Pod "simple-zk-server-primary-0" is invalid: spec.contai
ners[0].name: Invalid value: "zookeeper_bad": a lowercase RFC 1123 label must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name',  or '123-abc', reg
ex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?')

So im in favor of merging this to have the operator error out.

Ok so it seems like it's validated when creating a Pod itself, but not immediately when creating something containing a PodTemplate (like Deployment, StatefulSet, and so on). In that case I'm :+1: on doing this.

Oops, sorry, Gitlab habit (GitLab can show the diff quite nicely when --force-with-lease is used`. Won't rebase in future. Thank you.


Integrate new test runner and test selector api's in stryker

Resolve #34
There is still work to be done, but i hope we can now start releasing a 0.1 release (beta)

There are still two issues, as far as I know:

No specnames are printed if a mutant survives
Tests are ran even when the mutant is not covered

We'll fix the coverage in a later version.


orthanc-anon ReceivedInstanceCallback must not allow exceptions to go uncaught or anonymisation will be compromised

Definition of Done / Acceptance Criteria

[ ] ReceivedInstanceCallback must only contain a big try/except statement, to maximise the chances of intercepting any exception that's thrown (would recommend sticking its current contents into another method)
[ ] A comment is added to explain why it's very important to keep all contents of this method wrapped in this try/except statement.
[ ] Probably do the same for the orthanc raw callback to aid in debugging, although I don't believe it's a security issue as we don't tend to reject stuff anyway.

Details and Comments
Repro steps

Add the following line to simulate a bug in the code that causes an exception to be raised in the ReceivedInstanceCallback of the orthanc anon plugin. (It would have to be an intermittent or data dependent bug to not be caught by system testing).

--- a/orthanc/orthanc-anon/plugin/pixl.py
+++ b/orthanc/orthanc-anon/plugin/pixl.py
@@ -281,6 +281,8 @@ def ReceivedInstanceCallback(receivedDicom: bytes, origin: str) -> Any:
     # Read the bytes as DICOM/
     dataset = dcmread(BytesIO(receivedDicom))

+    raise ValueError
+

Run the system test without performing teardown afterwards (the system test should fail)
Inspect the orthanc anon server in your browser at http://localhost:7003/
Navigate to "All studies"

Expected
Due to the raised exception, I would expect no data to have reached orthanc anon. And if any data has, it absolutely shouldn't contain any PII.
Actual
orthanc-anon contains the same data as orthanc-raw. Ie. two studies with full, original patient name, patient ID, accession number.
Luckily, it seems that FTP upload hasn't happened, but I haven't investigated why.
Discussion
It seems that orthanc requires an explicit return value (eg. return orthanc.ReceivedInstanceAction.DISCARD, None) to discard or modify a received instance. The default is to accept unmodified.
When anonymising, this "fail dangerous" paradigm is very much not ideal.
I can't be sure that in real life we wouldn't perform the FTP upload. It could be the way the system test is set up to test for success before proceeding that stops it from doing so. A human operator may not have realised this has happened and may manually run the upload command, thus putting patient data at risk.

I can't be sure that in real life we wouldn't perform the FTP upload.

We query the hashed_image_idenfifier in pipeline.image table, so if the

Possible duplicate of #122


Figure out what to do with Website list

Feedback by @kelimuttu in https://github.com/mozilla/firefox57-sprint/issues/34#issuecomment-325583354:

And apparently, Alexa only provide a limited number of 50 websites for the free account. In order to see the full list, we need to subscribe. So, I think we better come up with another options for the website reference. I can't think of anything right now tho'.
And we can probably put something in the website to encourage the host to ask the participants test their own favorite websites. Just so people is not only focus on the Alexa list.

@couci what do you think?

I like the idea of checking their own websites as well. As for the Alexa list, does it make sense for me to buy a subscription and copy past somewhere were Mozillians can check it out?

As far as I can see it's $25 per 100 URLs. That's gonna be really expensive if we want to cover most countries.

.. and I'm not sure if that is even allowed to be republished. We'll figure it out.

Priyanka will provide the India site
@couci to check with George about buying the data

Hi Micheal,
I have checked Alexa Indian List of websites and found that only few of websites are listed in trail version.
I would like to suggest you the below websites with more traffic :
*Linkedin.com (https://www.linkedin.com)
*Alibaba (http://india.alibaba.com/index.html)
*Daink Bhasker(https://www.bhaskar.com/site-take-over-ads)
*Make my trip(https://www.makemytrip.com/)
*99 acres (https://www.99acres.com/NRI-Real-Estate.htm)
*Bharath Matrimony(https://www.bharatmatrimony.com/)
*viu(https://www.viu.com/in-hindi/en)
Best,
Sandeep

Hi Sandeep
Some Reps have started to do their own list for India: https://public.etherpad-mozilla.org/p/firefoxquantumsprint-india

Thank you Michael i will do check it out

@couci ready to change the link/text whenever you are

In this Post I have found am S3 link to the top 1Mio list form Alexa. The Comment says its auto generated every day. The one I downloaded is from yesterday. The entries are matching with the ones from alexa's top 500

We need the region-specific data so we decided to purchase the list anyway. But yeah, we can definitely use the 1 million list as a global reference as well. Thanks for pointing out @lkappeler

Here is the list of the 200 first websites for the countries participating shorted alphabetically:  https://docs.google.com/spreadsheets/d/1KCT5Ytuu8tyC559TXVo-lIlv4mZVPGhn8qQe-qSTgm8/edit#gid=1569236874
@MichaelKohler here is the text: Check this list of the 200 most famous website in your local and choose the website you want to test. If you can't find your country you can also check the Alex list. Please don't start at the top to make sure we do not only test the first 10.

Will add that tonight.

Hi @MichaelKohler ,
I have a concern with the leader board points in below case: Please check :

For the first time when we tested a site ex: www.google.co.in and we didn't find any issues submit as" we didn't find any issue with the site by selecting .No" .This will be added as 1 point.
But in case if we find the issue when we tested for 2nd time for the same website (www.google.co.in) and reported, it was submitted and could see that there no increment in the points.

Can you please provide your thoughts for the same.
Best,
Sandeep

@sonusandeep we filter "double" entries to avoid testing the same site over and over again to get points. This is done by nickname+sitetested.
@couci thoughts?

This is caused by https://github.com/mozilla/firefox57-sprint/issues/88. Since we count domains unique the issues form the entry are counted. I think we should add an additional check if there are more issues than in the fist check and count the check with more issues

Thank you @MichaelKohler  for clarifying.
@lkappeler  +1

@sonusandeep this will be handled in https://github.com/mozilla/firefox57-sprint/issues/88

Sure Thank you @MichaelKohler


io_uring_wait_cqe_timeout() + IORING_ENTER_EXT_ARG doubling timeout duration

So far I have only observed this behavior with one sqe pending submission. If there are >1 sqes pending submission, the first io_uring_enter() call returns immediately.
When there is one cqe pending submission and a timeout is being provided via IORING_ENTER_EXT_ARG through io_uring_wait_cqe_timeout(), io_uring_enter() will submit the single pending sqe and wait for completion until the specified timeout. If io_uring_enter() successfully submits the sqe but the event has not completed within the timeout duration, _io_uring_get_cqe() will loop around to io_uring_enter() once more, this time with nothing to submit, and wait the timeout duration before returning -1 and setting errno to ETIMEDOUT.
In this scenario io_uring_enter() blocks for the entire timeout duration for both calls.
Summary:
struct io_uring_sqe *sqe;

sqe = io_uring_get_sqe(&ring);
io_uring_prep_whatever(sqe, ...);

/* one sqe not yet seen by the kernel */
io_uring_wait_cqe_timeout()

/*
 * io_uring_wait_cqe_timeout()
 * -> io_uring_wait_cqes()
 *    -> io_uring_wait_cqes_new()
 *       -> _io_uring_get_cqe()
 *            __sys_io_uring_enter2()   sqe submitted, waiting for completion or timeout
 *            __sys_io_uring_enter2()   nothing to submit, waiting for completion or timeout
 */

Tried to reproduce with the test below, no luck, it enters the kernel only once. Also, I don't see io_uring_wait_cqes_new() flushing requests. See @data in the function, data->submit is defaulted to 0.
What liburing version do you use? liburing git hash? What kernel version it was? Also, can you confirm whether IORING_FEAT_EXT_ARG is set? E.g. ring->features & IORING_FEAT_EXT_ARG.
static int test(struct io_uring *ring)
{
	struct __kernel_timespec ts = { .tv_sec = 0, .tv_nsec = 10000000 };
	struct io_uring_cqe *cqe;
	struct io_uring_sqe *sqe;
	int ret, fds[2];
	char buffer[200];

	if (pipe(fds)) {
		perror("pipe");
		return 1;
	}
	sqe = io_uring_get_sqe(ring);
	io_uring_prep_read(sqe, fds[0], buffer, 10, 0);

	ret = io_uring_wait_cqe_timeout(ring, &cqe, &ts);
	fprintf(stderr, "ret %d\n", ret); // -62
	return 0;
}

@kplantjr, any comments?

@isilence apologies, I missed your first reply.
I have re-tested with a fresh pull of liburing 2.1 and am currently on 5.14, it's been a while but I believe my previous tests were all on 5.10/5.11.
My experience now mirrors yours exactly, no double timeout. My previous tests were likely invalid or something has changed to fix this. Either way thanks for taking a look, going to close this issue.
Thanks!


LEGO: Pull request from lego/hb_839_20170317091229724 to master

LEGO: Pull request from lego/hb_839_20170317091229724 to master with localized lcls

@csigs,
Thanks for your contribution.
To ensure that the project team has proper rights to use your work, please complete the Contribution License Agreement at https://cla.microsoft.com.
It will cover your contributions to all Microsoft-managed open source projects.
Thanks,
Microsoft Pull Request Bot


Help with running a PHP module via HAB

HABridge is running well for me so far and I'd like to add a Nest component to my controls.
A very helpful chap has created a PHP module that can trigger the Hot Water boost that many users want but NEST refuse to supply.
His write up about it can be found here
(http://www.aimlesswandering.uk/web/2017/11/nest-thermostat-hot-water-controlled-by-alexa/)
I've created the relevant PHP file and now wondering how I can call that module from within HABridge
Im sure many people here might find this useful so was wondering if you are able to shed any light on it

You could do what the article suggests as one tool does not fit everything. Or you could create a script to call the PHP code on your ha-bridge host. FYI - I am not the one to do the script....

I've got a script but it was HOW I call a PHO script that I was asking about as this wasn't immediately obvious to me

Here is the construct of calling a script. https://github.com/bwssytems/ha-bridge#script-or-command-execution


Docs generation always includes imported packages

If I have a Pony program with a use statement, all code from other packages will be included in code documentation generation. Running ponyc mypkg --docs --pass docs on the following code:
use "collections"

class MyClass
  ...

will generate docs for mypkg, builtin, and collections. This is also the case for packages in Stable environments, and other packages that these packages include.
I understand how this is necessary for the stdlib, since it generates the docs for all packages through use statements. However, regular users are only interested in generating documentation of their own libraries. There should be some way to specify for which packages documentation should be generated, or at least the --docs flag description should be changed to describe this behaviour.

@EpicEric going to disagree with you on the "should be changed to point to official Pony docs page". Official pages update with each release. Documentation generation happens at a point in time. For a library, that could be different than the standard library because..

it was generated from a previous version.
it was generated from a custom version.

All that aside, how do you propose the compiler should know what packages it should generate documentation for and which it shouldn't?

Sorry for the late reply.
I think that the entire issue with the official doc pages being updated with each release would warrant different versions of the documentation to be hosted for every ponyc release, as to avoid API issues with mismatched versions, as well as reasons unrelated to this specific issue. But as you pointed out, this would not be suitable for custom ponyc distributions.
In that case, I would be fine with the alternative of simply removing links while still keeping a reference to stdlib/builtin objects, i.e. call objects by a full import path such as builtin.Array – similar to what Java does for generated documentation (example).
This could be extended to exclude different packages from docgen. If I was generating documentation for mypkg – and nothing else – with the following source:
use "collections"
use "./mysubpkg"

class MyClass
  ...

then any references to an object x from builtin, collections, and ./mysubpkg could be changed in the gendoc to, say, builtin.x, stdlib.collections.x and mypkg.mysubpkg.x respectively, or a different convention if there are better suggestions.
As to how the compiler would know which source files to consider part of generated documentation, there are a few options we could pass to a new ponyc flag:

Pass a whitelist of dirs with Pony source files (./mypkg,../anotherpkg) (the source dir is implicitly included)
Pass a whitelist of the Pony source files (*.pony,./mypkg/*.pony,../anotherpkg/*.pony) (the source files are NOT implicitly included, making this a breaking change)
Pass a whitelist of package names (mypkg,../anotherpkg) (the source package is implicitly included)
Pass a blacklist of package names (builtin,collection,notthispkg)

I don't know much about the compiler itself, but if it correlates objects with its origin package during compilation, the third option might be simpler to implement and easier to use, possibly even with the use of pony-stable. Of course, someone who has actually touched the compiler might know if these assumptions are baseless or not.


 We do --no-cache-dir on
pip install, but turns out, this isn't enough. Stuff from within
get-pip.py ends up leveraging the http cache as well, so we should
just nuke it entirely.

Oh pip... :disappointed:  LGTM

Can we use ~/.cache instead just to help insulate ourselves a tiny bit from HOME screwiness? :innocent:

Can we use ~/.cache instead just to help insulate ourselves a tiny bit from HOME screwiness? :innocent:

Is it possible that it'd be built while running under a different user? I'm not against the change, just curious why that scenario may happen.

Updated to use ~ instead of /root

Not today, but we've been bitten by that very bug before, so I'm just
trying to be extra careful -- if you don't have any cycles to spare to
update that, I can do the minor change post-merge too. :+1:

You were faster than GitHub's email to comment gateway :laughing:
LGTM, thanks :+1:


SourceLink does not work

Issue / Steps to reproduce the problem
SourceLink cannot be loaded in vs2019
Relevant parts of the log file
Here is the sourcelink test in the following pdb:
%userprofile%.nuget\packages\identityserver4\3.1.2\lib\netcoreapp3.1\IdentityServer4.pdb
Command:
>sourcelink test IdentityServer4.dll

Result:
1 Documents with errors:
fbcb5dddb109a7a0ec1695c07690b932b9abe38b6b7dbc56ee75b3c78b582f03 sha256 csharp /Users/dominick/Documents/Repos/IdentityServer/IdentityServer4/src/IdentityServer4/src/obj/Release/netcoreapp3.1/IdentityServer4.AssemblyInfo.cs
https://raw.githubusercontent.com/IdentityServer/IdentityServer4/3efcef29c14d0d8756fd8af63a0474091e8fac51/src/IdentityServer4/src/obj/Release/netcoreapp3.1/IdentityServer4.AssemblyInfo.cs
error: url failed NotFound: Not Found
sourcelink test failed

Fiddler log when loading symbol from nuget in vs2019

Seems all good to me.


Fix 7 vulnerabilities, add softether.net after-2038 year certificate hash, add vpncmd message about Developer Edition

(1) Fix Vulnerability: CVE-2023-27395 TALOS-2023-1735
SoftEther VPN vpnserver WpcParsePacket () heap-based buffer overflow vulnerability
https://www.softether.org/9-about/News/904-SEVPN202301
https://jvn.jp/en/jp/JVN64316789/
(2) Fix Vulnerability: CVE-2023-22325 TALOS-2023-1736
SoftEther VPN DCRegister DDNS_RPC_MAX_RECV_SIZE denial of service vulnerability
https://www.softether.org/9-about/News/904-SEVPN202301
https://jvn.jp/en/jp/JVN64316789/
(3) Fix Vulnerability: CVE-2023-32275 TALOS-2023-1753
SoftEther VPN CtEnumCa () information disclosure vulnerability
https://www.softether.org/9-about/News/904-SEVPN202301
https://jvn.jp/en/jp/JVN64316789/
(4) Fix Vulnerability: CVE-2023-27516 TALOS-2023-1754
SoftEther VPN CiRpcAccepted () authentication bypass vulnerability
https://www.softether.org/9-about/News/904-SEVPN202301
https://jvn.jp/en/jp/JVN64316789/
(5) Fix Vulnerability: CVE-2023-32634 TALOS-2023-1755
SoftEther VPN CiRpcServerThread () MitM authentication bypass vulnerability
https://www.softether.org/9-about/News/904-SEVPN202301
https://jvn.jp/en/jp/JVN64316789/
(6) Fix Vulnerability: CVE-2023-31192 TALOS-2023-1768
SoftEther VPN ClientConnect () information disclosure vulnerability
https://www.softether.org/9-about/News/904-SEVPN202301
https://jvn.jp/en/jp/JVN64316789/
(7) Fix Vulnerability: CVE-2023-25774 TALOS-2023-1743
SoftEther VPN vpnserver ConnectionAccept () denial of service vulnerability
(8) Add four new certificate hashes to the DDNS_CERT_HASH list by Daiyuu Nobori. These certificates will be used to University of Tsukuba's built-in "softether.net" DDNS server after year 2038.
(9) Showing an explanation of the purpose of the Developer Edition and the difference from the Stable Editon by Daiyuu Nobori

I was worried about security fixes made on recent a few years are not imported to the developer edition.
Thanks for clarifying the difference between editions and importing security fixes!


release: update dependencies

update core-sdk package and dependencies

:tada: This PR is included in version 1.0.43 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[karma-esm] version higher than 2.9.8 has problems with karma-viewport framework

Starting from karma-esm 2.10.1 when I'm including karma-viewport like this:
frameworks: ['viewport'],
esm: {
    nodeResolve: true,
    exclude: ['**/node_modules/karma-viewport/**/*.json.js'],
},

I have errors:
START:
31 01 2020 09:37:51.879:INFO [karma-server]: Karma v4.4.1 server started at http://<IP_ADDRESS>:9876/
31 01 2020 09:37:51.882:INFO [launcher]: Launching browsers ChromeHeadlessNoSandbox with concurrency unlimited
31 01 2020 09:37:51.892:INFO [launcher]: Starting browser ChromeHeadless
31 01 2020 09:37:52.573:INFO [HeadlessChrome 79.0.3945 (Windows 10.0.0)]: Connected on socket qTee_q5we908UT_zAAAA with id 18822107
[karma-esm]: Could not find requested file: /node_modules/karma-viewport/dist/config/default.json.js?d74df6fc8ebb31118b0c9710eab2dba5f09da45c
[karma-esm]: Could not find requested file: /node_modules/karma-viewport/dist/config/default.json.js?d74df6fc8ebb31118b0c9710eab2dba5f09da45c
HeadlessChrome 79.0.3945 (Windows 10.0.0) ERROR
  Uncaught TypeError: Cannot read property 'context' of undefined
  at node_modules/karma-viewport/dist/adapter/index.js:1:2575

  TypeError: Cannot read property 'context' of undefined
      at new e (node_modules/karma-viewport/dist/adapter/webpack:/src/adapter/viewport.ts:134:56)
      at node_modules/karma-viewport/dist/adapter/webpack:/src/adapter/index.ts:47:21
      at Object.call (node_modules/karma-viewport/dist/adapter/webpack:/src/adapter/index.ts:46:1)
      at n (node_modules/karma-viewport/dist/adapter/webpack:/webpack/bootstrap:19:22)
      at Object.call (node_modules/karma-viewport/dist/adapter/index.js:1:993)
      at __webpack_require__ (node_modules/karma-viewport/dist/adapter/webpack:/webpack/bootstrap:19:22)
      at node_modules/karma-viewport/dist/adapter/webpack:/webpack/bootstrap:83:10
      at node_modules/karma-viewport/dist/adapter/index.js:1:965

I'm using reateDefaultConfig from @open-wc/testing-karma, version 2.9.8 is working with this configuration.
I believe this is connected with this lines
esmConfig.babelExclude = [
    ...(esmConfig.babelExclude || []),
    // exclude regular test scripts/libs from babel
    ...karmaConfig.files
      .filter(f => f.pattern.endsWith('.js') && f.type !== 'module')
      .map(f => f.pattern),
  ];

from commit.

Hi there,
We have exactly the same problem in our team:
13 05 2020 15:24:50.636:INFO [karma-server]: Karma v4.4.1 server started at http://<IP_ADDRESS>:9876/
13 05 2020 15:24:50.637:INFO [launcher]: Launching browsers ChromeHeadlessNoSandbox with concurrency unlimited
13 05 2020 15:24:50.642:INFO [launcher]: Starting browser ChromeHeadless
13 05 2020 15:24:51.255:INFO [HeadlessChrome 81.0.4044 (Mac OS X 10.14.6)]: Connected on socket wWFIFhHa0C3MclV8AAAA with id 94238269
[karma-esm]: Could not find requested file: /node_modules/karma-viewport/dist/config/default.json.js?f98e38d0e10ae3a2ebc58886afa55ba0a891f91a
[karma-esm]: Could not find requested file: /node_modules/karma-viewport/dist/config/default.json.js?f98e38d0e10ae3a2ebc58886afa55ba0a891f91a
HeadlessChrome 81.0.4044 (Mac OS X 10.14.6) ERROR
  Uncaught TypeError: Cannot read property 'context' of undefined
  at node_modules/karma-viewport/dist/adapter/index.js:1:1592

  TypeError: Cannot read property 'context' of undefined
      at new e (node_modules/karma-viewport/dist/adapter/webpack:/src/adapter/viewport.ts:134:56)
      at node_modules/karma-viewport/dist/adapter/webpack:/src/adapter/index.ts:47:21
      at Object.call (node_modules/karma-viewport/dist/adapter/webpack:/src/adapter/index.ts:46:1)
      at n (node_modules/karma-viewport/dist/adapter/webpack:/webpack/bootstrap:19:22)
      at Object.call (node_modules/karma-viewport/dist/adapter/index.js:1:993)
      at __webpack_require__ (node_modules/karma-viewport/dist/adapter/webpack:/webpack/bootstrap:19:22)
      at node_modules/karma-viewport/dist/adapter/webpack:/webpack/bootstrap:83:10
      at node_modules/karma-viewport/dist/adapter/index.js:1:965
HeadlessChrome 81.0.4044 (Mac OS X 10.14.6) ERROR
  Uncaught TypeError: Cannot read property 'context' of undefined
  at node_modules/karma-viewport/dist/adapter/index.js:1:1592

  TypeError: Cannot read property 'context' of undefined
      at new e (node_modules/karma-viewport/dist/adapter/webpack:/src/adapter/viewport.ts:134:56)
      at node_modules/karma-viewport/dist/adapter/webpack:/src/adapter/index.ts:47:21
      at Object.call (node_modules/karma-viewport/dist/adapter/webpack:/src/adapter/index.ts:46:1)
      at n (node_modules/karma-viewport/dist/adapter/webpack:/webpack/bootstrap:19:22)
      at Object.call (node_modules/karma-viewport/dist/adapter/index.js:1:993)
      at __webpack_require__ (node_modules/karma-viewport/dist/adapter/webpack:/webpack/bootstrap:19:22)
      at node_modules/karma-viewport/dist/adapter/webpack:/webpack/bootstrap:83:10
      at node_modules/karma-viewport/dist/adapter/index.js:1:965

This is our karma.config file:
/* eslint-disable import/no-extraneous-dependencies */
const { createDefaultConfig } = require('@open-wc/testing-karma');
const deepmerge = require('deepmerge');
const { breakpoints } = require('./viewports.config.js');

module.exports = config => {
  config.set(
    deepmerge(createDefaultConfig(config), {
      files: [
        // runs all files ending with .test in the test folder,
        // can be overwritten by passing a --grep flag. examples:
        //
        // npm run test -- --grep test/foo/bar.test.js
        // npm run test -- --grep test/bar/*
        {
          pattern: config.grep ? config.grep : 'packages/*/*/test/**/*.test.js',
          type: 'module',
        },
      ],

      // see the karma-esm docs for all options
      esm: {
        // if you are using 'bare module imports' you will need this option
        nodeResolve: true,

        // Custom babel config to replace the process node enviroment
        // Right now this get used by Redux
        // Note: We could change this for redux.mjs
        babelConfig: {
          plugins: [
            [
              'transform-define',
              {
                'process.env.NODE_ENV': 'production',
              },
            ],
          ],
        },
      },

      frameworks: ['viewport'],

      // This allow us to test our component with different viewports
      viewport: {
        context: '#viewport',
        breakpoints,
      },

      // Test coverage settings:
      coverageIstanbulReporter: {
        thresholds: {
          emitWarning: false,
          global: {
            statements: 80,
            branches: 80,
            functions: 80,
            lines: 80,
          },
        },
      },
    }),
  );

  return config;
};

Any idea is this could be fixed or another way to have unit-test and test different viewports using karma-esm?
Thank you 🙏

I have the same issue, any idea on when this will be fixed?

Same issue here, did you guys got a fix for this one.

01 09 2020 16:28:18.029:INFO [karma-server]: Karma v5.1.1 server started at http://localhost:9876/
01 09 2020 16:28:18.031:INFO [launcher]: Launching browsers ChromeHeadlessNoSandbox with concurrency unlimited
01 09 2020 16:28:18.037:INFO [launcher]: Starting browser ChromeHeadless
01 09 2020 16:28:20.601:INFO [Chrome Headless 85.0.4183.83 (Windows 10)]: Connected on socket 4nRHOcOUlyVl9aHlAAAA with id 38001012
01 09 2020 16:28:20.805:WARN [middleware:karma]: Unable to determine file type from the file extension, defaulting to js.
  To silence the warning specify a valid type for D:/Work/Source/SupportTools/PLPWebComponents/node_modules/karma-viewport/dist/adapter/index.js.map in the configuration file.
  See http://karma-runner.github.io/latest/config/files.html
[karma-esm]: Could not find requested file: /node_modules/karma-viewport/dist/config/default.json.js?d74df6fc8ebb31118b0c9710eab2dba5f09da45c
[karma-esm]: Could not find requested file: /node_modules/karma-viewport/dist/config/default.json.js?d74df6fc8ebb31118b0c9710eab2dba5f09da45c
Chrome Headless 85.0.4183.83 (Windows 10) ERROR
  Uncaught TypeError: Cannot read property 'context' of undefined
  at node_modules/karma-viewport/dist/adapter/index.js:1:1618

  TypeError: Cannot read property 'context' of undefined
      at new e (node_modules/karma-viewport/dist/adapter/webpack:/src/adapter/viewport.ts:134:56)
      at node_modules/karma-viewport/dist/adapter/webpack:/src/adapter/index.ts:47:21
      at Object.call (node_modules/karma-viewport/dist/adapter/webpack:/src/adapter/index.ts:46:1)
      at n (node_modules/karma-viewport/dist/adapter/webpack:/webpack/bootstrap:19:22)
      at Object.call (node_modules/karma-viewport/dist/adapter/index.js:1:993)
      at __webpack_require__ (node_modules/karma-viewport/dist/adapter/webpack:/webpack/bootstrap:19:22)
      at node_modules/karma-viewport/dist/adapter/webpack:/webpack/bootstrap:83:10
      at node_modules/karma-viewport/dist/adapter/index.js:1:965

Finished in 0.529 secs / 0 secs @ 16:28:21 GMT-0400 (Eastern Daylight Time)

Karma viewport doensn't work with karma-esm because we have to hack around karma to make it work with native es modules. There isn't anything we can do to change that.
We have been working on a new test runner, Web Test Runner, we will recommending this from open-wc soon. https://modern-web.dev/docs/test-runner/overview/
This properly supports testing different viewports: https://modern-web.dev/docs/test-runner/commands/#viewport
I will close this issue as we cannot change anything here.


fix: allow global components to be stubbed

Coming from https://github.com/posva/vue-router-mock/issues/96
The warning happens because I do this: https://github.com/posva/vue-router-mock/blob/v2/src/injections.ts#L37-L41
I think this change is useful to toggle stubs with stubs: {RouterLink: false}

I'm realizing @cexbrayat already submitted a different fix at https://github.com/vuejs/vue-test-utils-next/pull/483/files. I think it's more reliable to not use the internal API of Vue if possible and avoid registering the global component when possible like in this PR


More speed

Current dependencies on/for this PR:

main

PR #3831

PR #3860

PR #3861   👈

This comment was auto-generated by Graphite.


Could u explain what is the meaning of _inst_nostuff.pth?

Hello, I am a new beginner in 3D deep learning, could u explain what is the meaning of _inst_nostuff.pth and what is used to do?
I open it, I found a data.pkl and version, what they are standing for?
I will appreciate u if u could answer these questions.

a .pth file can be read by torch.load function. Try. Hope that helps.


Don't break long lines mid-word or at hyphens, to avoid breaking URLs

Fixes gh-68
This should probably have a test added, but I don't dare touch those
files and generate even more conflicts while gh-66 and gh-68 are
outstanding...

Codecov Report

Merging #69 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master      #69   +/-   ##
=======================================
  Coverage   81.81%   81.81%
=======================================
  Files          12       12
  Lines         407      407
  Branches       51       51
=======================================
  Hits          333      333
  Misses         53       53
  Partials       21       21

Impacted Files
Coverage Δ

src/towncrier/_builder.py
97.1% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e07a7b7...cbd165c. Read the comment docs.

@njsmith A test would be nice


Add snake game

This is with reference to #10 .

You weren't assigned to this issue. Please close your PR.

I thought the readme file can have many list of users thats why I gave this PR. What should I do @mishal23

@pulkonet That is fine, you may also, send a PR with your project. We are looking for all the projects of that kind.
So feel free to send a PR

@prakamya-mishra Done close the PR, yes many of them can be there.


Runtime should not be depending on module entry

Hi. Escalating this from https://github.com/unjs/nitro/issues/1629 while I was debugging, noticed your module is trying to bundle c12 into the runtime.
[12:21:18 AM]  ERROR  "rm" is not exported by<EMAIL_ADDRESS>imported by<EMAIL_ADDRESS>
This happens because the utils in runtime directory are trying to import from nuxt-site-config-kit which is the module entry containing build time dependencies including @nuxt/kit.
You should never combine them. If you really need shared dependencies, they should be exported from a subpath export of the module, ideally residing within src/runtime directory.
BTW nuxt should have failed with import protection plugin not sure why is not caught. /cc @danielroe

Hmm interesting, I figured that the tree shaker would have been able to remove the dependency since there's no side effects :thinking: Will update soon.

Actually, digging further into it, I can't see any evidence of what you're describing @pi0, could you provide the code that's causing the issue?
There's nothing in the runtime directory importing from nuxt-site-config-kit. The only thing that uses it is the module entry itself.

Okay on even further digging, it seems like the issue is that the site-config-stack package has a dependency for @nuxt/kit.
  "dependencies": {
    "@nuxt/kit": "^3.6.5",
    "defu": "^6.1.2",
    "pkg-types": "^1.0.3",
    "ufo": "^1.2.0"
  }

However, this dependency is never used in the code, so not sure why it's getting bundled exactly. Seems like something the upstream can handle better.

Adding nuxt/kit in a package.json shouldn't result in it getting bundled. I'd suggest we should keep looking.

I'm going to close this as I believe it was a false positive anyway (see my above comment).


Add columns in get_query

When I use get_query I would also like to have access to the column names of the table in order to create a DataFrame with the proper column names. Normally those can be found in the cursor.description.
Just converting to dataframe does not preserve column names:
res = conn.get_query(query, query_params=query_params)
data = pd.DataFrame(res)

Is there maybe another way to get dataframe from the output if get_query?

You can use the existing get_dict method to create a dataframe. Does that work for you?
res = conn.get_dict(query, query_params=query_params)
data = pd.DataFrame.from_records(res)

Yes it did! I had to inject parameter because get_dict does not have query_params, like this:
query = inject_parameters(query, query_params)
res = context.resources.db_connection.get_dict(query)
data = pd.DataFrame.from_records(res)


utils: rename Exporter.finished() signal

There is already a QThread.finished() signal that is emitted
automatically when the thread exits:
https://doc.qt.io/qt-5/qthread.html#finished
In PyQt5 you could overwrite parent classes signals. With PySide2 the
existing QThread.finished() is still triggered, even though we could
think it was overwritten by Exporter.finished(), leading to connected
slots being called twice.
In the case of Exporter.finished(), the signal must only be emitted if the
export actually succeeded. Hence the chosen solution here is to rename
the signal to Exporter.export_finished().

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


SAMA5D2x SDMMC peripheral support (SD Card driver)

Summary

SDMMC peripheral support for SAMA5D2 series microprocessors from Microchip
provides an MMC/SD Card driver that works for reading and writing
can be used with or without DMA
UHS_SDR50, UHS_DDR50, and UHS_SDR104 high-speed modes are supported, with
read/write speeds of 400Mbit/s in UHS_DDR50 (50MHz Double Data Rate) mode.
This is a port of the NuttX imxrt_usdhc.c driver, as the SDMMC peripheral
uses a similar Cadence IP block.
Basic instructions are in the README.txt.

Impact

None if you don't enable the SDMMC peripheral using the Kconfig system.
Enables reading and writing to MMC or SD Cards. Only SD Cards have been tested.
VFAT filesystem has been tested and works. On the SAMA5D2-XULT board, reading and writing to a bootable SD Card is supported.

Testing

Only tested on the SAMA5D2-XULT board from Microchip.
Only manual read and write testing on a VFAT-formatted memory card has been done.
Driver was developed with extensive use of a logic analyzer to ensure the SDMMC driver was functioning correctly.
Only UHS_DDR50 mode was tested, as all my SD Cards support only that.

How to Test

Format an SD Card with VFAT format.
Put some text files on the card.
Insert the card into the SAMA5D2-XULT SD Card slot.
Build the sama5d2-xult:sdmmcnsh configuration
Load NuttX onto the board
/mnt/mmcsd1 should have your files. You can cd, ls, cp, rm, and cat
to test the filesystem.

TODO / Limitations

e.MMC should work, but was not tested
Performance tests have not been done
UHS_SDR50 and UHS_SDR104 modes were not tested as I don't have any
SD Cards that support those modes.

Notes

nxstyle shows some formatting errors, but I couldn't find good ways to make them go away without mangling the files. I welcome ideas on how to better format the code so nxstyle passes!

@btashton would you be willing to help me find someone who can review this?
I'm working to get Groboard's GiantBoard and the ACME RoadRunner module supported and added to NuttX— they're both SAMA5 boards so having SDMMC support would help. I know it's a lot of code, not sure how to avoid that.

@Ouss4 Re: headers, will fix that. Thank you.

@Ouss4 Ok, I changed the headers on the new files to Apache License 2.0, and removed the obsolete config value. Anything else to do here?

@Ouss4 BTW the two main files, sam_sdmmc.h and sam_sdmmc.c are ported from imxrt_usdhc.h and imxrt_usdhc.c.

@Ouss4 Thanks, I pushed changes to remove the obsolete config values, I'll watch the CI and try to resolve any other problems.

@Ouss4 Ok, looks like all the non-nxstyle checks are passing. I ran nxstyle locally and fixed all the problems I could before submitting this PR; the remaining nxstyle issues I don't have good ideas about how to fix... if you have suggestions, please let me know!

the remaining nxstyle issues I don't have good ideas about how to fix... if you have suggestions, please let me know!

For header file the "error: Invalid section for this file type" is a section named "Public Function" that needs to be "Public Function Prototypes".  There are also a couple of long line and "too many blank lines" that can easily be fixed.  For the rest, I don't see how to fix them, it's hard to make nxstyle happy with register definition files.

@Ouss4 Oops, I am sorry I missed the easy ones. I'll fix the header name, the long lines, and the blank lines.
For the rest of the nxstyle errors, what do you suggest? Should we just ignore them?

For the rest of the nxstyle errors, what do you suggest? Should we just ignore them?

I don't see a way to make nxstyle happy without messing up the file, so that would be my suggestion.
One last thing, could you please squash the commits if you are happy with the changes?

@Ouss4 Thank you. I will run one more set of manual tests on the SAMA5D2-XULT board, and also wait for the current set of CI builds to finish. If that all goes well, I'll squash and force-push, and then let you know I've done it.

@Ouss4 I retested on hardware and force-pushed. Now it looks like the macOS (arm-12), (mips-riscv-x86-xtensa), and (sim) builds failed. But the Linux versions succeeded. Is this expected?
If these failures are unexpected— will you let me know how to debug? I don't see any detail from the CI runs.
If these failures are ok, then this PR is ready to merge.

@adamfeuer Yes, sometimes some of the macOS checks don't make it to the end.  It's Github things apparently.

@adamfeuer Yes, sometimes some of the macOS checks don't make it to the end.  It's Github things apparently.

I actually re-kicked the build a while back and they are passing now. For some reason the old canceled jobs are showing up on this run. But you will also see the new Mac jobs that passed.
All that is failing is the style check.

@btashton Thanks for doing that. How does one restart the build? Is that something I can do without pushing a new change?

@Ouss4 This is ready to merge now— the only thing that's not passing is the style check for some header files, as we discussed it's hard to get nxstyle to be happy with them.

How does one restart the build? Is that something I can do without pushing a new change?

In the checks tab, there is a button at the right that says "Re-run jobs".

@Ouss4 You're welcome.
By the way, the "Re-run jobs" button doesn't show up for me. Maybe because I'm not a member of the NuttX Github team?


test: functional: Fix mempool-* functional tests.

Goal of this pull request is to fix mempool_* functional tests in bitcoinsync2022 branch.
Notes
bitgesell-2022$ test/functional/test_runner.py test/functional/mempool_*

...

TEST                             | STATUS    | DURATION

mempool_accept.py                | ✓ Passed  | 1 s
mempool_accept_wtxid.py          | ✓ Passed  | 12 s
mempool_datacarrier.py           | ✓ Passed  | 1 s
mempool_expiry.py                | ✓ Passed  | 1 s
mempool_limit.py                 | ✓ Passed  | 3 s
mempool_package_limits.py        | ✓ Passed  | 2 s
mempool_package_onemore.py       | ✓ Passed  | 1 s
mempool_packages.py              | ✓ Passed  | 6 s
mempool_persist.py --descriptors | ✓ Passed  | 14 s
mempool_reorg.py                 | ✓ Passed  | 2 s
mempool_resurrect.py             | ✓ Passed  | 1 s
mempool_spend_coinbase.py        | ✓ Passed  | 1 s
mempool_unbroadcast.py           | ✓ Passed  | 6 s
mempool_compatibility.py         | ○ Skipped | 0 s
mempool_updatefromblock.py       | ○ Skipped | 0 s

ALL                              | ✓ Passed  | 51 s (accumulated)

Hey @gitcoindev,
This is coming along great, were you able to build the bitcoinsync2022 branch successfully?

Hi @naftalimurgor thank you!. The 2022 branch does not build out of the box, seems due to some integration hiccup, build system complains about QT and cpp tests Makefiles and this will need to be repaired in the future.
The exact error message is already shown in the autogen.sh run:
configure.ac:1968: error: required file 'src/qt/Makefile' not found
configure.ac:1969: error: required file 'src/qt/test/Makefile' not found
configure.ac:1970: error: required file 'src/test/Makefile' not found

Since I am working on functional tests, I do not need QT nor cpp unit tests and I was able to copy missing parts from my master branch workspace and build as follows:
$ cp ../bitgesell-integration/src/qt/Makefile src/qt/Makefile
$ cp ../bitgesell-integration/src/qt/test/Makefile src/qt/test/Makefile
$ touch src/test/Makefile
$ ./autogen.sh
$ ./configure --with-incompatible-bdb --with-gui=no --disable-tests
$ make -j16

--with-gui=no disables QT part
--disable-tests disables CPP unit tests
I had to use --with-incompatible-bdb as well (BDB from contrib/install_sb4.sh works perfectly fine on my master branch and I used it for wallet tests).
I hope that helps and will enable you to build as well.

Thanks @gitcoindev What's your discord/Telegram?

I am on discord @korrrba , but mostly offline, checking it just from time to time.

Awesome, I'll shoot you a message there

Hi @naftalimurgor thank you!. The 2022 branch does not build out of the box, seems due to some integration hiccup, build system complains about QT and cpp tests Makefiles and this will need to be repaired in the future.
The exact error message is already shown in the autogen.sh run:
configure.ac:1968: error: required file 'src/qt/Makefile' not found
configure.ac:1969: error: required file 'src/qt/test/Makefile' not found
configure.ac:1970: error: required file 'src/test/Makefile' not found

The above is not an issue but left of from legacy code. Bitcoin added some lines to check and link those Makefile during autogen.sh run. However, we have not updated .gitignore to properly pick those files.

Since I am working on functional tests, I do not need QT nor cpp unit tests and I was able to copy missing parts from my master branch workspace and build as follows:
$ cp ../bitgesell-integration/src/qt/Makefile src/qt/Makefile
$ cp ../bitgesell-integration/src/qt/test/Makefile src/qt/test/Makefile
$ touch src/test/Makefile
$ ./autogen.sh
$ ./configure --with-incompatible-bdb --with-gui=no --disable-tests
$ make -j16

--with-gui=no disables QT part --disable-tests disables CPP unit tests
I had to use --with-incompatible-bdb as well (BDB from contrib/install_sb4.sh works perfectly fine on my master branch and I used it for wallet tests).
I hope that helps and will enable you to build as well.

I would go through your code (review).

Goal of this pull request is to fix mempool_* functional tests in bitcoinsync2022 branch.
Notes
bitgesell-2022$ test/functional/test_runner.py test/functional/mempool_*

...

TEST                             | STATUS    | DURATION

mempool_accept.py                | ✓ Passed  | 1 s
mempool_accept_wtxid.py          | ✓ Passed  | 12 s
mempool_datacarrier.py           | ✓ Passed  | 1 s
mempool_expiry.py                | ✓ Passed  | 1 s
mempool_limit.py                 | ✓ Passed  | 3 s
mempool_package_limits.py        | ✓ Passed  | 2 s
mempool_package_onemore.py       | ✓ Passed  | 1 s
mempool_packages.py              | ✓ Passed  | 6 s
mempool_persist.py --descriptors | ✓ Passed  | 14 s
mempool_reorg.py                 | ✓ Passed  | 2 s
mempool_resurrect.py             | ✓ Passed  | 1 s
mempool_spend_coinbase.py        | ✓ Passed  | 1 s
mempool_unbroadcast.py           | ✓ Passed  | 6 s
mempool_compatibility.py         | ○ Skipped | 0 s
mempool_updatefromblock.py       | ○ Skipped | 0 s

ALL                              | ✓ Passed  | 51 s (accumulated)

Got the following:
TEST                             | STATUS    | DURATION
mempool_accept.py                | ✓ Passed  | 8 s
mempool_accept_wtxid.py          | ✓ Passed  | 19 s
mempool_datacarrier.py           | ✓ Passed  | 8 s
mempool_expiry.py                | ✓ Passed  | 10 s
mempool_limit.py                 | ✓ Passed  | 7 s
mempool_package_limits.py        | ✓ Passed  | 7 s
mempool_package_onemore.py       | ✓ Passed  | 4 s
mempool_packages.py              | ✓ Passed  | 18 s
mempool_persist.py --descriptors | ✓ Passed  | 33 s
mempool_reorg.py                 | ✓ Passed  | 9 s
mempool_resurrect.py             | ✓ Passed  | 6 s
mempool_spend_coinbase.py        | ✓ Passed  | 8 s
mempool_unbroadcast.py           | ✓ Passed  | 21 s
mempool_compatibility.py         | ○ Skipped | 1 s
mempool_updatefromblock.py       | ✖ Failed  | 5 s
ALL                              | ✖ Failed  | 164 s (accumulated)
Runtime: 48 s
I used your branch, * bugfix/bitcoinsync2022-fix-mempool-functional-tests b598dc76a test: functional: Fix mempool-* functional tests.
Did you flag updatefromblock.py to be skipped? If yes, it looks like you didn't include it to your commit.  I am going to re-run your branch.

Hi @janus let me check again. I have worked on an older laptop before on Ubuntu 18.04. Now will try with Ubuntu 20.04, both clang and gcc. I did not mark it as skipped, but if it is skipped will check why.

Hi @janus just a follow up on this, the test was not enabled due to bdb not installed. I installed bdb and it fails, but there is indeed something wrong on bitcoin2022 branch with the memory pool. The test case prepares 100 transactions and mines them in batches of 25 using generate function. After a batch of transactions is mined, the memory pool should be emptied, but last 2-4 transactions are left out which causes the test case to fail:
-4675-> getrawmempool {}
<-4675- [0.000928] ["573ed9b27f5707afc2a1898df1f4991224fd3049d50bf7684d633d47467bd491", "1c8c4b22a0c2ca19816f15afdcb21c2ba4ee1130dad478736996eceffaa19e45", "5bbda968352510a890aee6a0cc06479f8cf9c7cf6bfcdf2f8ab39dd8634514a9", "37c64a364fa1b6cbe19adff3f0301ad86d477ef75654ad0ced74429c3d2ecddd", "d102d04c4af891568e550771c747c66759460dd6f93d3e85cbbddf295fd5d0d8", "35b4d2edd1565f257bcb2d177dc48e2c7a2c59063a9dac4a422b7c574fd266af", "d7b53e2af901b7945570e49d5bf634831dc247d47a41562bd7d55a55d048f8e1", "6fde1af0bf44d61335f7edc36880c7bd5029b6303e13ae227dc2226fd71d9b60", "dfc788544bcae8667969ae744d575028bff4c800e1909b0695f86f6d551d6b1b", "ece96d0219733abf777644500c0c1663e1c4baab2d16b5d7b83eb62a05613c5b", "9fbeaee77f821a59dd9da67e339956595045e30225f8fd864e1ee3c0987bb49e", "7b5c68a74cab10efdfe8bced52ae5a0be079be473bba39f1f46d35409829c2ac", "68410c6cf79871a5fedf416d6314bd2d997974e7d52bc26c59eb050401459ec2", "de856d95f665a16ae25af394eb3f8c7df09126a979da8eb0a19b7744bf7dd3a3", "e8171f9dbfcd621039488fe321088b3cc1f32257154a8325ffcd8c5d001e3f97", "826ca1c4865de847097b3ad35d99f807044246e941ef24bdd4835199ef2fc0be", "b28fc0999669932499df5a4b2116b5225d7edd8b2047f1f7220a963230b60dbf", "2318a82f9ebbbaef1771d8a6c7076b71d537bb1dcbb97e7c97ad448052dfe8d8", "6c937bcb3125881500e12c740b9902cb07de6ee11d51eb726c78357d7e641d08", "b8b7b7d77985aa634f2e34480897a49b661596a67cb0ec32572112bb764b8fbf", "08ea01e3c9d4bf5627ed9b9e81d0e9034dbf974163a0402bd83674fbd0c7a3c7", "d2a049d6eb7d8cb0560268b6aabdc0461962811a92c3d733a5dc2d3127a7a774", "142fe7bf3e52ec881ec13c2495004d2a3a86084fe945663aa712e1cfbc000b8b", "fede1e02c3e0902ed5d74546da6c599d141e5d2348610df68f7d70a8fcaf66d2", "c9848ca040f31b822bd51e906a307acdfc0f720d96d5e1807e13f9dc3df7f22f"]
2023-07-03T15:53:15.610000Z TestFramework (INFO): The batch of 25 transactions has been accepted into the mempool.
2023-07-03T15:53:15.610000Z TestFramework.node0 (DEBUG): TestNode.generate() dispatches `generate` call to `generatetoaddress`
-4676-> generatetoaddress {"nblocks": 1, "address": "rbgl1qqpmtreykaj2jphmyf3358sn3wudm6thfypzzut", "maxtries": 1000000}
<-4676- [0.014817] ["46e1e2e652518ae03ce38471982047647369ba79534b86f209fa0b7350b5e891"]
-4677-> getbestblockhash {}
<-4677- [0.000486] "46e1e2e652518ae03ce38471982047647369ba79534b86f209fa0b7350b5e891"
-4678-> getrawmempool {}
<-4678- [0.000579] ["fede1e02c3e0902ed5d74546da6c599d141e5d2348610df68f7d70a8fcaf66d2", "c9848ca040f31b822bd51e906a307acdfc0f720d96d5e1807e13f9dc3df7f22f"] <--- this list should be empty

I need some more time to find a root cause and fix this.

Hi @janus just a follow up on this, the test was not enabled due to bdb not installed. I installed bdb and it fails, but there is indeed something wrong on bitcoin2022 branch with the memory pool. The test case prepares 100 transactions and mines them in batches of 25 using generate function. After a batch of transactions is mined, the memory pool should be emptied, but last 2-4 transactions are left out which causes the test case to fail:
-4675-> getrawmempool {}
<-4675- [0.000928] ["573ed9b27f5707afc2a1898df1f4991224fd3049d50bf7684d633d47467bd491", "1c8c4b22a0c2ca19816f15afdcb21c2ba4ee1130dad478736996eceffaa19e45", "5bbda968352510a890aee6a0cc06479f8cf9c7cf6bfcdf2f8ab39dd8634514a9", "37c64a364fa1b6cbe19adff3f0301ad86d477ef75654ad0ced74429c3d2ecddd", "d102d04c4af891568e550771c747c66759460dd6f93d3e85cbbddf295fd5d0d8", "35b4d2edd1565f257bcb2d177dc48e2c7a2c59063a9dac4a422b7c574fd266af", "d7b53e2af901b7945570e49d5bf634831dc247d47a41562bd7d55a55d048f8e1", "6fde1af0bf44d61335f7edc36880c7bd5029b6303e13ae227dc2226fd71d9b60", "dfc788544bcae8667969ae744d575028bff4c800e1909b0695f86f6d551d6b1b", "ece96d0219733abf777644500c0c1663e1c4baab2d16b5d7b83eb62a05613c5b", "9fbeaee77f821a59dd9da67e339956595045e30225f8fd864e1ee3c0987bb49e", "7b5c68a74cab10efdfe8bced52ae5a0be079be473bba39f1f46d35409829c2ac", "68410c6cf79871a5fedf416d6314bd2d997974e7d52bc26c59eb050401459ec2", "de856d95f665a16ae25af394eb3f8c7df09126a979da8eb0a19b7744bf7dd3a3", "e8171f9dbfcd621039488fe321088b3cc1f32257154a8325ffcd8c5d001e3f97", "826ca1c4865de847097b3ad35d99f807044246e941ef24bdd4835199ef2fc0be", "b28fc0999669932499df5a4b2116b5225d7edd8b2047f1f7220a963230b60dbf", "2318a82f9ebbbaef1771d8a6c7076b71d537bb1dcbb97e7c97ad448052dfe8d8", "6c937bcb3125881500e12c740b9902cb07de6ee11d51eb726c78357d7e641d08", "b8b7b7d77985aa634f2e34480897a49b661596a67cb0ec32572112bb764b8fbf", "08ea01e3c9d4bf5627ed9b9e81d0e9034dbf974163a0402bd83674fbd0c7a3c7", "d2a049d6eb7d8cb0560268b6aabdc0461962811a92c3d733a5dc2d3127a7a774", "142fe7bf3e52ec881ec13c2495004d2a3a86084fe945663aa712e1cfbc000b8b", "fede1e02c3e0902ed5d74546da6c599d141e5d2348610df68f7d70a8fcaf66d2", "c9848ca040f31b822bd51e906a307acdfc0f720d96d5e1807e13f9dc3df7f22f"]
2023-07-03T15:53:15.610000Z TestFramework (INFO): The batch of 25 transactions has been accepted into the mempool.
2023-07-03T15:53:15.610000Z TestFramework.node0 (DEBUG): TestNode.generate() dispatches `generate` call to `generatetoaddress`
-4676-> generatetoaddress {"nblocks": 1, "address": "rbgl1qqpmtreykaj2jphmyf3358sn3wudm6thfypzzut", "maxtries": 1000000}
<-4676- [0.014817] ["46e1e2e652518ae03ce38471982047647369ba79534b86f209fa0b7350b5e891"]
-4677-> getbestblockhash {}
<-4677- [0.000486] "46e1e2e652518ae03ce38471982047647369ba79534b86f209fa0b7350b5e891"
-4678-> getrawmempool {}
<-4678- [0.000579] ["fede1e02c3e0902ed5d74546da6c599d141e5d2348610df68f7d70a8fcaf66d2", "c9848ca040f31b822bd51e906a307acdfc0f720d96d5e1807e13f9dc3df7f22f"] <--- this list should be empty

I need some more time to find a root cause and fix this.

Thanks for the update. I will also check it out.

The error we are witnessing may not have much to do with the branch, bitcoinsync2022. Bitgesell has less block weight than Bitcoin.
`
Block Weight [10 times smaller than Bitcoin]
<= 400,000
However, the below passed confirming the above statementself.transaction_graph_test(size=40, n_tx_to_mine=[10, 20, 30])`
I am going to do thorough checks.

Hi @janus ah this makes sense! I tried to go back in time and use bisection to get a commit that works but struggled with build errors on most commits, the ones I found still had an error. But I also made the test case to pass by asking generate to mine more blocks with this patch:
diff --git a/test/functional/mempool_updatefromblock.py b/test/functional/mempool_updatefromblock.py
index 6bae162539..0504e63939 100755
--- a/test/functional/mempool_updatefromblock.py
+++ b/test/functional/mempool_updatefromblock.py
@@ -18,6 +18,9 @@ from test_framework.key import ECKey

 class MempoolUpdateFromBlockTest(BGLTestFramework):
+    def add_options(self, parser):
+        self.add_wallet_options(parser)
+
     def set_test_params(self):
         self.num_nodes = 1
         self.extra_args = [['-limitdescendantsize=1000', '-limitancestorsize=1000', '-limitancestorcount=100']]
@@ -103,7 +106,7 @@ class MempoolUpdateFromBlockTest(BGLTestFramework):
             if tx_count in n_tx_to_mine:
                 # The created transactions are mined into blocks by batches.
                 self.log.info('The batch of {} transactions has been accepted into the mempool.'.format(len(self.nodes[0].getrawmempool())))
-                block_hash = self.generate(self.nodes[0], 1)[0]
+                block_hash = self.generate(self.nodes[0], 3)[0]
                 if not first_block_hash:
                     first_block_hash = block_hash
                 assert_equal(len(self.nodes[0].getrawmempool()), 0)

with the result
$ test/functional/mempool_updatefromblock.py
2023-07-05T15:01:55.693000Z TestFramework (INFO): Initializing test directory /tmp/BGL_func_test_gjjve9sl
2023-07-05T15:01:56.023000Z TestFramework (INFO): Creating 100 transactions...
2023-07-05T15:02:01.995000Z TestFramework (INFO): The batch of 25 transactions has been accepted into the mempool.
2023-07-05T15:02:02.014000Z TestFramework (INFO): All of the transactions from the current batch have been mined into a block.
2023-07-05T15:02:09.493000Z TestFramework (INFO): The batch of 25 transactions has been accepted into the mempool.
2023-07-05T15:02:09.506000Z TestFramework (INFO): All of the transactions from the current batch have been mined into a block.
2023-07-05T15:02:18.747000Z TestFramework (INFO): The batch of 25 transactions has been accepted into the mempool.
2023-07-05T15:02:18.767000Z TestFramework (INFO): All of the transactions from the current batch have been mined into a block.
2023-07-05T15:02:27.661000Z TestFramework (INFO): The last batch of 25 transactions has been accepted into the mempool.
2023-07-05T15:02:27.888000Z TestFramework (INFO): All of the recently mined transactions have been re-added into the mempool in 0.22525739669799805 seconds.
2023-07-05T15:02:27.888000Z TestFramework (INFO): Checking descendants/ancestors properties of all of the in-mempool transactions...
2023-07-05T15:02:28.057000Z TestFramework (INFO): Stopping nodes
2023-07-05T15:02:28.160000Z TestFramework (INFO): Cleaning up /tmp/BGL_func_test_gjjve9sl on exit
2023-07-05T15:02:28.160000Z TestFramework (INFO): Tests successful

perhaps we could use this one or your approach after you finish thorough checks.

Hi @janus you were absolutely right about 400 000 block limit for BGL, I added a new list in the test case which calculated transaction weight and logged the size:
@@ -59,6 +62,7 @@ class MempoolUpdateFromBlockTest(BGLTestFramework):
         first_block_hash = ''
         tx_id = []
         tx_size = []
+        tx_weight = []
         self.log.info('Creating {} transactions...'.format(size))
         for i in range(0, size):
             self.log.debug('Preparing transaction #{}...'.format(i))
@@ -99,7 +103,10 @@ class MempoolUpdateFromBlockTest(BGLTestFramework):
             signed_raw_tx = self.nodes[0].signrawtransactionwithkey(unsigned_raw_tx, self.priv_keys)
             tx_id.append(self.nodes[0].sendrawtransaction(signed_raw_tx['hex']))
             tx_size.append(self.nodes[0].getmempoolentry(tx_id[-1])['vsize'])
-
+            tx_weight.append(self.nodes[0].getmempoolentry(tx_id[-1])['weight'])
+            self.log.info(sum(tx_weight))

which gave:
$ test/functional/mempool_updatefromblock.py
2023-07-05T18:09:20.480000Z TestFramework (INFO): Initializing test directory /tmp/BGL_func_test_9rjttozh
2023-07-05T18:09:20.841000Z TestFramework (INFO): Creating 100 transactions...
2023-07-05T18:09:21.181000Z TestFramework (INFO): 13777
2023-07-05T18:09:21.528000Z TestFramework (INFO): 27733
2023-07-05T18:09:21.892000Z TestFramework (INFO): 42141
2023-07-05T18:09:22.240000Z TestFramework (INFO): 57001
2023-07-05T18:09:22.591000Z TestFramework (INFO): 72313
2023-07-05T18:09:22.951000Z TestFramework (INFO): 88077
2023-07-05T18:09:23.311000Z TestFramework (INFO): 104293
2023-07-05T18:09:23.670000Z TestFramework (INFO): 120961
2023-07-05T18:09:24.042000Z TestFramework (INFO): 138081
2023-07-05T18:09:24.411000Z TestFramework (INFO): 155653
2023-07-05T18:09:24.784000Z TestFramework (INFO): 173677
2023-07-05T18:09:25.161000Z TestFramework (INFO): 192153
2023-07-05T18:09:25.545000Z TestFramework (INFO): 211081
2023-07-05T18:09:25.945000Z TestFramework (INFO): 230461
2023-07-05T18:09:26.323000Z TestFramework (INFO): 250293
2023-07-05T18:09:26.707000Z TestFramework (INFO): 270577
2023-07-05T18:09:27.101000Z TestFramework (INFO): 291313
2023-07-05T18:09:27.489000Z TestFramework (INFO): 312501
2023-07-05T18:09:27.887000Z TestFramework (INFO): 334141
2023-07-05T18:09:28.248000Z TestFramework (INFO): 356233
2023-07-05T18:09:28.567000Z TestFramework (INFO): 378777
2023-07-05T18:09:28.866000Z TestFramework (INFO): 401773
2023-07-05T18:09:29.139000Z TestFramework (INFO): 425221
2023-07-05T18:09:29.424000Z TestFramework (INFO): 449121
2023-07-05T18:09:29.704000Z TestFramework (INFO): 473469
2023-07-05T18:09:29.706000Z TestFramework (INFO): The batch of 25 transactions has been accepted into the mempool.
2023-07-05T18:09:29.726000Z TestFramework (ERROR): Assertion failed

...

  File "/tests/bitgesell-2022/test/functional/test_framework/util.py", line 56, in assert_equal
    raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not(4 == 0)

In this case 4 transactions were over 400 000 limit, and were not mined. I will provide a fix for this test and push to this branch now.

I pushed a new commit, all functional mempool test cases are passing now
TEST                             | STATUS    | DURATION

mempool_accept.py                | ✓ Passed  | 1 s
mempool_accept_wtxid.py          | ✓ Passed  | 7 s
mempool_datacarrier.py           | ✓ Passed  | 1 s
mempool_expiry.py                | ✓ Passed  | 1 s
mempool_limit.py                 | ✓ Passed  | 4 s
mempool_package_limits.py        | ✓ Passed  | 2 s
mempool_package_onemore.py       | ✓ Passed  | 1 s
mempool_packages.py              | ✓ Passed  | 8 s
mempool_persist.py --descriptors | ✓ Passed  | 27 s
mempool_reorg.py                 | ✓ Passed  | 2 s
mempool_resurrect.py             | ✓ Passed  | 1 s
mempool_spend_coinbase.py        | ✓ Passed  | 1 s
mempool_unbroadcast.py           | ✓ Passed  | 9 s
mempool_updatefromblock.py       | ✓ Passed  | 42 s
mempool_compatibility.py         | ○ Skipped | 0 s

ALL                              | ✓ Passed  | 107 s (accumulated)
Runtime: 51 s

I pushed a new commit, all functional mempool test cases are passing now
TEST                             | STATUS    | DURATION

mempool_accept.py                | ✓ Passed  | 1 s
mempool_accept_wtxid.py          | ✓ Passed  | 7 s
mempool_datacarrier.py           | ✓ Passed  | 1 s
mempool_expiry.py                | ✓ Passed  | 1 s
mempool_limit.py                 | ✓ Passed  | 4 s
mempool_package_limits.py        | ✓ Passed  | 2 s
mempool_package_onemore.py       | ✓ Passed  | 1 s
mempool_packages.py              | ✓ Passed  | 8 s
mempool_persist.py --descriptors | ✓ Passed  | 27 s
mempool_reorg.py                 | ✓ Passed  | 2 s
mempool_resurrect.py             | ✓ Passed  | 1 s
mempool_spend_coinbase.py        | ✓ Passed  | 1 s
mempool_unbroadcast.py           | ✓ Passed  | 9 s
mempool_updatefromblock.py       | ✓ Passed  | 42 s
mempool_compatibility.py         | ○ Skipped | 0 s

ALL                              | ✓ Passed  | 107 s (accumulated)
Runtime: 51 s

Alright.


Manual Data Fixes is missing from the settings page

#version
#Fri Sep 07 06:08:29 UTC 2018
version.display=6.6-Stable.Edalex
version.mm=6.6
version.mmr=6.6.r11
version.commit=gb6c5a72
Manual data fixes is missing from the settings page.  It is still accessible by manually going to: .. access/manualdatafixes.do

Verified in 2018.2


Available transport is determined at connection time

:tada: This PR is included in version 5.3.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


feat: build backend assets with docker

Build frontend within docker and copy it to the backend

The idea is to add a docker compose or just build the frontend in a container ?

Just build the frontend in the container to remove the frontend dependencies for backend changes

Ok, not sure why we need docker compose though.

Just simplification, don’t need it but it’s available anyway

We can also remove it and use directly docker, as you like. But then we need to build the image first and then run it.

I was more or less thinking about something like this:
docker run --rm --entrypoint sh -v ${PWD}/frontend:/frontend -w /frontend node:20-alpine -c "npm install && npm run build"

If this works also fine for me

@eddycharly updated


fixFix the bug that does not stop when reading json files

Proposed changes
one_array_json.json ：  This file has only 10 lines .
before :
select * from HDFS(
"uri" = "hdfs://<IP_ADDRESS>:8120/user/doris/preinstalled_data/json_format_test/one_array_json.json",
 "fs.defaultFS"= "hdfs://<IP_ADDRESS>:8120",
"hadoop.username" = "doris",
 "format" = "json",
 "strip_outer_array" = "true",
 "read_json_by_line" = "false" ) limit 13;
+------+-----------+---------+
| id   | city      | code    |
+------+-----------+---------+
| 1    | beijing   | 1454547 |
| 2    | shanghai  | 1244264 |
| 3    | guangzhou | 528369  |
| 4    | shenzhen  | 594201  |
| 5    | hangzhou  | 594201  |
| 6    | nanjing   | 2345672 |
| 7    | wuhan     | 2345673 |
| 8    | chengdu   | 2345674 |
| 9    | xian      | 2345675 |
| 10   | hefei     | 2345676 |
| 1    | beijing   | 1454547 |
| 2    | shanghai  | 1244264 |
| 3    | guangzhou | 528369  |
+------+-----------+---------+
13 rows in set (0.13 sec)

when you select * from ...    , read json file will not stop
after :
select * from HDFS(
"uri" = "hdfs://<IP_ADDRESS>:8120/user/doris/preinstalled_data/json_format_test/one_array_json.json",                             "fs.defaultFS"= "hdfs://<IP_ADDRESS>:8120",
"hadoop.username" = "doris",
"format" = "json",
"strip_outer_array" = "true",
"read_json_by_line" = "false" ) limit 21;
+------+-----------+---------+
| id   | city      | code    |
+------+-----------+---------+
| 1    | beijing   | 1454547 |
| 2    | shanghai  | 1244264 |
| 3    | guangzhou | 528369  |
| 4    | shenzhen  | 594201  |
| 5    | hangzhou  | 594201  |
| 6    | nanjing   | 2345672 |
| 7    | wuhan     | 2345673 |
| 8    | chengdu   | 2345674 |
| 9    | xian      | 2345675 |
| 10   | hefei     | 2345676 |
+------+-----------+---------+
10 rows in set (0.03 sec)

Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.49 seconds
stream load tsv:          538 seconds loaded<PHONE_NUMBER>9 Bytes, about 132 MB/s
stream load json:         21 seconds loaded<PHONE_NUMBER> Bytes, about 107 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select:          29.3 seconds inserted 10000000 Rows, about 341K ops/s
storage size:<PHONE_NUMBER>3 Bytes

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.71 seconds
stream load tsv:          541 seconds loaded<PHONE_NUMBER>9 Bytes, about 131 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size:<PHONE_NUMBER>4 Bytes