Disable/exclude packaging requirements for nodejs functions

Similar to #29, and specifically as mentioned by @trecouvr in https://github.com/UnitedIncome/serverless-python-requirements/issues/29#issuecomment-329426474 it'd be good to disable this plugin for nodejs functions that are sitting next to python functions so the python requirements don't bloat the nodejs functions
Made this separate issue because it's a little bit of a different use case, and hopefully easier to implement? (feel free to close it if it'd just be done with #29 though)

Am I right to assume this is for when using the package.individually=true option? If so, #117 will fix this!

Cool! 👍 Wasn't using package.individually=true yet in case i didn't need to, but I think that's a decent approach, thanks 😄

Awesome. I'll close this when I merge and release that feature.


Agenda table

@emanuelschuetze csv added and internal items are now working. As well as removing inline edit for title and set number removed.

All done. Last open remark moved to #2728. Ready for merge...


Support for external open with

Phiola as an audio player seems to work fine with Internal file picker.
Can you also include a feature so that we can play recorded audio files through phiola from external file managers ?

Hi!  I've just checked on my phone, and indeed, phiola is not displayed in the list of apps for playing an audio file.  I'll see what I can do.


'parameter.description' vs. 'parameter.schema.description'

I had defined a parameter with its description in the schema field, like so:
parameters:
  Page:
    in: query
    name: page
    schema:
      description: A page number within the paginated result set.
      title: Page
      type: integer

This seems to be valid for the Schema Object [1]. However, I note that the Parameter object itself also has a description field [2].
parameters:
  Page:
    in: query
    name: page
    description: A page number within the paginated result set.
    schema:
      title: Page
      type: integer

Is this duplication and, if so, what is intended to take priority?

I wouldn't call this duplication as neither field is mandatory. The parameter description describes the use of the parameter and schema description (which may be reused or referenced from elsewhere) covers the shape / serialization of the parameter.


[backport] Make names of test classes start with Test

Backport of #4320

[automatic post] Jenkins, test this please.

Jenkins CI test (for commit 0217d0927197738a792c0780265c15190ece6ecc, target branch v8) failed with status FAILURE.

Jenkins, test this please.

Jenkins CI test (for commit 0217d0927197738a792c0780265c15190ece6ecc, target branch v8) succeeded!


random UTF-8-MAC failures

Following up on this thread (https://github.com/conda-forge/git-feedstock/issues/50).
We should either package a patch from this project
https://github.com/fumiyas/libiconv-utf8mac
or build this project as the libiconv on osx.

cc @conda-forge/core any thoughts here?

Has this been reported upstream? If so, are there patches upstream that could be used?

Yes please read the associated issue in the GitHub feedstock. The upstream devs refuse to incorporate the apple encoding.

see https://github.com/conda-forge/git-feedstock/issues/50#issuecomment-624852740
but I don't really understand what UTF-8"-MAC" is? isn't UTF-8 a deterministic encoding of strings? what changes with UTF-8"-MAC"?

IDK what changes either, but they are different.

isn't UTF-8 a deterministic encoding of strings?

The same Unicode string can be represented in different ways.
There different normalizations: https://en.wikipedia.org/wiki/Unicode_normalisation#Normalization

It's always a good thing to look out for what other distributors to.
For Linux I usually refer to Fedora/Debian/Arch's work.
For macOS it's usually Homebrew.
Homebrew has https://github.com/Homebrew/homebrew-core/blob/9911c3276bcc2a8f2e8a8359370c217e43cd8104/Formula/lib/libiconv.rb#L26 which is https://github.com/Homebrew/formula-patches/blob/9be2793af22750a2d1d053af0a9e2257d4ce7f4e/libiconv/patch-utf8mac.diff from 2015 -- i.e., a very low-maintenance patch.
I diff'ed https://github.com/fumiyas/libiconv-utf8mac against upstream and it is essentially the exact same patch.
It's probably reasonable for us to just apply that patch here on osx-* targets.
(N.B.: I'm not a Mac dev or user, so this is only an "outsider" assessment.)

I think carrying that patch would be fine.

Yeah patching the existing package would be my preference as well


EthStorage Devs Meeting #108 Agenda

Meeting Info

Dec 17, 2024, 11:00 UTC+8
Duration: 60 minutes
Meeting link
Anyone can comment on the agenda where they would like this to be discussed.

Agenda

Protocol/App/Research progress
BD/Marketing progress

Goal:

EthStorage Mainnet Ready

Updates:

EigenDA investigation: https://docs.google.com/presentation/d/1e2xeE7YdU58K28vSspdiFyBJbJEkrvxB65Z_kZgVdXo/edit?usp=sharing
CircleCI:

Filled a ticket for the permission issue
Resolve conflict: https://github.com/ethstorage/optimism/commit/e619d04edff02f85303a007e4b0a5460e961f465

Filled the EthStorage OP grant
Code reviews

TODO:

Follow up EigenDA's discussion
Nand's PR review: https://github.com/ethstorage/web3url-gateway/pull/42
https://github.com/ethstorage/pm/issues/149

Goal:

SWC Testenet

Updates:

Promote Long-term DA https://x.com/qc_qizhou/status/1867733927953412325
Devnet supports dac server https://github.com/ethstorage/optimism/pull/124
Discuss on scalar https://github.com/ethstorage/optimism/issues/93
Go through devnet and run a simple airdrop test based on web3.py

TODO:

https://github.com/ethstorage/pm/issues/149

Goal:

Prepare WSC L2 testnet launch

Updates:

merge upstream:

https://github.com/ethstorage/optimism/pull/130
https://github.com/ethstorage/op-geth/pull/21

review:

https://github.com/ethstorage/optimism/pull/125
https://github.com/ethstorage/optimism/pull/126
https://github.com/ethstorage/optimism/pull/127
https://github.com/ethstorage/optimism/pull/128
https://github.com/ethstorage/optimism/pull/129

TODO:

deploy beta testnet
https://github.com/ethstorage/optimism/issues/103
https://github.com/ethstorage/pm/issues/149

Walrus: talk about walrus https://docs.google.com/presentation/d/1-z3uxELxOEiEEEQmwbUSBReyytH0FwBuCwr07onG2Uw/edit#slide=id.g31ec906a383_0_0

WSC L2

https://github.com/ethstorage/optimism/issues/102
Faucet: Done some preparation and waiting for the Beta network to be ready and replace the Alpha faucet UI. The

EthStorage x Taiko:

[x] investigation: read docs and go through the taiko codes (Merge the latest code)
[ ] integration test
[ ] Run on Holesky chain

Add the readme to the qkc airdrop script and upload the address set to GitHub.
https://github.com/ethstorage/quarkchain-airdrop
Deploy a swap website on the qkc network.

TODO:

deploy uniswap?
Devnet does not support gas tokens yet. It will be migrated to devnet after it is supported.

CircleCI

https://github.com/ethstorage/optimism/pull/129
https://github.com/ethstorage/optimism/pull/128
https://github.com/ethstorage/optimism/pull/127
https://github.com/ethstorage/optimism/pull/126

New Errors:
https://app.circleci.com/pipelines/circleci/VJoJbnHRXJiFR13mWFvRfZ/SHyYDZiLFYMtveJSZVFKbw/60/workflows/758669d0-9f4d-4c0e-862f-3c4c0eb194e5
TODO

merge code
fix comments

Gateway

https://github.com/ethstorage/web3url-gateway/pull/42

Goal:

Prepare WSC L2 testnet launch

Updates

CircleCI

https://github.com/ethstorage/optimism/issues/113

super world computer launch and marketing campaign
taiwan blockchain week


Automerge failure: "found removed line"

I'm submitting a ...
[v] bug report
[ ] feature request
[ ] question about the decisions made in the repository
[ ] question about how to use this project

Summary
Decimal of my token is 8. and I want to update decimal to 9, but automerger always throw error with "found removed line: - "decimals": 8".
Is there a way to fix this error? or just can't update fields for token information created once?

Other information (e.g. detailed explanation, stack traces, related issues, suggestions how to fix, links for us to have context, eg. StackOverflow, personal fork, etc.)

Pull Request: https://github.com/solana-labs/token-list/pull/18517

merged


Minor capitalization and spelling corrections

Thank you for putting this excellent resource together!

Thank's for the contribution!

My pleasure, thank you again for the resource, very useful!
Sent from my iPhone

On Sep 19, 2019, at 2:15 AM, Adrien Torris<EMAIL_ADDRESS>wrote:
Thank's for the contribution!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.


Provide PHPUnit class aliases to other projects implementing zend-test

Since the file autoload/phpunit-class-aliases.php is stored within autoload-dev, no remote project will ever get those aliases which will cause to problems if they're using PHPUnit in versions less than 6.0.
We should provide those aliases to them from within autoload so there is no need in remote projects to manually add those aliases themselves or by using a creepy composer configuration like:
"autoload-dev": {
    "files": [
        "vendor/zendframework/zend-test/autoload/phpunit-class-aliases.php"
    ]
}

This is something that should be proposed to PHPUnit upstream, or else it
forces more maintenance burden on any library relying on it.
On 21 May 2017 14:56, "Max"<EMAIL_ADDRESS>wrote:

Since the file autoload/phpunit-class-aliases.php is stored within
autoload-dev, no remote project will ever get those aliases which will
cause to problems if they're using PHPUnit in versions less than 6.0.
We should provide those aliases to them from within autoload so there is
no need in remote projects to manually add those aliases themselves or by
using a creepy composer configuration like:
"autoload-dev": {    "files": [        "vendor/zendframework/zend-test/autoload/phpunit-class-aliases.php"    ]}

You can view, comment on, or merge this pull request online at:
https://github.com/zendframework/zend-test/pull/49
Commit Summary

Provide PHPUnit class aliases to other projects implementing
zend-test

File Changes

M composer.json
https://github.com/zendframework/zend-test/pull/49/files#diff-0 (10)

Patch Links:

https://github.com/zendframework/zend-test/pull/49.patch
https://github.com/zendframework/zend-test/pull/49.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/zendframework/zend-test/pull/49, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAJakArjNYJ7LCzXsxkvMJtp4RD1zhelks5r8DSNgaJpZM4Nhl3i
.

@Ocramius Maybe, but this library is allowing PHPUnit less than 6. I dont think that PHPUnit should handle this. PHPUnit is not responsible for libraries which allow multiple, BC broken versions.
This library whether should change its dependencies to a version which does not need those aliases or it has to provide proper autoloading which can be used in all projects relying on this one.

To anyone needing this change, you can install it here:
composer require illchuk/phpunit-class-aliases

@dillchuk thats definitely a useful package but I am still thinking that this package might provide those aliases since it provides testing abilities to other projects.
Therefore, this project is supposed to be "required" as a development package and could therefore just use your package or create an own alias map so no other project which is using zend-test have to bother with that error.

Merging it because aliases are required for zend-test itself to work with older phpunit, not for tests using zend-test


wip: sketch out a recommended release process

Status
READY/IN DEVELOPMENT/HOLD
Description

I think we can close since we merged #296


Add coverage report and badge

Solves #18
Adds two new scripts: test:coverage and test:badge. test:badge - generates a coverage badge and saves it as assets/badge.svg
Adds the generated badge to README.md
Adds test:badge script into github workflow to generate a new badge

Thanks for contributing!! I have a few small things, but overall it looks great.

Thanks for the feedback! I did the changes, please take another look.

Merging now. 🚀
Thanks again!


[fix][broker] Broker incorrectly validates proxy role for http request

Motivation
When Proxy and appropriate Proxy authentication is introduced in the Broker, the Proxy first authenticates the request coming from the client and retrieves its principal name, and passes it to the broker as an original-principal name along with its certificates so, the broker can authenticate the proxy with the proxy's certs and broker can also get an original-principal name in the same Connect request. Broker performs the authorization on the original-principal name and validates that proxy role exists into ServiceConfigiration::proxyRoles. With this auth model, the client can manage its principal-name into authorization policies and still can access broker API via proxy without any extra policy management and it works seamlessly.
However, the above auth model was implemented correctly for binary protocol but it was broken in HTTP authentication which is used for HTTP admin requests. In case of HTTP reverse proxy authentication, Broker performs authorization for both proxy's principal name and original principal name and both principal names must be present in namespace authorization policy. It means if client wants to access Pulsar-Broker via proxy then proxy's role must be added into namespace policies and if proxy server's certs are compromised then client data can be in risk. This behavior is incorrect, insecure and not compatible with existing binary authentication.
Broker should perform policy-authorization on original principal and proxy authorization should be done via proxy-roles provided into broker's configurations.
Modifications
This PR fixes the security risk of http-proxy authentication and makes it compatible with binary protocol.
Verifying this change

[ ] Make sure that the change passes the CI checks.

(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

[ ] Dependencies (add or upgrade a dependency)
[ ] The public API
[ ] The schema
[ ] The default values of configurations
[ ] The threading model
[ ] The binary protocol
[ ] The REST endpoints
[ ] The admin CLI options
[ ] The metrics
[ ] Anything that affects deployment

Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Matching PR in forked repository
PR in forked repository:

Broker performs authorization for both proxy's principal name and original principal name and both principal names must be present in namespace authorization policy.

When using a proxy, the broker must check the proxy's principal name and original principal name, regardless of the protocol you are using.

It means if client wants to access Pulsar-Broker via proxy then proxy's role must be added into namespace policies and if proxy server's certs are compromised then client data can be in risk. This behavior is incorrect, insecure and not compatible with existing binary authentication.

Do you mean that the client uses the proxy's principal name?

In the binary protocol, the broker doesn't allow the proxy's principal name and original principal name to be the same(https://github.com/apache/pulsar/pull/19455).
In the HTTP protocol, the broker allows the proxy's principal name and original principal name to be the same( https://github.com/apache/pulsar/pull/19557).

I forgot why I approved to #19557, which maybe introduces the security risk on http-proxy authorization.
This PR changes the authorization logic, right now you only check the original principal name, and the proxy's principal name was ignored.

When using a proxy, the broker must check the proxy's principal name and original principal name, regardless of the protocol you are using.

Of course, yes but proxy role must not be part of the namespace policy authorization and client doesn't have to explicitly grant permission to proxy-role. Broker checks proxy role by checking authenticating proxy request and proxy's principal name must be part of ServiceConfiguration::proxyRoles. Binary Protocol does it correctly because that's we have introduced when it first implemented proxy and authentication. but later on someone implemented this model incorrectly for HTTP requests.

When using a proxy, the broker must check the proxy's principal name and original principal name, regardless of the protocol you are using.

Of course, yes but proxy role must not be part of the namespace policy authorization and client doesn't have to explicitly grant permission to proxy-role. Broker checks proxy role by checking authenticating proxy request and proxy's principal name must be part of ServiceConfiguration::proxyRoles. Binary Protocol does it correctly because that's we have introduced when it first implemented proxy and authentication. but later on someone implemented this model incorrectly for HTTP requests.

agree, could you help add a test for this ?

Sounds good, but if someone uses the proxy to control access, this will break the user behavior.
user1->proxy1-
                           \
                            topic-1
                           /
user2->proxy2-

the pulsar manager uses multiple proxies to control access, for example: proxy1 can produce, but not consume, and proxy2 can consume, but not produce.

the pulsar manager uses multiple proxies to control access, for example: proxy1 can produce, but not consume, and proxy2 can consume, but not produce.

How does it matter? ServiceConfiguration::proxyRoles is a list of proxy roles and broker can have multiple proxy roles to allow access.

If you want the proxy to be a super user and want to functionally skip the permission check, just add it to the super user list.

No, that is incorrect, we are not proposing proxy-roles to be super-user. Proxy role has to be authenticated at Broker first and also bring the original principal role which must pass the authorization. Super user role doesn't have to authenticate client and doesn't have to pass original principal as well.
Here, fundamental thing is broken where client has to explicitly authorize proxy role to the namespace policy which is not secured, transparent to users and also not compatible behavior with existing binary protocol.
HTTP can't have different behavior where client has to grant permission to proxy roles as well.
Proxy is optional and must be abstract to users and users don't have to know it explicitily.

Regarding the change itself, it was introduced in https://github.com/apache/pulsar/pull/7788.

I don't see any issue with this PR because super-user can also go via proxy and that should be taken care by above PR.

. We verify both the proxy and the original role permission level on the binary protocol here:
https://github.com/apache/pulsar/blob/53df683b0f78f5f7c12f87e6fbb4d73637ca5bd5/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java#L468-L494

No, that is incorrect as well. you can check the code and you will see that
isProxyAuthorizedFuture = service.getAuthorizationService().allowTopicOperationAsync performs authorization on originalPrincipal and ProxyRole gets validated by checking if it's in proxyRole here
https://github.com/apache/pulsar/blob/branch-2.10/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java#L667

. We verify both the proxy and the original role permission level on the binary protocol here:
https://github.com/apache/pulsar/blob/53df683b0f78f5f7c12f87e6fbb4d73637ca5bd5/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java#L468-L494

No, that is incorrect as well. you can check the code and you will see that isProxyAuthorizedFuture = service.getAuthorizationService().allowTopicOperationAsync performs authorization on originalPrincipal and ProxyRole gets validated by checking if it's in proxyRole here
https://github.com/apache/pulsar/blob/branch-2.10/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java#L667

The names of the variables appear inverted, but the logic confirms that both the originalPrincipal and the authRole have permission to perform the action. The futures are chained together and the call is only authorized if both futures return true.

How does it matter? ServiceConfiguration::proxyRoles is a list of proxy roles and broker can have multiple proxy roles to allow access.

ServiceConfiguration::proxyRoles checks if the authentication principal is a proxy role. If you want to allow the proxy's principal to do anything, you can add the proxy's principal to ServiceConfiguration::superUserRoles.
Proxy cases:

Proxy case1 - one proxy
Usually, we add the proxy's principal to the super user list and proxy role list. When do authorization, the broker will check whether both the proxy's principal(superuser) and the original principal have been authorized.

Proxy case2 - multiple proxy
Use multiple proxy to do fine-grained permission control. Assume, there are proxy-a and proxy-b proxies, and which has been added to the proxy role list, and not exits in super user list. We need to grant the permission for the proxy-a and proxy-b, otherwise, it is not allowed to pass.

Your PR breaks the case2, and introduces the security risk:

If I understand it correctly, it implicitly elevates all proxy roles to super user privilege level. Users can currently configure proxy roles to lesser privileges, so that would present a problem for those users.


CO-OPS AWS development server

@ZacharyWills @KatherinePowell-NOAA
We received requirements from CO-OPS regarding needs for a development space. I want to ask your advice about setting up a space or possibly even a separate head node for them.
What do you think? Can we make something like this work in the near-ish future?
Problem Statement
CO-OPS currently has two AWS servers for OFS applications; ofs-east1 is an operational server to generate all OFS operational products; ofs-work-east1 is presently used as a developmental server for software and product development and testing, and to create real-time products for all OFS which are run in a semi-operational mode for development and testing. Therefore, ofs-work-east1 is mainly used as a parallel/staging server. There is an urgent need to build a new developmental AWS server that the modeling team can use for OFS software and product development and testing.
This document contains requirements for ofs-dev-east1 to support CO-OPS development work. Any requests made herein to increase license count, CPU, etc., will need to be reviewed and/or approved and will impact the system's delivery date.
Requirements
Hardware requirements (Same hardware as ofs-work-east1, number of CPUs, memory, etc.)
Modeling team (9 modelers) heavily relies on ofs-dev
OS Change
For awareness the OS will change from RHEL 7 which is running on ofs-work-east1 and ofs-east1 to RHEL 8
Disk space
5TB EBS for /opt/archive (software, and semi-operational OFS under development)
8 TB EBS mount as /opt/ofsdev/users - used by modelers for R&D related projects, such as testing, developing, skill assessment etc.
+space exclusive to the SCI/SA project
What models will run on this server? Only OFS under testing is run on dev server, no semi-operational OFS runs on dev using cron jobs.
Mount /opt/ofs-bdp and /opt/ofs-all S3 buckets for data access (READ ONLY)
Other NOAA s3 bucket of data sources (e.g. satellite data, NCEI archives, etc.) might be needed to be mounted on. It can be figured out as needed in the future.
1TB /cache drive
This is required for the requested s3fs mounts
List of Software and Compilers
Matlab (1 concurrent license)
IDL (1 license)
ifort Fortran compiler
Icc C compiler
Totalview or similar debugger
Eclipse software development kit
Java
NCL command
Emacs
GIT ()
Anaconda with Python 3.9 or newer, Jupyter Notebook & Library
NCO NetCDF operators
Latest version NetCDF C and Fortran
NetCDF commands (e.g. ncdump)
Xmgrace
xmgredit5
Special Needs: commands saved in /usr/bin, nedit, gedit, gthumb
Possibly Allow CO-OPS external users to access the dev server
co-development with partners critical for closing the O2R cycle
OFS Data Processing System (SCI/SA) Requirements
Compute
Storage/Data:
Local disk storage (DEV possibly 1TB - it is covered by 8 TB under /opt/ofsdev/users )
Need 2 years of historical OFS data.
Historical OFS data at the NODD arn:aws:s3:::noaa-nos-ofs-pds
/opt/ofs-bdp will be mounted for READ ONLY
Note the SCI/SA project requires NODD S3 access to various data sets, including STOFS-2D-Global, STOFS-3DAtlantic, GFS, HRRR, etc.)
Software:
Apache Tomcat
Python 3 (3.9 or newer)
Anaconda and Pipenv

@cpaternostro Tagging so you can track too.

I sent an email to<EMAIL_ADDRESS>to ask for accounts to be created with the project as IOOS and Arnold Mendez as the POC and they WOC helpdesk created a ticket. Here is the ticket creation and request email:
Thank you for contacting the National Service Desk (NSDesk). Your ticket has been logged in our system and a technician will begin working on it shortly.
Some details of the ticket are included below:
End User:	Michael Lalime
Short description:	Enterprise Hosted Services - Web Services
State:	Open
Priority:	Low
Assignment group:	OCIO NEWS Support
Description:	Call Record Short Description:
Request for accounts creation
Call Record Description:
Received by: woc@noaa.gov,arnold.mendez@noaa.gov,tiffany.c.vance@noaa.gov,aijun.zhang<EMAIL_ADDRESS>received from<EMAIL_ADDRESS>Hello,
I need to get some accounts set up in the cloud sandbox IOOS is building in
the NOAA AWS cloud environment.
The project is IOOS
The main POC on the WOC engineering side is: Arnold Mendez
We need to create accounts for the following people:
Aijun Zhang
Andrew Kammerer
Cristina Urizar
Ilya Rivin
Jiangtao Xu
Lianyuan zheng
Ling Liu
Lixia Wang
Lucila houttuijn bloemendaal
Machuan Peng
Thank you,
Michael
--
Michael Lalime
Supervisory Technical Analyst
Contractor with Integrated Systems Solutions Inc.
in support of NOAA/NOS/IOOS
National Ocean Service, NOAA
1315 East West Highway, Rm. 2610
Silver Spring, MD 20910
Work:<PHONE_NUMBER>
Due date:	2024-04-04 02:43:28 EDT
Click here to view the Requested Item: RITM0290860
Thank you for using the NSDesk.
NSDesk – One NOAA. One Service.
1-855-NSDesk1 (1-855-673-3751)
<EMAIL_ADDRESS>This message has been generated by the National Service Desk (NSDesk) ticketing system.
Ref:MSG11142439

I think some further discussion about CO-OPS needs/goals would be helpful.  These requirements, for me at least, bring up the question of scope of the Sandbox, because I think these are broader than what we initially planned for.
It may be that some aspects belong in the Sandbox, whereas others do not.
Also, I don't see in these two descriptions the need to actually run ocean models, which is our primary mission:

ofs-east1 is an operational server to generate all OFS operational products;
ofs-work-east1 is presently used as a developmental server for software and product development and testing, and to create real-time products for all OFS which are run in a semi-operational mode for development and testing

I don't know how best to have this conversation.  Perhaps we can do it here in this GitHub issue.  I think once we can get a feel on the scope question, it may make sense to break these requirements down into more manageable pieces to implement the parts we feel belong in the Sandbox.


chore: Update user data docs to correct mistake of showing bootstrap_extra_args when prepending user data (default AMI)

Description

update user data docs to correct mistake of showing bootstrap_extra_args when prepending user data (default AMI)

Motivation and Context

Closes #1770

Breaking Changes

No

How Has This Been Tested?

[x] I have tested and validated these changes using one or more of the provided examples/* projects

user data and eks managed node group examples

This PR is included in version 18.1.0 :tada:


[Question] How to produce dataset with millions of data in batches

Hi, we love using hollow, it is very nice.
I wanna know if there is a properly  way to produce data in batches? Like I have 10 million objects to produce, I wanna produce them divided into 10 parts and produce 1 million objects every time. I need to produce data in batches because my vm does not have enough memory to store 10 million objects.
I am using Incremental and withNumStatesBetweenSnapshots to make it publish snapshot only at begining and at last so that it run like "in batches". But I met a problem that sometimes the Incremental did not publish dataset because some batch do not change the dataset.
I have fork hollow-reference-implementation and make 2 test cases to show what we are looking for. You can check my test cases: ProducerTest

@Q-Bug4  Were you able to find a solution?

@prasoon16081994 @Q-Bug4 Were you guys able to find any solution for this.

@prasoon16081994 @shyam4u Not yet.
Now we are still using Incremental to get close to it. The "dataset not changed" issue can be avoided if you check the version at each producing time. Example below:
List<Data> datas = getMillionDatas();
Incremental incremental = getIncremental();

long lastVersion = 0;

for (...) {
    let version = incremental.produce(...);
    // check if version changed, if not, means dataset not changed.
    if (version == lastVersion ) {
        throw new RuntimeException("dataset not changed!");
    }
    lastVersion = version;
}

@Q-Bug4
Hey, finally developed a halfway decent understanding of hollow.
Isn't is the correct implementation to not write a delta if the dataset didn't change?
Perhaps if each of the records were needed, there could be some field that could be added to the record that is unique across all records, and marked as the primary key.


Tolerate leading v in version string

espressif's rustc started including a v in front of the version number with 1.74, like rustc 1.74.0-nightly (bd6bb31df 2023-11-15) (v<IP_ADDRESS>)

Nothing is supposed to use portable-atomic at this point 😭

cc: @georgik was this an intended change?

executor is pinned, CI is green now.

@bugadani Which OS? We needed to build some targets manually, so v might sneaked in.
I guess we can rebuild affected toolchains on Monday.

Windows for me. I actually don't think this is a big deal, the version string already doesn't conform to semver anyway, and now with the workaround I'm fine either way.

Ok, thank you for the confirmation. Based on other issue we will consider rebuild on Monday.


Video Demo

Make a video demonstrating the finished project and upload it to Youtube.

...We aren't getting this video demo done.


VR manager

fixes #379
fixes #234

SonarQube analysis reported 10 issues

 10 minor

Watch the comments in this conversation to review them.
9 extra issues
Note: The following issues were found on lines that were not modified in the pull request. Because these issues can't be reported as line comments, they are summarized here:

 AncientFungus:LGEPR: Cppcheck cannot find all the include files (use --check-config for details)
 RenderManager.cpp#L281: Variable 'name' is assigned a value that is never used.
 RenderManager.cpp#L291: The function 'GetAnimations' is never used.
 RenderManager.cpp#L309: The function 'GetDirectionalLights' is never used.
 RenderManager.cpp#L348: The function 'GetMaterials' is never used.
 RenderManager.cpp#L366: The function 'GetMeshes' is never used.
 RenderManager.cpp#L386: The function 'GetPointLights' is never used.
 RenderManager.cpp#L407: The function 'GetSpotLights' is never used.
 FrameBuffer.cpp#L10: Variable 'textures' is assigned in constructor body. Consider performing initialization in initialization list.

SonarQube analysis reported 10 issues

 10 minor

Watch the comments in this conversation to review them.
9 extra issues
Note: The following issues were found on lines that were not modified in the pull request. Because these issues can't be reported as line comments, they are summarized here:

 AncientFungus:LGEPR: Cppcheck cannot find all the include files (use --check-config for details)
 RenderManager.cpp#L281: Variable 'name' is assigned a value that is never used.
 RenderManager.cpp#L291: The function 'GetAnimations' is never used.
 RenderManager.cpp#L309: The function 'GetDirectionalLights' is never used.
 RenderManager.cpp#L348: The function 'GetMaterials' is never used.
 RenderManager.cpp#L366: The function 'GetMeshes' is never used.
 RenderManager.cpp#L386: The function 'GetPointLights' is never used.
 RenderManager.cpp#L407: The function 'GetSpotLights' is never used.
 FrameBuffer.cpp#L10: Variable 'textures' is assigned in constructor body. Consider performing initialization in initialization list.

SonarQube analysis reported 10 issues

 10 minor

Watch the comments in this conversation to review them.
9 extra issues
Note: The following issues were found on lines that were not modified in the pull request. Because these issues can't be reported as line comments, they are summarized here:

 AncientFungus:LGEPR: Cppcheck cannot find all the include files (use --check-config for details)
 RenderManager.cpp#L281: Variable 'name' is assigned a value that is never used.
 RenderManager.cpp#L291: The function 'GetAnimations' is never used.
 RenderManager.cpp#L309: The function 'GetDirectionalLights' is never used.
 RenderManager.cpp#L348: The function 'GetMaterials' is never used.
 RenderManager.cpp#L366: The function 'GetMeshes' is never used.
 RenderManager.cpp#L386: The function 'GetPointLights' is never used.
 RenderManager.cpp#L407: The function 'GetSpotLights' is never used.
 FrameBuffer.cpp#L10: Variable 'textures' is assigned in constructor body. Consider performing initialization in initialization list.


Update demo_maps vextex params

Related to this: https://github.com/open-rmf/rmf_traffic_editor/pull/351. Added

is_cleaning_zone property: cleaning waypoint for Clean
pickup_dispenserproperty: pickup waypoint for Delivery
dropoff_ingestorproperty: dropoff waypoint for Delivery

Also, cleaned up  rmf_demos_dashboard_resources

@vallq , let me know if the minor changes to the folder structure and addition to rmf_demos_dashboard_resources make sense. thanks

Hello @tanyouliang95, yep the folder structure is cleaner now! Thank you


Syntax error: word unexpected (expecting ")")

Hi,
Im trying to compile project, but I got this following error :
make all CROSS_COMPILE="/home/pi/gcc-linaro-aarch64-elf/bin/aarch64-elf-"

Syntax error: word unexpected (expecting ")")
make: *** [Makefile:77: out/main.o] Error 2

on Makefile 77's line is :
$(CC) $(CC_FLAGS) -c $< -o $@
Im using RasperryPi
Any suggestions are welcome.
Thank you.

I was using linaro x64 toolchain, therefore it wasnt compile. Switching rasp to x64, it compiles.

Hi,
Glad to see you were able to resolve this yourself. Out of interest, which version of the Linaro toolchain were you using when you faced the build error?
I can successfully build using aarch64-elf-gcc (Linaro GCC 7.5-2019.12) 7.5.0 directly from Linaro and using aarch64-linux-gnu-gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0 from the Ubuntu 18.04 package repos.
Were you using an older version?


[suggestion] gem socketing

i would love to see the sigil of advanced socketing to be added back. was there a reason to remove it on the latest version? i realy love this mod and thx for the continued work on it

It is planned to be replaced with something a little less low-effort.  In theory, anyone could make their own item which acts as a sigil of socketing with a different cap, just by adding another recipe - https://github.com/Shadows-of-Fire/Apotheosis/blob/1.20/src/main/resources/data/apotheosis/recipes/add_sockets.json


Type error using MuiTelInputProps after updating from MUI v6.0.2 to v6.1.0

Hello, as the title say, here is the reproduction.

@viclafouch Can you take a look at this please? This issue is preventing me from updating MUI in our codebase :/
Thank you.

Hello ! Fixed in https://github.com/viclafouch/mui-tel-input/releases/tag/v6.0.1 !

Thank you


2 support for loading model

Adds support for

[x] loading the model
[ ] loading model activations from artifacts

Welcome to Codecov :tada:
Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.
Thanks for integrating Codecov - We've got you covered :open_umbrella:


Freeing an SSL BIO can result in "shutdown while in init" error

Freeing an SSL BIO should check whether we are still in the init stage before calling SSL_shutdown(), otherwise we will get a stale error on the error queue.

Yes, and as we know SSL_shutdown can trap if previously an API function returned
an SSL_ERROR_SSL

Not a 1.1.1 specific issue so removing that milestone.


🔧 Changed proper escape sequence for TravisCI version number

Codecov Report

Merging #34 into master will not change coverage.
The diff coverage is n/a.


generalised power comment and corrected metering includes

Signed-off-by: edward.ross<EMAIL_ADDRESS>

@KoviaX Hi Hugo - this is a fix from your comment on my original metering pull request.

Looks good to me! Thanks for another contribution!
Normally I would expect the addition of the changelog file to be a seperate PR, but since this is so small, I'm fine with it this way.

@KoviaX Thanks for reviewing Hugo - I've noted your comment about the separate PR for the changelog.


witmotion_ros: 1.2.27-4 in 'melodic/distribution.yaml' [bloom]

Increasing version of package(s) in repository witmotion_ros to 1.2.27-4:

upstream repository: https://github.com/ElettraSciComp/witmotion_IMU_ros.git
release repository: https://github.com/twdragon/witmotion_ros-release.git
distro file: melodic/distribution.yaml
bloom version: 0.11.2
previous version for package: 1.2.27-2

witmotion_ros
* Merged pull request #12 <https://github.com/ElettraSciComp/witmotion_IMU_ros/issues/12> from fllay/main
  Migration to ROS2 made by @fllay approved. The information about the existence of the ROS2 branch will be added to README.md
* Added ROS2 branch information to README
* ros2 code
  Contributors: Andrei Vukolov, Andrey Vukolov, fllay
* Update .gitmodules
  Updated URLs to proper HTTPS
* Rename the project to witmotion_ros - cancelled
* fixed segmentation fault when Ctrl-C
* fixed polling interval and threading
* Fix link error
* package.xml version bump

Did you intend to release 1.2.28 that I see came out 2 days ago: https://github.com/ElettraSciComp/witmotion_IMU_ros/commit/9f8429035e6ca61b7e6c884bae0ef3cdd55c1caa?
This is a rerelease of 1.2.27


Reduction of concepts: NamedFluidDataStoreRegistryEntries

The following helper classes are added:
FluidDataStoreRegistry class - can wrap 3 forms of input into IFluidDataStoreRegistry - accepts Iterable<> of:

[name, Promise<factory>] - this is NamedFluidDataStoreRegistryEntries
[name, factory]
factory

MultipleDataStoreRegistries can be used to combine  multiple IFluidDataStoreRegistry objects into one.
Majority of NamedFluidDataStoreRegistryEntries usage is changed to IFluidDataStoreRegistry.
Other than small usage in tests, it's mostly DataObject that is left as is for now.
If there is no objection to such changes, then next step would be to fully eliminate NamedFluidDataStoreRegistryEntries concept from repo.

So this is similar to how it used to be. This created a couple of problems, consumer doing bad things in the passed in registry, and consumers not understanding the passed in registry isn't the actual registry that get's used, as it is wrapped. there was also some problems with dynamic registries/sub-registries that the wrapping pattern create problems for as well. might want to see if this change would cause problems in kampa.
4193 is also related. this would hopefully simplify the interface, we could even us map directly, so Map<string, EntryWrapper>. EntryWrapper would be something like {factory: Promise<RegistryEntry> | RegistryEntry}, and consumer could sick additional data on the entry wrapper

I'd appreciate more details here. It's functionally the same as before (in terms of actual logic / order / etc.), so I have hard time computing statement that it causes problems (unless there are bugs, of course).
In my view what ContainerRuntime uses is strictly ContainerRungime business. Nobody should depend on it - it is used only by this layer to serialize and deserialize data stores.
Users of ContainerRuntime can store source of that information in any format they want. It's their own business. But when they create ContainerRuntime  they need to convert it to IFluidDataStoreRegistry and forget about ContainerRuntime representation. I think logically it's much better to model it as ContainerRuntime having this data in its own copy in its own format.
This change is indeed related to #4193 - I created this PR as result of discussion in this issue, because I believe we should separate those concerns (what container runtime uses vs. what user code uses), not merge them.

Ping.
I've glanced at Bohemia repo, and I do not see anybody reinterpreting, casting, or reaching inside of IFluidDataStoreRegistry, neither on container not on individual objects.

That's because we can't get the information we need from the registry today so we have workarounds, not that there isn't a use path for it :) If you look at createComponentHandler.ts we make extensive use of the "container registry" entries in order to create components. We just can't source it from the actual registry.

@leeviana , please take a look. Here is what I've done in latest iteration:

All interesting layers (runtime, Aqueduct) use only interfaces, and propagate interfaces without any changes through the system, allow composition of custom objects (implementing factories).

if you pass IFluidDataStoreRegistry to DataObjectFactory, that's precisely the object you get when asking for registry from factory
there are no promises or other wrappers.

That said, to support any form of composition, createDataStoreRegistry() can be used to produce IFluidDataStoreRegistry, and it can take input in multiple formats, including old NameFluidDataStoreRegistryEntries, same without promises, as well as IFluidDataStoreRegistry itself.

This way, we push out composition problem out of runtime & Aqueduct and allow users to come up with their own data format to represent registries - wrapping it with IFluidDataStoreRegistry can happen outside of these layers. Or it can happen at Aqueduct layer, if we use one of the 3 supported formats.
The only place where it's not true is ContainerRuntime layer that wraps passed in IFluidDataStoreRegistry with another object. If absolutely required, we can expose "user" IFluidDataStoreRegistry object, though I'd prefer to avoid it, as the caller (instantiateRuntime) can pass same data to root data store directly, in any format it desires, which seems better than relying on this layer not to do anything with data.

I want to better address "why" and overall concerns in this post.
This change is about

Runtime focusing on interfaces and not care about data representation to build registry
Have less concepts in the system
Particular format (like NamedFluidDataStoreRegistryEntries) is pushed out and becomes input into specific utility function (createDataStoreRegistry) to form registry. Any alternative format is possible and becomes a matter of building similar utility function
This indirectly simplifies composition, as it's possible (but not required) to use factories themselves as registry functions. More concepts can be removed in the future, for example PureDataObjectFactory.registryEntry - we can just use factory itself.

Tangential:

This change brings us closer to treating container runtime, data store and dds and similar layers (nested channels) - that's the directly we want to move for various reasons. There will be specific extensions (to support IRuntime specific behaviors), but registry handling should have same form across container runtime and child registries (data stores).
This change allows removal of promises from APIs in the future. I find it odd to use promise of promise (that's what it is when we use Promise or equivalent, given that IFluidDataStoreRegistry has only one async funciton on it - nothing else!)

What it's not:

It does not add more capabilities (outside of described above). For example, fixed container registry in main is not a design point that matters or one that we can enforce - it's trivial to overcome that by using 2+ level dynamic registry with fixed first-level registry.

any limitations that specific eco-system wants to enforce here should be done through adapter layer, built on top of runtime, not attempting to bake it into runtime, as that violates over design point of system being flexible, dynamic and supporting delay-loading..

@vladsud Is there further discussion needed on this PR? I suggest closing it or converting to draft if it's not going to be merged in the next month.


Support new deb/rpm permissions changes in #4043 on integTest

Description
Support new deb/rpm permissions changes in #4043 on integTest
Issues Resolved
https://github.com/opensearch-project/opensearch-build/issues/3815
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Awaiting https://github.com/opensearch-project/opensearch-build/pull/4043.


Idea: let PyScript target WASM

Instead of transpiling to JavaScript, we could compile to WASM. This would make PyScrupt a language that is really fast, we can support ndarrays, and a safer language that fails hard when it does (unlike JS's undefined for nonexisting attributes).
This will be really hard if you want to make it feel fully like Python. But if you make it a typed language (i.e. requiring the user to use Python's type annotations) it should be possible to statically compile it. Even then it will be a lot of work though. Probably too much, but we should at least consider it.

Maybe targeting WASM is a bit of a long shot, but in this line of thought, we can consider an Elm-like approach, by making PyScript more type-aware so that we can prevent errors at compile time.

I second that!  i have been reading elm but pyscript is a new groundbreaker

As mentioned many times, a major problem with PyScript is that it looks like Python, but is really JavaScript. I wonder whether this is a problem because its not Python, or because it is worse than Python, by which I mean that JS is a language with no compile time checks so more errors at runtime. Or worse, without runtime errors (e.g. attribute access) causing the code to behave in unpredictable ways, or runtime errors in other places.
I suspect that if PyScript would catch most errors at compile time, the fact that its not Python would bit be that bad. Compiling to WASM is one option, but we could could still compile to JS and do type checking. The main idea is to fail early instead of later. Elm even promises to have no runtime exceptions.

An example that just popped to mind is properties that are tuples. In Python these are immutable, so you need to make a copy to change them. In JS these are a list, and one can mutate the value and then set it, but Flexx will then not detect that the value has changed. I've seen this go wrong several times.

BTW, if PyScript would target WASM instead of JS, there would be some pretty interesting implications, because WASM can (eventually) run anywhere ... For instance, one can write code to do performance critical stuff (e.g. image processing) which you could run inside the Python process.

Beeware's Batavia approach also seems interesting https://github.com/pybee/batavia
Instead of writing an interpreter or transpiler, execute the Python bytecode from JS.

Yes, it's interesting, but really really slow :)

Letting PyScript target WASM is not a good idea. I changed the title to reflect a more achievable goal: to support writing specific functions that run in WASM. This will likely be done by compiling typed Python to WASM. Windel is already working on something like this in the PPCI project (which, btw, may also support executing WASM in Python at some point).
The purpose would be to write compute-intensive (but otherwise relatively simple) in a way that can run faster than we can in Python/JS.

Issue moved to flexxui/pscript #6 via ZenHub


Add spew debugging

Signed-off-by: John Howard<EMAIL_ADDRESS>@cpuguy83 PTAL as I know you like your debugging stuff :smile:
Adding some extremely useful additional debugging when reacting to SIGUSR1 (Linux) or the debugging event being signalled on Windows. I've debugged so many deadlocks now, and while the stacks are useful, getting the daemon data-structures out as well is likely to prove invaluable.
This PR uses https://github.com/davecgh/go-spew to do the reflection/pretty-printing etc.
Note that NO LOCKS ARE TAKEN. Hence the data-structure could potentially be changing during the dump and cause a panic. It is very deliberate to not take any locks, as if the daemon is deadlocked, that would cause the dumper routine to potentially deadlock too. For that reason, there's an explicit recover added in a goroutine to ensure that the daemon does not panic.
Here's a sample output on Windows:
PS C:\control> $p=(get-process dockerd).id;docker-signal -pid $p
Daemon signalled successfully. Examine its output for stacks

time="2016-10-12T16:00:40.720524700-07:00" level=info msg="API listen on //./pipe/docker_engine"
time="2016-10-12T16:00:47.961900400-07:00" level=info msg="goroutine stacks written to C:\\control\\goroutine-stacks.log"
time="2016-10-12T16:00:48.212779200-07:00" level=info msg="daemon dump written to C:\\control\\daemon-data.log"

PS C:\control> type .\daemon-data.log
=== BEGIN daemon dump ===
2016-10-12 16:00:47.9638575 -0700 PDT
(*daemon.Daemon)(0xc0423bc540)({
 ID: (string) (len=59) "W4GO:UXRR:PWJE:CLWX:3K6O:PFGX:I2KN:WVRR:V4G4:NKAR:QOBY:AEW7",
 repository: (string) (len=21) "C:\\control\\containers",
 containers: (*container.memoryStore)(0xc042155ee0)({
  s: (map[string]*container.Container) (len=1) {
   (string) (len=64) "3bcc1fd2c454fa545ce23ff7b9976a27c916fa58ac450079e6e9aba79b991055": (*container.Container)(0xc042423d40)()
  },
  RWMutex: (sync.RWMutex) {
   w: (sync.Mutex) {
    state: (int32) 0,
    sema: (uint32) 0
   },
   writerSem: (uint32) 0,
   readerSem: (uint32) 0,
   readerCount: (int32) 0,
   readerWait: (int32) 0
  }
 }),
 execCommands: (*exec.Store)(0xc042155f00)({
  commands: (map[string]*exec.Config) {
  },
  RWMutex: (sync.RWMutex) {
   w: (sync.Mutex) {
    state: (int32) 0,
    sema: (uint32) 0
   },
   writerSem: (uint32) 0,
   readerSem: (uint32) 0,
   readerCount: (int32) 0,
   readerWait: (int32) 0
  }
 }),
 referenceStore: (*reference.store)(0xc04203cb40)({
  mu: (sync.RWMutex) {
   w: (sync.Mutex) {
    state: (int32) 0,
    sema: (uint32) 0
   },
   writerSem: (uint32) 0,
   readerSem: (uint32) 0,
   readerCount: (int32) 0,
   readerWait: (int32) 0
  },
  jsonPath: (string) (len=48) "C:\\control\\image\\windowsfilter\\repositories.json",
  Repositories: (map[string]reference.repository) (len=4) {
   (string) (len=7) "busybox": (reference.repository) (len=1) {
    (string) (len=14) "busybox:latest": (digest.Digest) (len=71) sha256:eaaaeb371ce6803f3a54287767df5bbbd3b2150b385cacdf202b81419640c62f
   },
   (string) (len=10) "nanoserver": (reference.repository) (len=2) {
    (string) (len=25) "nanoserver:10.0.14393.206": (digest.Digest) (len=71) sha256:57be450d96ed8aefd3ad7b5ce32ec5d370debad0ae88a2182a57bff052055cf7,
    (string) (len=17) "nanoserver:latest": (digest.Digest) (len=71) sha256:57be450d96ed8aefd3ad7b5ce32ec5d370debad0ae88a2182a57bff052055cf7
   },
   (string) (len=4) "psql": (reference.repository) (len=1) {
    (string) (len=11) "psql:latest": (digest.Digest) (len=71) sha256:f594941fa618d5fb8e46495ff5b31a804b6b58f0d88fd9bc5531402a51c53c96
   },
   (string) (len=17) "windowsservercore": (reference.repository) (len=2) {
    (string) (len=32) "windowsservercore:10.0.14393.206": (digest.Digest) (len=71) sha256:d754b1c586b2c9a96baea0d9367b43c315a8b5c181284fa8baa8c8286a4183d8,
    (string) (len=24) "windowsservercore:latest": (digest.Digest) (len=71) sha256:d754b1c586b2c9a96baea0d9367b43c315a8b5c181284fa8baa8c8286a4183d8
   }
  },
  referencesByIDCache: (map[digest.Digest]map[string]reference.Named) (len=4) {
   (digest.Digest) (len=71) sha256:eaaaeb371ce6803f3a54287767df5bbbd3b2150b385cacdf202b81419640c62f: (map[string]reference.Named) (len=1) {
    (string) (len=14) "busybox:latest": (*reference.taggedRef)(0xc0423ce380)(busybox:latest)
   },
   (digest.Digest) (len=71) sha256:57be450d96ed8aefd3ad7b5ce32ec5d370debad0ae88a2182a57bff052055cf7: (map[string]reference.Named) (len=2) {
    (string) (len=25) "nanoserver:10.0.14393.206": (*reference.taggedRef)(0xc0423ce1c0)(nanoserver:10.0.14393.206),
    (string) (len=17) "nanoserver:latest": (*reference.taggedRef)(0xc0423ce210)(nanoserver:latest)
   },
   (digest.Digest) (len=71) sha256:f594941fa618d5fb8e46495ff5b31a804b6b58f0d88fd9bc5531402a51c53c96: (map[string]reference.Named) (len=1) {
    (string) (len=11) "psql:latest": (*reference.taggedRef)(0xc0423ce280)(psql:latest)
   },
   (digest.Digest) (len=71) sha256:d754b1c586b2c9a96baea0d9367b43c315a8b5c181284fa8baa8c8286a4183d8: (map[string]reference.Named) (len=2) {
    (string) (len=32) "windowsservercore:10.0.14393.206": (*reference.taggedRef)(0xc0423ce2d0)(windowsservercore:10.0.14393.206),
    (string) (len=24) "windowsservercore:latest": (*reference.taggedRef)(0xc0423ce320)(windowsservercore:latest)
   }
  }
 }),
 downloadManager: (*xfer.LayerDownloadManager)(0xc0423c3440)({
  layerStore: (*layer.layerStore)(0xc042007e40)({
   store: (*layer.fileMetadataStore)(0xc0423ce6a0)({
    root: (string) (len=38) "C:\\control\\image\\windowsfilter\\layerdb"
   }),
   driver: (*windows.Driver)(0xc0423c6c30)(windowsfilter),
   layerMap: (map[layer.ChainID]*layer.roLayer) (len=11) {
    (layer.ChainID) (len=71) sha256:ff3afc74726160ea24b664529ad3ab983b7a572c82742e00c53ad409bcfa643d: (*layer.roLayer)(0xc0423e0460)({
     chainID: (layer.ChainID) (len=71) sha256:ff3afc74726160ea24b664529ad3ab983b7a572c82742e00c53ad409bcfa643d,
     diffID: (layer.DiffID) (len=71) sha256:a6e513bc458471199d83f8f6c2f0593f2f505ef451133cfc4734628abe4fef41,
     parent: (*layer.roLayer)(0xc0423e0140)({
      chainID: (layer.ChainID) (len=71) sha256:0ce98b97c17e1007db26ace432c35229aaa7d48ab6632989e6dd44e588c871a9,
      diffID: (layer.DiffID) (len=71) sha256:c3f6e5273094be71a347dc0dda60b72e60415b9c61b60f6dadf8aa89bbc0c9d7,
      parent: (*layer.roLayer)(0xc0423e01e0)({
       chainID: (layer.ChainID) (len=71) sha256:e654f3dbb45c05c83942df4d0f67d60f29d3182c923cee7a170c7b7f12baa869,
       diffID: (layer.DiffID) (len=71) sha256:0fe6c66aab9eed3388ed31a88faa4d1e2f4f4f194cc8d396d45e1f71005f32e2,
       parent: (*layer.roLayer)(0xc0423e0320)({
        chainID: (layer.ChainID) (len=71) sha256:2c08913762fe083474c0a9eec5405fdd99203472ccfcfbedecaf768612b68332,
        diffID: (layer.DiffID) (len=71) sha256:2c08913762fe083474c0a9eec5405fdd99203472ccfcfbedecaf768612b68332,
        parent: (*layer.roLayer)(<nil>),
        cacheID: (string) (len=64) "c38ac738bb8c6e079718266f60a4223aab5848214b5cd6caeadb30c6f11e2a89",
        size: (int64) 662468910,
        layerStore: (*layer.layerStore)(0xc042007e40)(<already shown>),
        descriptor: (distribution.Descriptor) {
         MediaType: (string) (len=57) "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
         Size: (int64) 242834639,
         Digest: (digest.Digest) (len=7) -diffid,
         URLs: ([]string) (len=1 cap=4) {
          (string) (len=14) "layer_is_local"
         }
        },
        referenceCount: (int) 5,
        references: (map[layer.Layer]struct {}) (len=1) {
         (*layer.referencedCacheLayer)(0xc042028378)({
          roLayer: (*layer.roLayer)(0xc0423e0320)(<already shown>)
         }): (struct {}) {
         }
        }
       }),
       cacheID: (string) (len=64) "d74115d4724657f657d0408f01c5d899fe0d414dd847b605e37918fbb2933720",
       size: (int64) 533504,
       layerStore: (*layer.layerStore)(0xc042007e40)(<already shown>),
       descriptor: (distribution.Descriptor) {
        MediaType: (string) "",
        Size: (int64) 0,
        Digest: (digest.Digest) ,
        URLs: ([]string) <nil>
       },
...<snipped>
=== END daemon dump dump ===
PS C:\control>

I don't follow docker dev but I saw this linked from the go-spew issue, so here I am.

Note that NO LOCKS ARE TAKEN. Hence the data-structure could potentially be changing during the dump and cause a panic.

This worries me a lot.  Data races are a violation of the Go memory model, and create the same situation of undefined behavior you might expect to see in a C++ program.  This can result in the entire program's execution, from start to finish, to be undefined (even before UB "happens").
The most reliable way to inspect the memory of a deadlocked program (at least in Unix) would be to dump a core.  I'm sure similar methods exist in Windows.

I'm sure this is safe having artificially forced a few deadlocks in testing it. Unfortunately no good tools really exist to do the equiv of a core dump that can examine go structures on Windows. Worst case, the daemon crashes. But if it's already deadlocked, that's the least concern. On Windows it may well be deadlocked due to a win32 call not returning (several past examples) while the daemon is holding a lock which blocks other threads. In those cases, this is also perfectly safe as nothing will be changing under it. Just more data to aide debugging.

And to add, the added code also has a recover go function if there is a panic, and I've verified that that copes by forcing a nil pointer indirection. The spew code should just be a reader, not a writer.

ping @LK4D4 @stevvooe ptal

This looks very useful. I wish there was a standard Go way to do this. Is there something we're doing that makes it hard to use delve?
I am slightly worried about the lack of locking here. Is the daemon memory safe after doing a debugging run or should we exit? Considering this is for debugging deadlocks, that may not be a real concern.

Delve is an option if you can keep the machine for further investigation. That's not always the case for me over the past few months. However, you need source access on the machine, and delve or GDB are difficult to use on a nano server host, almost next to impossible.
I think this is safe and the benefits outweigh the risks. I would also rather keep the daemon running as that does also leave the option of subsequent attaching using delve or GDB. Given that this is a reader, it shouldn't have any further adverse effects.

@jhowardmsft Fair enough.
@diogomonica @NathanMcCauley Let's do a security review on this one.

Rebased. Can we move this forward to code review? I'd really like to get this in for 1.13. @thaJeztah

@cpuguy83 Yes, you're right. Removed the if block. I was blindly copying an example.

Rebased

ping @cpuguy83 @stevvooe PTAL

Rebased again.

ping @AkihiroSuda perhaps you can have another look, to see if we can get this moving?

LGTM
I am not sure of a better way forward here. We should reserve the right to remove this at any time when the debuggers can do core analysis.

@thaJeztah LGTM (sorry for being late 😅)

@AkihiroSuda no worries!

In linux this debug file is 6.6 MB, over 100000 lines and contains a precalculated array of 65536 IP addresses for an empty daemon. Surely we don't need all that data and it just makes it harder to move around that file. Of course, we should also see why we are allocation so many things.

@tonistiigi can you open an issue for tracking that?


createa an association between two traces

We are looking for a way to associate two different traces so that they show as related to each other in the interface. Our usecase is: each transaction is a trace and has a bunch of spans. Sometimes transaction A conflicts with something that some other transaction B did. If are looking at A's spans, we want to be able to "jump" to B's spans. Currently it it very hard to identify transaction B and its spans.
One possible way to do that through the opentracing API would be to create a Span that derives from both traces - StartSpan("conflict", FollowsFrom(ctxA), FollowsFrom(ctxB)). Unofrtunately the basictracer-go code ignores everything but the first reference: https://github.com/opentracing/basictracer-go/blob/c7c0202a8a77f658aeb2193a27b6c0cfcc821038/tracer.go#L159
We could try to extend basictracer-go to support multiple references if lightstep would be able to handle them.

CC @tschottdorf @andreimatei

@RaduBerinde ACK
This is something I'd love to implement. There are a few issues here...

As you observed, BasicTracer doesn't do the right thing with N>1 references which is basically a bug.
Even if it did the right thing, the LightStep Frontend doesn't do anything special with multiple parents [yet]...
... and most importantly, I don't think of these sorts of inter-trace references as being "multiple parents"; it's more like a pointer, and IMO deserves its own reference type at the OT layer.

Since you all actually have a legit production use case (yay), I would suggest opening an issue at https://github.com/opentracing/opentracing.github.io/issues describing the semantics you need. We can then go forward and talk about what that reference type should be (in a language-neutral way), then finally figure out how to represent it in the Go/etc APIs.
(PS: I think it's reasonable to add a pointer to another trace during the lifetime of a Span, not just at StartSpan time... so maybe it would make sense to have something like log.Reference(...) a la https://github.com/opentracing/opentracing-go/pull/108)

Thanks, filed https://github.com/opentracing/opentracing.github.io/issues/119

Thanks! I will close this as the real issue here (IMO) is the OT one. The larger feature request for LightStep is/was already on our radar and probably doesn't belong in the Go client lib regardless.


not honoring nested includes inside include that uses leveloffset

I am trying to use this to build a single README out of a collection of docs in a subfolder.  There are nested includes, however, only the initial (top level) includes are being actually proceed.  includes in the those included files are being missed.  An example
README.adoc has a line
include::docs/workflows.adoc[leveloffset=+1]
workflows.adoc has a line
include::scripts.adoc[leveloffset=+1]  // scripts is also in docs
scripts.adoc has a line
include::rmdvc.adoc[leveloffset=+1] // rmdvc alsi is in the docs dir
The result in the rendered README-OUT.adoc is
:leveloffset: +1

# Workflows

include::scripts.adoc[leveloffset=+1]

:leveloffset!:

You can see that it didn't process the scripts.adoc include (therefore, none of the others).
No errors get thrown.
I thought it might be a path problem, but when I change all of the include paths to include 'docs' as the parent dir, it does throw an error about not being able to find '/home//repo/docs/scripts.adoc'
What am I missing?

I just (re)discovered this issue this past week and came here to file this issue. It seems like if an include uses leveloffset, then includes inside that include don't get replaced properly. And, lo and behold, we are missing a test for it.
The reason this happens is because the reader injects leveloffset attribute entries into the included file, but the replacer is not accounting for these lines. Therefore, it can't find the include directive to replace.
In the meantime, the workaround is to move leveloffset to around the include directive.
:leveloffset: +1

include::scripts.adoc[]

:!leveloffset:

I'll go ahead and add a test and fix this issue for the next release.

The fix for this issue is now available in the latest release.


Describe link to DK Core and IPA

@AsgerSDSArkitekt can I request you review this section https://build.fhir.org/ig/hl7dk/dk-smart/branches/master/index.html#i-still-dont-follow---why-do-we-need-this - hopefully it brings the needed explanation.

@jkiddo : I believe the sections 1.3 and 1.4 improves the IG and makes it more understandable, why that IG is actually needed. I remember, than Christian Gasser fra Lakeside originally had similar "confusion comments" like mine, so maybe also send him a link to the new improved version for more review eyes.

@christiangasser-lakeside will you have a look as well?

@christiangasser-lakeside ping?

Yes, the updated section definitely make the IG more understandable :-). Minor detail: The two web-addresses at the end of section 1.4 should be links, so the reader doesn't have to do copy/paste to access the sites.


[Bitwarden] Error: Command failed with EACCES

Extension
https://www.raycast.com/jomifepe/bitwarden
Description

Error:
Error: Command failed with EACCES: /usr/local/Cellar/bitwarden-cli/2023.1.0 status
spawn /usr/local/Cellar/bitwarden-cli/2023.1.0 EACCES

ChildProcess._handle.onexit:child_process:283:19
onErrorNT:child_process:476:16

Steps To Reproduce

Installed Bitwarden CLI via brew
Entered client ID, client secret and CLI installation path into Raycast Bitwarden extension settings
Use command search vault
"Something went wrong" screen

Current Behaviour
Search vault command returns the "Something went wrong" screen
When reloading it flashes Vault status and Master password screen before then showing "Something when wrong" screen again.
Expected Behaviour
No response

[FIXED] Removed the path returned by the home-brew installation terminal and left it as the default path. Seems to have resolved the issue. User error perhaps...my bad


Fix python notebook mode freezing after showing matplotlib plot

Fixes the bug discussed in #103 where calling plt.show() for a matplotlib plot in python notebook mode would freeze the runtime and prevent running code again while leaving a runaway python instance in the background. The fix is to just change the matplotlib backend to the non-interactive 'agg' option if matplotlib is used in the code block.

Amazing; thanks so much!!
I have one question: why call matplotlib.use() in every code block? It seems to make more intuitive sense to me to call it once in the initial setup code. Although it couldn't be conditional on use of matplotlib, it could be based on the pythonEmbedPlots setting.

I've moved the matplotlib import to the initial code setup, however I've left the matplotlib.use() calls up to each code block so that it's possible to toggle the 'embed plots' option, I hope this is an ok way to do it. Looking into this issue further though, we can't use a non-interactive backend when embed plots is off, so the original issue persists when not embedding plots. I've tracked it down to plt.plot() spawning the backend process, but I couldn't find a way to fix this yet.

That makes sense. Thank you!

I looked through plt.plot()'s source code, and I'm not sure if it does spawn that background process. It might be different on windows/mac/linux, but on Windows 10, the only process I see is the Python process that hosts the REPL. I'm quite not sure that that's what you're talking about, but I wanted to let you know :) sorry

I just observed on my mac when for example setting matplotlib.use('tkagg'), as soon as I run plt.plot() in my terminal, the tk feather icon pops up in my dock, and it stays there until the REPL is closed, it looks like it's opening a window in the background, not sure what it's doing exactly though, haven't looked into the source code yet.

Looks good, thank you!


Track objects allocated with new

Note: Depends on https://github.com/amzn/merlin-on-demand-callgraph/pull/21, please review that one first.
Previously, only NewObjectNodes (i.e. object literals such as {}) were treated as allocations and tracked by the points-to analysis. this commit adds tracking for objects allocated by new.
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

are we handling Object.create ?

No, at the moment we don't handle any builtin functions yet. I'll create an issue for Object.create and handle that separately.


Sharing with Facebook on Android empty

When trying to share with Facebook on Android, only an empty share box opens. No text in it, no image linked...
this.socialSharing.share(
  res['SHARE_TEXT'],
  res['SHARE_TITLE'],
  this.image,
  this.url
)

@JerryBels it's not an issue. FB and Instagrams don't allow to make direct input of text. User should add it from clipboard

hi all, i have the same issue, but facebook contacted me some days ago that if i dont fix it they will remove my app ...
anyone has an idea to fix this?
if not its no problem.. i simply will remove the shitty facebook login from my app ;-)


Annotations

Added annotated doc
Updated README

I wonder how this will be impacted by https://github.com/rjrodger/seneca-store-test/pull/18 that is still waiting for final approval


"unknown" database and user take the brunt of the stats

New user here.
I've ran pgbadger on the stats collected overnight simply with pgbadger -I -q -f stderr over ssh.
All good, but there's this unknown user & databse that show up everywhere and get the brunt of the queries.
Trying to find it in the stderr logs, I could only find this type of connection logs actually using it:
[unknown]@[unknown] LOG:  connection received
And only about ~1500 matches, where I get like ~300,000 queries referenced by unknown as a user.
Everything should be either default postgres or a user database.
Is there anything wrong in my postgres logs?

Hi, this is because you don't have %u and %d in the log_line_prefix configuration directive. In this case pgBadger will use "unknown" as user and database name for all queries executed on a connection.
See documentation about the log_line_prefix tha pgbadger can use.

Hi, thanks for your prompt answer.
Actually, the default for a postgres v12 install is log_line_prefix = '%m [%p] %q%u@%d '
This is what I have, so the user and database are specified.
But that rings a bell, I should probably use `--prefix '%m [%p] %q%u@%d ' to make sure pgBadger knows how to read the log_line_prefix.
And thanks for this fantastic piece of software!


Cannot read property 'scale' of undefined

When I load images after rendering I get "Cannot read property 'scale' of undefined".

For some reason it started working.

I tested and now I understand the bug.
I can't render ImageView with the initial image being null.
Which means that I need to create some dummy image for it to work.
This will not work:
const DEFAULT_STATE: any = {
item: null,
key: 0,
isImageViewVisible: false
}

When I replace the image in reducer the ImageView still shows me the dummy image.
render()
{
return (

);
}

The problem is that you take the 'images' from state instead of props.

Thanks for issue, let me investigate it. The problem is that in this version there are a lot of logic happens when component receive new images. I think you right and maybe it is not necessary to store images in state, but I need some free time to test it all. Or you can send PR :)

Sorry, don't have the time.

Any progress with this issue?

If you want to fix it yourself, you just need to take the images in render() from props instead of state.


[Meson] CI for meson project

Description
Happy new Year!
As Meson support is becoming more stable it would be good to have a CI for the meson project so we can check any bug or edge case like #394 or #378 this issue should be a good starting point in order to define how the CI should look like along with any feedback that is required in order to complete it, additionally having this CI will help us add any new feature that may not be included con CMake or Meson itself so it can be fixed and tested carefully before merging
an initial proposed format should look like the example of the first comment
as you may notice it is pretty similar to the CI that is already in CMake, but needs to be checked/reviewed first in order to check if everything is working good before opening a PR.
please let me know what do you think or if you have any idea/sugestion.

name: build

on: [push, pull_request]

jobs:
  build:
    name: ${{ matrix.config.name }}
    runs-on: ${{ matrix.config.os }}
    strategy:
      fail-fast: false
      matrix:
        config:
        - {
            name: "Windows Latest MSVC",
            os: windows-latest,
            generator: "Visual Studio 17 2022",
            build_type: "Debug",
            cc: "cl", cxx: "cl",
            extra_path: "",
          }
        - {
            name: "Windows Latest MinGW",
            os: windows-latest,
            generator: "MinGW Makefiles",
            build_type: "Debug",
            cc: "gcc", cxx: "g++",
            extra_path: "C:\\ProgramData\\chocolatey\\lib\\mingw\\tools\\install\\mingw64\\bin",
          }
        - {
            name: "Ubuntu Latest GCC",
            os: ubuntu-latest,
            generator: "Unix Makefiles",
            build_type: "Debug",
            cc: "gcc", cxx: "g++",
            extra_path: ""
          }
        - {
            name: "macOS Latest Clang",
            os: macos-latest,
            generator: "Unix Makefiles",
            build_type: "Debug",
            cc: "clang", cxx: "clang++",
            extra_path: ""
          }

    steps:
    - uses: actions/checkout@v2
    - name: submodule
      run: git submodule update --init --recursive
    - name: extra_path
      shell: bash
      run: echo "${{matrix.config.extra_path}}" >> $GITHUB_PATH
    - name: configure
      shell: cmake -P {0}
      run: |
        set(ENV{CC} ${{matrix.config.cc}})
        set(ENV{CXX} ${{matrix.config.cxx}})
    - name: install prerequisites
      run: |
        # asuming that python and pip are already installed
        pip3 install meson
    - name: setup meson project
      run: |
        # setup the build directory with tests and examples enabled
        meson setup builddir -DSQLITECPP_BUILD_TESTS=true -DSQLITECPP_BUILD_EXAMPLES=true
    - name: build meson project
      run: |
        # build the project
        meson compile -C builddir
    - name: test
      run: |
        # run the tests
        meson test -C builddir

This is awesome!
And yes obviously we do need Meson to be tested by the build system: everything should be tested on CI, and as you say it's becoming more mature.
As for reviewing it, it seems good to me, but I am not a meson expert; I think we will discover the little details as soon as you try to propose a PR
(in fact, you might not even be allowed to provide a new Github Action file in a PR, but this I can remediate :) )

PR opened in #407 that should include it and solve this issue


Hit scaffolding error

Hi,
I'm running Fast-Plast v.1.2.9 on my Illumina PE data, but execution stopped after afin step with error message:
Hit scaffolding. at /home/tkiy/Fast-Plast/fast-plast.pl line 733.
Please find the log files attached. Could you help me please in fixing the error?
Solanum-A1_results_error.log-2.txt
Solanum-A1_Fast-Plast_Progress.log-2.txt
Regards,
Denis

Dear Dr. McKain,
I much appreciate your detailed response! I'll try to rerun the software
with the  --min_length_trim option.
Regards,
Denis
вт, 29 нояб. 2022 г. в 01:59, Michael McKain @.***>:

Hi Denis,
There are five contigs in the afin output, which is what triggered this
outcome. Fast-Plast tries to identify if those contigs are nested, so they
can be removed. When that is not the case, it is supposed to try to
scaffold contigs together using SSPACE. You can comment out line 733 in
your file to have it proceed to this step.
I would recommend another approach, though, where you use the
--min_length_trim option to reduce the minimum viable length of a trimmed
tread to 100. The default is 140. You are dropping 33% of your read pairs
from that trim step (Line 7 in error.log). This may help with recovery.
Best,
Michael
—
Reply to this email directly, view it on GitHub
https://github.com/mrmckain/Fast-Plast/issues/54#issuecomment-1329862692,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACR2TOPDRZ3B6C6A7UIF43TWKU2M7ANCNFSM6AAAAAASN2ZMTQ
.
You are receiving this because you authored the thread.Message ID:
@.***>


All System.Timers.Timer's created duplicate after each hot reload

An alternative well performing class is needed, preferably one that's already in the osu!framework.

Fixed (hopefully temporarily) by returning if there is more than one ScoreImporter initalization.

NVM now that implementation freezes the cursor each time it runs.... should really use something from o!f.


Default values no longer returned as response?

Description
I am no longer getting the default values of inputs in request.inputs, see code below. Trying to access the inputs that are not provided crash the process. See code below and the error returned by the WPS process in the comments:
from pywps import Process,get_format,configuration
from pywps import LiteralInput,LiteralOutput,ComplexOutput

# Example usage:
#
#localhost/pywps?service=WPS&request=execute&version=1.0.0&identifier=test3&DataInputs=limit=2
#
# However, not providing the limit input fails:
#
#localhost/pywps?service=WPS&request=execute&version=1.0.0&#identifier=test3&DataInputs=
#
# Error:
# exceptionCode="NoApplicableCode"
# Process error: wps_test3.py._handler Line 42 'limit'

class Test3(Process):
    def __init__(self):
        inputs = [LiteralInput('limit',
                               'Some limit',
                               data_type='integer',
                               default=10,
                               min_occurs=0),]

        outputs = [LiteralOutput('test_result',
                                 'Test result',
                                 data_type='string')]

        super(Test3,self).__init__(
            self._handler,
            identifier='test3',
            title='test3 process',
            version='0.1',
            inputs=inputs,
            outputs=outputs,
            store_supported=True,
            status_supported=True)

    def _handler(self,request,response):
        # This fails with
        # exceptionCode="NoApplicableCode"
        # Process error: wps_test3.py._handler Line 42 'limit'
        limit = request.inputs['limit'][0].data
        response.outputs['test_result'].data = str(limit)

        # If we output the keys to request.inputs, we see that 'limit' is
        # not there (in this case it's an empty list):
        #response.outputs['test_result'].data = str(request.inputs.keys())

        # The only workaround I find currently is to fetch the default from
        # the initial list of inputs:
        #if 'limit' not in request.inputs:
        #    for one_input in self.inputs:
        #        if one_input.identifier == 'limit':
        #            default_str = str(one_input.default)
        #            response.outputs['test_result'].data = default_str
        #            break

        return response

Environment

operating system: Ubuntu 16.04
Python version: 2.7.12
PyWPS version: 4.0.0 (2017-01-13)
source/distribution

[x] git clone
[ ] Debian
[ ] PyPI
[ ] zip/tar.gz
[ ] other (please specify):

web server

[x] Apache/mod_wsgi
[ ] CGI
[ ] other (please specify):

Steps to Reproduce
Additional Information

I have a follow up question. Is pywps supposed to make the distinction between an input that was not provided at all and an input that was set to its default value? Here's a simple example where this matters:
Let's say I have a python function that has a "limit" parameter that's an integer, but that can also be "None" (for no limits). In WPS, since the LiteralInput is set to data_type integer, I believe it's default value must also be an integer. If the pywps "request" object tells me whether the limit input was provided or not, I could set it to None inside the _handler and proceed to call the python function that accepts either an integer or None. The other option is to add a boolean LiteralInput that determines whether my "limit" should be set or not. Does that make sense?

I have the same problem. If an optional input is not given it results in an KeyError in Apache (like in issue #154), and an "NoApplicableCode"-exception in the response. Should I provide more details?

Hi,
you are both right. The input parameter "default" was available there for LiteralInput, but it was not used. You could theoreticaly use it manually
        limit = request.inputs['limit'][0].default

But that was not nice.
I've introduced more robust default value handler and it's now availabe for testing tin #236 pull request - could you check, if it works for you, please?
Thanks

Hey,
unfortunately it seems like I still get the same error after trying out your commit.:
56 [Mon Feb 20 09:57:29.939029 2017] [wsgi:error] [pid 9140] 2017-02-20 09:57:29,938 INFO sqlalchemy.engine.base.Engine COMMIT
257 [Mon Feb 20 09:57:29.953593 2017] [wsgi:error] [pid 9140] Traceback (most recent call last):
258 [Mon Feb 20 09:57:29.954230 2017] [wsgi:error] [pid 9140]   File "/usr/local/wps/src/pywps/pywps/app/Process.py", line 222, in     _run_process
259 [Mon Feb 20 09:57:29.954247 2017] [wsgi:error] [pid 9140]     wps_response = self.handler(wps_request, wps_response)
260 [Mon Feb 20 09:57:29.954559 2017] [wsgi:error] [pid 9140]   File "/usr/local/pywps/processes/elevationXML.py", line 56, in _han    dler
261 [Mon Feb 20 09:57:29.954575 2017] [wsgi:error] [pid 9140]     requestedNumberOfPoints = request.inputs['points'][0].data
262 [Mon Feb 20 09:57:29.954630 2017] [wsgi:error] [pid 9140] KeyError: 'points'
When I output the keys to request.inputs (as bstdenis did), the key "points" does not exist.
Here is the relevant process (the commented out try-except was just a hack to go around the problem in the first place):
def __init__(self):
    inputs = [
        ComplexInput(identifier="gpx", title="GPX tracklog",
                     abstract="A tracklog in GPX format to be charted",
                     min_occurs=1, max_occurs=1,
                     supported_formats=[Format("application/gml+xml")],
                     mode=MODE.STRICT),
        LiteralInput(identifier="points", title="number of points",
                     abstract="Number of points to chart", min_occurs=0,
                     max_occurs=1, data_type='integer', default=1)]
    outputs = [
        ComplexOutput(identifier='xml', title='Enhanced track data',
                      abstract='Elevation and terrain',
                      supported_formats=[Format("application/xml")])

    super(ElevationXML, self).__init__(
        self._handler,
        identifier="elevationXML",  # the same as the file name
        title="elevationXML",
        version="0.2",
        inputs=inputs,
        outputs=outputs,
        store_supported=True,
        status_supported=True,
        abstract=self.__doc__,
    )

def _handler(self, request, response):
    # Input
    gpxfile = request.inputs['gpx'][0].file
    #try:
    requestedNumberOfPoints = request.inputs['points'][0].data
    #except:
    #    requestedNumberOfPoints = 1

    # Run operations

And the request which fails:
http://localhost/wps?request=Execute&service=WPS&version=1.0.0&identifier=elevationXML&datainputs=gpx=@xlink:href=http://remotehost/elevation_readable.gpx

I second @kvold observation, the default value is not returned in the "request" object. Looking at the modifications you made in your branch, I understand that you are setting the data to the default value when the input object is created, but that does not trigger its inclusion in the "request" object.
There is a design decision to make here, is the "request" object strictly reserved for inputs that were provided by the user? If not, the unprovided inputs should be activated and set to their default value in the "request" object. Otherwise, we have to keep track of our inputs by name (instead of just creating an "inputs" list as we do above) and check for the presence of the key in the "request" object, e.g.:
`
class Test3(Process):
def init(self):
self.limit_input = LiteralInput('limit',
'Some limit',
data_type='integer',
default=10,
min_occurs=0)
# potentially more inputs
inputs = [self.limit_input,]
    outputs = [LiteralOutput('test_result',
                             'Test result',
                             data_type='string')]

    super(Test3,self).__init__(
        self._handler,
        identifier='test3',
        title='test3 process',
        version='0.1',
        inputs=inputs,
        outputs=outputs,
        store_supported=True,
        status_supported=True)

def _handler(self,request,response):
    if 'limit' not in request.inputs:
        response.outputs['test_result'].data = str(self.limit_input.default)
    else:
        response.outputs['test_result'].data = str(request.inputs['limit'][0].data)

    return response

`

well, seems that this should be done along with min and max occurs attributes too :-/ sorry about it

I've just uploaded new version of the fix, in the https://github.com/jachym/pywps/tree/226_default_value branch. Will try to test it with provided process example from @bstdenis  but would appreciate, if someone had a look at it.

I tested your branch and the default value seems to work as expected, thx!
However, while playing around I noticed something else: if I set a default value and min_occurs/max_occurs, I can get some strange behaviours, e.g.:
inputs = [LiteralInput('limit',
                               'Some limit',
                               data_type='integer',
                               default=10,
                               min_occurs=2,
                               max_occurs=3),]

If I do not provide the 'limit' parameter in my wps call, I will get the default value (once) as the result, which is inconsistent with the min_occurs parameter as far as I can tell.
If I provide the limit parameter only one time (which is not enough because of the min_occurs), I get a
File "/usr/local/lib/python2.7/dist-packages/pywps-4.0.0-py2.7.egg/pywps/app/Service.py", line 503, in create_literal_inputs
[Wed Mar 29 19:22:35.526774 2017] [wsgi:error] [pid 23:tid<PHONE_NUMBER>95808] [remote <IP_ADDRESS>:63125]     raise MissingParameterValue(locator=source.identifier)
[Wed Mar 29 19:22:35.526800 2017] [wsgi:error] [pid 23:tid<PHONE_NUMBER>95808] [remote <IP_ADDRESS>:63125] TypeError: init() takes at least 2 arguments (2 given)
If I provide more than 2 'limit' parameter, I get the proper behaviour.
I must say though, I'm not really sure I would ever want to use the default value in combination with a parameter that also has min_occurs... just food for thoughts...

I've hit the same problem. Has this been merged into develop ?


remove unexpected side effect: do not clean expectedEvents

when submitting PRs, please don't commit the version bump.  there is now a conflict and it must be resolved before merging

What should I do in case of interface changes?

Do you mean in general, if you want to change an interface?  In that case, we need to agree the change should be made, and then I will bump the minor version.  This does not look to be a breaking change, so  please revert changes to gradle.properties altogether.

Done


State wrap-load-mutate-flush lifecycle

We currently use orga::state::State by wrapping a store at the beginning of a state machine transition function, then write each mutation to the underlying store on each operation. If we instead change this lifecycle to automatically load relevant data on creation, then mutate state in memory, then "flush" at the end, we can likely get more convenient APIs and perform less writes (just a single write in cases where a key is mutated multiple times).
For an example, consider the Value type. Its usage currently looks like this:
fn increment_count<R: Read>(count: &mut Value<u32>) -> Result<()> {
  let value = count.get()?;
  count.set(value + 1)?;
}

If we automatically load the value when accessed and write it at the end, we can get this instead:
fn increment_count<R: Read>(count: &mut Value<u32>) {
  count += 1;
}

Here, Value<T: AddAssign> would implement AddAssign (plus Deref/DerefMutand other useful ops), then we could treat theValueas essentially just a normalu32. Alternatively, we could just access the inner value with an as_mut(&mut self) -> &mut Tmethod forValue`.
I believe this could also apply to e.g. getting a field from a higher-level collection such as Map or Deque and mutating them without explicit get/set steps.

Another benefit of this lifecycle is that we only need to wrap a Read instead of a Store, then use the Store during the Flush step, which may make some borrow-checker problems easier (it's easier to share an immutable Read between multiple wrappers instead of a mutable Store).


TypeError: Subscripted generics cannot be used with class and instance checks

hi, I installed the required environment following the steps, and then directly ran the Lora fine-tuning script for llama3-8b-instruct, but the following error occurred:
Traceback (most recent call last):
File "/home/ubuntu/D/swift/swift/cli/sft.py", line 5, in
sft_main()
File "/home/ubuntu/D/swift/swift/utils/run_utils.py", line 21, in x_main
args, remaining_argv = parse_args(args_class, argv)
File "/home/ubuntu/D/swift/swift/utils/utils.py", line 94, in parse_args
parser = HfArgumentParser([class_type])
File "/home/ubuntu/anaconda3/envs/swift/lib/python3.9/site-packages/transformers/hf_argparser.py", line 137, in init
self._add_dataclass_arguments(dtype)
File "/home/ubuntu/anaconda3/envs/swift/lib/python3.9/site-packages/transformers/hf_argparser.py", line 264, in _add_dataclass_arguments
self._parse_dataclass_field(parser, field)
File "/home/ubuntu/anaconda3/envs/swift/lib/python3.9/site-packages/transformers/hf_argparser.py", line 172, in _parse_dataclass_field
field.type.args[0] if isinstance(None, field.type.args[1]) else field.type.args[1]
File "/home/ubuntu/anaconda3/envs/swift/lib/python3.9/typing.py", line 720, in instancecheck
return self.subclasscheck(type(obj))
File "/home/ubuntu/anaconda3/envs/swift/lib/python3.9/typing.py", line 723, in subclasscheck
raise TypeError("Subscripted generics cannot be used with"
TypeError: Subscripted generics cannot be used with class and instance checks
how to fix this? thx!

same problem i meet

Use pip install 'ms-swift[all]' -U to solve my problem

same problem i meet

Use pip install 'ms-swift[all]' -U to solve my problem

I am new, and I am trying to fine-tune DeepSeekv1, and I have done so following your instructions, but nothing  changed. How can I solve this problem? Are there any other methods I could try? Thank you!

Use pip install 'ms-swift[all]' -U to solve my problem

I am new, and I am trying to fine-tune DeepSeekv1, and I have done so following your instructions, but nothing changed. How can I solve this problem? Are there any other methods I could try? Thank you!

The following is my solution. I hope it will be useful to you:

uninstall swift pip uninstall ms-swift
use pip install 'ms-swift[all]' -U to install swift

Use pip install 'ms-swift[all]' -U to solve my problem

I am new, and I am trying to fine-tune DeepSeekv1, and I have done so following your instructions, but nothing changed. How can I solve this problem? Are there any other methods I could try? Thank you!

The following is my solution. I hope it will be useful to you:

uninstall swift pip uninstall ms-swift
use pip install 'ms-swift[all]' -U to install swift

Thank you ! It works !!!

Use pip install 'ms-swift[all]' -U to solve my problem

I am new, and I am trying to fine-tune DeepSeekv1, and I have done so following your instructions, but nothing changed. How can I solve this problem? Are there any other methods I could try? Thank you!

The following is my solution. I hope it will be useful to you:

uninstall swift pip uninstall ms-swift
use pip install 'ms-swift[all]' -U to install swift

Thank you ! It works !!!

I find that:

pip uninstall ms-swift  : uninstalling ms-swift-2.1.0.dev0
pip install 'ms-swift[all]' -U : Successfully installed ms-swift-2.0.4 peft-0.10.0

It is possible that the problem appears on the version？


Revert "Clean up the code and some of the API of compiling plugins, and improve persistence of cached compiler outputs"

Reverts apple/swift-package-manager#4290

@swift-ci please smoke test

CC: @abertelrud @tomerd - the change regressed on Windows, which has been broken multiple times in the past week.  I'd like to revert rather than fix forward, and then retry with testing.

@abertelrud @compnerd can we fix this forward easily? if not, we should revert

I can't test it locally atm, can you test and merge this quickly?  I think it is better to revert and then fix.

@compnerd  What's the failure?  I understand wanting to revert, but will need to fix it.

@abertelrud https://ci-external.swift.org/job/oss-swift-windows-toolchain-x86_64-vs2019/509/console

I C:\Users\swift-ci\jenkins\workspace\oss-swift-windows-toolchain-x86_64-vs2019\swift-corelibs-libdispatch\src\BlocksRuntime -I T:\13\swift @CMakeFiles\Workspace.rsp    -Xlinker -implib:lib\Workspace.lib  && cd ."
C:\Users\swift-ci\jenkins\workspace\oss-swift-windows-toolchain-x86_64-vs2019\swiftpm\Sources\Workspace\DefaultPluginScriptRunner.swift:250:27: error: type 'ProcessResult.ExitStatus' has no member 'signalled'

                    case .signalled(let signal):

                         ~^~~~~~~~~

@abertelrud https://ci-external.swift.org/job/oss-swift-windows-toolchain-x86_64-vs2019/509/console

Thanks for the reference.  Let's revert for now and I'll put up a fixed PR.  Is there any way to run the Windows build against CI so I can tell if I'm breaking it again?

Or probably better I can just #if out that part.

Put up https://github.com/apple/swift-package-manager/pull/4299.  Can we test that on Windows somehow?  Would hate to go back and forth on this.

It is possible to do cross-repository testing - that is the only option atm

More specifically you need to build a full toolchain for Windows ("@swift-ci please build toolchain Windows platform") with the associated PR

Great, I'll give that a try.  Thanks @compnerd

Great, I'll give that a try. Thanks @compnerd

Oh but I'd have to do that from within a PR in the Swift compiler I guess.

In this case I'd prefer to merge the fix in order to make progress.  If needed we can revert this and then I'll put up a PR to put it back with the fix, but there's still the question of testing that.  I never was able to get SwiftPM on a local Windows VM to work.

Right, until we get the CI setup on this repo, it would need to be done on that repo (and why it requires a full toolchain build)

Meanwhile, I'm preparing a cross-repo test run with a DNM branch in the Swift repo to test #4299. If we must revert quickly we can do that, then I would convert #4299 to putting the code back with the fix.

https://github.com/apple/swift/pull/42405


Changes to split up Ironic container

Allows Ironic container to be split between ironic, dnsmasq, and httpd
containers using this same image.

Build FAILURE, see build http://<IP_ADDRESS>:8080/job/dev-tools/348/

Build FAILURE, see build http://<IP_ADDRESS>:8080/job/dev-tools/354/

Build FAILURE, see build http://<IP_ADDRESS>:8080/job/dev-tools/371/

Doesn't appear to be working - the http request fails downloading the image with Failed connect to <IP_ADDRESS>:80; Connection refused ?

Doesn't appear to be working - the http request fails downloading the image with Failed connect to <IP_ADDRESS>:80; Connection refused ?

Did you include the dev-scripts changes at the same time (https://github.com/openshift-metalkube/dev-scripts/pull/204/)?  The dev-scripts change is needed to start the httpd container, which sounds like its not happening.


Fixed broken links

Fixed the broken links in Contributing.md.
Decided to remove src/ and not add the current parent path, as I felt it added to much clutter. I'll happily add it if needed.

Thank you!


chore: update Azure Spring Apps management IPs

chore: update Azure Spring Apps management IPs

@karlerickson
Can you review the proposed changes?
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off