allocate merkleizers inside functions

Instead of passing a merkleizer into the SSZ serialization functions,
they now allocate the merkleizer themselves. This makes it easier to
re-use those functions in future code (e.g., merkle multiproof code).
Also homogenized a couple inconsistencies:

hash_tree_root forward declaration pragmas not matching impl.
totalChunks vs totalElements naming.
merkleizer.addChunk() vs merkleizer.addChunk without braackets.
merkleizer.getFinalHash() vs getFinalHash(merkleizer) for
non-template where the dot and bracket syntax were already both mixed.
mixInLength at caller side instead of callee side so that the func
can be re-used for non-root GeneralizedIndex in the future.
Pull in List handling into hashTreeRootAux next to array.
Only HashArray|HashList now have special treatment.
Make type order consistent with order of their definition.
typeof instead of type as this is more modern.

Followup from https://github.com/status-im/nimbus-eth2/pull/3016

Split up into multiple commits to simplify the review.

Also, changed merkleizer functions to always use function call syntax instead of the other way around.

I have extracted the non-controversial cosmetic changes to a separate PR here to reduce the noise: https://github.com/status-im/nim-ssz-serialization/pull/12


[BUG] Cannot build windows static library with embedded source

slang-bootstrap relies symbols from slang.dll. When you build it as a static library, on linux it somehow resolves those symbols, but on windows it cannot resolve them. (I say somehow, because they are different executables (my app and slang-bootstrap). I'd like an explanation, if anyone knows how)
Error:
20>------ Build started: Project: slang-bootstrap, Configuration: Debug x64 ------
20>main.obj : error LNK2001: unresolved external symbol __imp_spCompile
20>main.obj : error LNK2001: unresolved external symbol __imp_spSetDiagnosticCallback
20>main.obj : error LNK2001: unresolved external symbol __imp_slang_shutdown
20>main.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) void __cdecl spSetCommandLineCompilerMode(struct slang::ICompileRequest *)" (__imp_?spSetCommandLineCompilerMode@@YAXPEAUICompileRequest@slang@@@Z)
20>main.obj : error LNK2001: unresolved external symbol __imp_spCreateCompileRequest
20>main.obj : error LNK2001: unresolved external symbol __imp_slang_createGlobalSession
20>main.obj : error LNK2001: unresolved external symbol __imp_spDestroyCompileRequest
20>main.obj : error LNK2001: unresolved external symbol __imp_spProcessCommandLineArguments
20>main.obj : error LNK2001: unresolved external symbol __imp_slang_createGlobalSessionWithoutCoreModule
20>D:\DEV\Github\PK_Gen\.PROJ\generators\Debug\bin\slang-bootstrap.exe : fatal error LNK1120: 9 unresolved externals
20>Done building project "slang-bootstrap.vcxproj" -- FAILED.
21>------ Build started: Project: slang-embedded-core-module, Configuration: Debug x64 ------
21>Generating slang-core-module-generated.h
21>'..\..\..\..\generators\Debug\bin\slang-bootstrap.exe' is not recognized as an internal or external command,
21>operable program or batch file.
21>C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'D:\DEV\Github\PK_Gen\.PROJ\CMakeFiles\053d2a5848ae9d6f48c061efd95af32d\slang-core-module-generated.h.rule' exited with code 9009.
21>Done building project "slang-embedded-core-module.vcxproj" -- FAILED.
22>------ Build started: Project: PK_Gen, Configuration: Debug x64 ------
22>Scanning sources for module dependencies...
22>main.obj : error LNK2001: unresolved external symbol __imp_spReflection_ToJson
22>main.obj : error LNK2001: unresolved external symbol __imp_slang_createGlobalSession
22>D:\DEV\Github\PK_Gen\.PROJ\Debug\bin\PK_Gen.exe : fatal error LNK1120: 2 unresolved externals
22>Done building project "PK_Gen.vcxproj" -- FAILED.

I tried to workaround that by calling slangc instead of slang-bootstrap by modifying source/slang-core-module/CMakeLists.txt:19 from this:
add_custom_command(
    OUTPUT ${core_module_generated_header}
    COMMAND
        slang-bootstrap -archive-type riff-lz4 -save-core-module-bin-source
        ${core_module_generated_header}
    DEPENDS slang-bootstrap
    VERBATIM
)

to this:
add_custom_command(
    OUTPUT ${core_module_generated_header}
    COMMAND
        slangc -archive-type riff-lz4 -save-stdlib-bin-source
        ${core_module_generated_header}
    VERBATIM
)

But since I removed slang-bootstrap from dependencies, my own app had unresolved symbols (you can see it at the end of the error log).
I tried to workaround that by linking to slang-common-objects, it seemed like working, but when I run my app, it just stalled forever.
It turns out createGlobalSession() fails to load embedded core module binary at source/slang/slang-api.cpp:103. I don't think binary would be incompatible and slang-bootstrap was just renamed slangc, but since -save-core-module-bin-source was incompatible with slangc, I think bootstrap was custom :D
At the end, I couldn't get a workaround working, but I hope my little research helps. I could probably fix it and create a pull request, if I was more familiar with the project and slang's custom cmake stuff

Not I got around this issue https://github.com/shader-slang/slang/issues/5821#issue-2730516243 by doing this in top level cmake:
set(CMAKE_SKIP_INSTALL_RULES ON)


"No free output buffer slot" logspam in wayland backend

With recent changes, sway on the Wayland backend produces a lot of "No free output buffer slot" error messages.
For me, an easy way to reproduce is:

Start an embedded sway session with the Wayland backend
Open two terminals
Focus one of them and spam the move left|right sway command to swap the window places

I see a new "No free output buffer slot" message on most moves. Sway appears to work OK despite the errors.

23:25 <emersion> hm, i think it's the cursor
23:28 <emersion> the previous code just created and destroyed the EGLSurface each time the cursor was updated
23:28 <emersion> which wasn't great
23:28 <emersion> i think we'll want to do like the drm backend and update the cursor on output commit


why are errors swallowed?

shouldn't the callback be called here

Good catch. Yes, it looks like there isn't any test coverage for the code path where the normal sink does the abort, so abort works if you call abortable.abort() but not if the sink does the abort.
If you can make a test case for this, I can help you fix it.


'errors' is an invalid keyword argument for this function

I am take this error.I loaded kickthemout and python-nmap.
I performed this step:
pip install -r ...
But doesn't run.
what should I do ?

You should do
§ pip install -r requirements.txt

had the same exact problem,please answer with a solution


feat: adds ability to share fragments across components

(e.g.) being able to utilize userFieldsFragment in query or other fragments such as the one below.
import { gql } from '@data-eden/codegen/gql';
import { userFieldsFragment } from './UserView.tsx';

const findUserQuery = gql`query findUser($userId: ID!) {
  user(id: $userId) {
    \${userFieldsFragment}
  }
}

fixes: getDefinition to throw an error that showcases the location of the error
chore: updates implementation for codegen in the readme
feat: enables the ability to import fragments from compnents and use them in queries
chore: adds test to validate sharing fragments between components
chore: adds a test that validates sharing fragments between components nested
fix(bug): properly thread through filename to getDefinition as documentNode does not
maintain ownership
feat: adds cli logging when complete
fix(bug): ensure we do not include duplicate types that are exported
fix: double fragment exports


Unable to load bootsrap.dll

Hello, I'm trying to deply unpackaged app, but receave this error: : 'Unable to load DLL 'Microsoft.WindowsAppRuntime.Bootstrap.dll' (0x8007007E)'.
Sdk .401
Microsoft.WindowsAppSDK 1.0.0-preview1
Visual studio 2022
I followed this guide.

Thank you for the reply. But I still barely understand what I should to do. I should just wait for preview 2?

Unpackaged WinUI3 apps aren't quite ready yet. We're working hard to connect the last dots for Preview3 so please stay tuned.


Fix TX ID bits 18-20 of mcp2515 can driver

Summary
Fix TX ID bits 18-20 of mcp2515 can driver, register TXB0SIDL is being filled inappropriately, values being overwritten.
Impact
Driver.
Testing
With mcp2515 shield in stm32h7.

Hi @robsonsopran thanks for finding this issue! There is a long line in the file:
drivers/can/mcp2515.c:1562:93: error: Long line found
This line was already there, but could you please break this line to be <=78 characters?

@acassis the long line is hard to fix, because it's a URL:
https://github.com/apache/incubator-nuttx/blob/master/drivers/can/mcp2515.c#L1562
Anyway, let's merge this PR and come up a fix later.


Update dependencies

Pull requests:

https://github.com/senzing-garage/validate/pull/75
https://github.com/senzing-garage/validate/pull/77


[Enhancement & Feature]: Drop down list for font-family & font-size. Option to insert table.

I'm submitting a

[ ] Regression (a behavior that used to work and stopped working in a new release)
[ ] Bug report
[x] Enhancement
[x] Feature request
[ ] Documentation issue or request

Description
It would be very helpful if you add drop down list for font-family and font-size.  From where we can choose necessary font family and font size. Option to insert table would be a very good feature.
Version Information
ngx-editor: 3.3.0
angular: 5.2.10

Browser

[x] Chrome - x.x.x
[x]  Mozilla Firefox - x.x.x
[x] Opera - x.x.x
[x] Microsoft Edge - x.x.x

Font size/family can be implemented using custom menu in v5. Table support will be added eventually.


Incorrect types of auto-imports in modules

Environment

Operating System: Darwin
Node Version:     v20.5.1
Nuxt Version:     3.7.0
CLI Version:      3.7.3
Nitro Version:    2.6.2
Package Manager<EMAIL_ADDRESS>Builder:          -
User Config:      modules, auth, security, typescript, devtools, i18n, postcss, tailwindcss, vue, vite
Runtime Modules<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Build Modules:    -

Reproduction
https://stackblitz.com/edit/nuxt-starter-xodh6t?file=modules%2Fexample%2Fpages%2Fsomething%2Findex.vue
Describe the bug
There are missing types from auto-imports, and that the ref, $fetch, or other variables are being inferred as any in modules.
Additional context
No response
Logs
No response

This is caused by the fact that we are excluding modules from being covered from the generated tsconfig.
I still think that's a good idea, but it might be worth either adding any added pages/composables/etc. base directories, or possibly automatically including a ./runtime path within modules (though I realise that's opinionated).
thoughts @pi0 @lihbr @Atinux?

I second @lihbr , I like the ./runtime convention. And well, Nuxt is convention over configuration :)

Okay, let's do it. We can add ./runtime path of local module directories to tsconfig.json, while adding ./runtime/server to exclude - and instead add to to the include of tsconfig.server.json.


Expose FileReader & way to create File instances

At the moment there is no way to read local files without resorting to node's require. I think it would be awesome if there was a way to use FileReader and File APIs for that purpose. Although one would obviously need a way to create a File instance from the file URL somehow.
It might be worth consulting https://wicg.github.io/writable-files/ spec.
I would also really love if we could somehow do a better job on sandboxing. My immediate idea would be to have another argument for webrun that would restrict fs access to specific paths. For example:
webrun main.js --read+write ./store/

That would allow main.js to read write under the given path but nowhere else. It could further be restricted to the code pulled in for example:
import staticServer from "./server?read=./store"

staticServer would not be capable of writing files anywhere but will be able to read from ./store.
It might be also worth checking how https://deno.land/ is going about it.

What do you think about a manifest or something for permissions?

I really want to add the new files API, but I don't think there's a node.js implementation available yet.

What do you think about a manifest or something for permissions?

Manifest is usually per application, but what if you want to use some module without giving it access to your filesystem ?

I really want to add the new files API, but I don't think there's a node.js implementation available yet.

I hacked something together for my use case for now

What about something like --allow-read and --allow-write which default to reading from the local directory or setting --allow-read=./store for a specific directory?

What about something like --allow-read and --allow-write which default to reading from the local directory or setting --allow-read=./store for a specific directory?

I'm honestly not sure. It's also unsure if sandboxing per module even makes sense, usually it's per process so maybe that's a better way to go about it.
I also looked at deno & approach there is pretty interesting. If you attempt to start a server it will request permission in CLI & you can provide CLI arg to grant permissions ahead of time, see:
https://github.com/denoland/deno/blob/master/Docs.md#tcp-echo-server
That is probably a good model and matches more or less the web.

Yeah, I was thinking per process, not module. Prompting is a pretty good idea. With regards to directories, I could see it being pretty annoying if it's asked a lot. Maybe you need to specify a path in the writable files API and that's when it asks?

Here is the list of APIs that I would really like to see that are related to working with files

[ ] Blob

[ ] File API allows file construction & reading. It also can be user as response / request body.

I think one way to possibly expose FS in web compatible manner would be through something like self.postMessage({ type:"fs.write", url: new URL("file:///...."), file: new File(...) }).

[ ] FileReader Not a great API but at least web compatible. It also seems to be embraced by https://wicg.github.io/writable-files/

[] FileReaderSync Not sure about this one, but node allow sync IO and browsers do allow it in web workers, so it might be reasonable request.


"graphical glitches" in grid/flow-layouts (wrap into next column)

The grid/flow-layouts have graphical glitches on chrome (the letter of the "avatar" wraps into the next column) and only layouts as a single column in FF. We could try the css-grid-layout here, that should be supported by almost all browsers by now.

For FF one fix is to set the flow-items to overflow: hidden so the layouting engine treats it as "monolithic" (as explained in this stack-overflow thread)

In general the support for break-inside: avoid & co seems to be rather fragile / uneven atm (read "it's there, but...")


Cannot connect to database: ArgumentError: wrong number of arguments (given 4, expected 1..3)

activerecord-jdbc-adapter 70.1
jRuby <IP_ADDRESS>
Rails (ActiveRecord) <IP_ADDRESS>
All attempts to connect to a MySQL db fails with ArgumentError: wrong number of arguments (given 4, expected 1..3).
Possibly related to an existing (closed but apparently unsolved) issue: ArgumentError: wrong number of arguments (given 4, expected 1..3) #1013
Backtrace:
# /app/gems/gems/activerecord-jdbc-adapter-70.1-java/lib/arjdbc/abstract/core.rb:12:in `initialize'
# /app/gems/gems/activerecord-jdbc-adapter-70.1-java/lib/arjdbc/abstract/statement_cache.rb:23:in `initialize'
# /app/gems/gems/activerecord-jdbc-adapter-70.1-java/lib/arjdbc/mysql/adapter.rb:38:in `initialize'
# /app/gems/gems/activerecord-jdbc-adapter-70.1-java/lib/arjdbc/jdbc/callbacks.rb:14:in `new'
# /app/gems/gems/activerecord-jdbc-adapter-70.1-java/lib/arjdbc/jdbc/connection_methods.rb:10:in `jdbc_connection'
# /app/gems/gems/activerecord-jdbc-adapter-70.1-java/lib/arjdbc/mysql/connection_methods.rb:101:in `mysql_connection'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/connection_pool.rb:676:in `new_connection'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/connection_pool.rb:723:in `checkout_new_connection'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/connection_pool.rb:702:in `try_to_checkout_new_connection'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/connection_pool.rb:654:in `acquire_connection'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/connection_pool.rb:353:in `checkout'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/connection_adapters/abstract/connection_pool.rb:182:in `connection'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/tasks/database_tasks.rb:501:in `block in with_temporary_connection'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/tasks/database_tasks.rb:518:in `with_temporary_pool'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/tasks/database_tasks.rb:500:in `with_temporary_connection'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/tasks/database_tasks.rb:377:in `schema_up_to_date?'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/migration.rb:762:in `block in any_schema_needs_update?'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/migration.rb:761:in `any_schema_needs_update?'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/migration.rb:715:in `load_schema_if_pending!'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/migration.rb:730:in `block in maintain_test_schema!'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/migration.rb:1026:in `suppress_messages'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/migration.rb:735:in `method_missing'
# /app/gems/gems/activerecord-<IP_ADDRESS>/lib/active_record/migration.rb:730:in `maintain_test_schema!'

Hi @number61971,
It looks like you are trying to use the 70.1 version of this gem with rails 7.1. It doesn't currently support 7.1 but there is work in the master branch towards compatibility. Once we have compatibility, a 71.0 version of the gem will be released. We unfortunately haven't made much progress on the mysql or Postgres adapters though.

Hi @rdubya,
Thank you for the prompt reply! I didn't realize that version numbers tracked like that. Does this mean I have to stick to Rails 7.0.x?

Unfortunately, yeah. Until we get more help with the gem we're kind of stuck with slow progress.


airportitlwm, Intel Wifi Card and Wifi 6. A solution

I noticed that some Intel Cards (dont know which ones to be exact) dont mix well with airportitlwm and wifi 6.
Symptoms only arise when using integrated wifi such as having inconsistent download speeds from 250mbit down to 0.05 mbit.
A solution to fix this issue at the cost of some features is to disable wifi 6 alltogether. Do this by adding -novht to your boot args. Straightforward on how to do it. Just add a whitespace after the last argument in the value.
I sadly had no luck by using only 2,4ghz as mentioned by others in other forums. Only this fixed the issue so far.

The speed issues sound similar to #113 🤔 @bobmarley2021

Sounds like it

Thanks for tagging me ellnic, unfortunately this doesn't seem to help my issue. I actually get pretty good speeds when uploading on the 5Ghz band, ~100mbit, just not downloading. I am using the 2.4Ghz band for now and that gives me about 100 each way. I tried the stock intel card, but I miss Airdrop and the universal clipboard. I also purchased a BCM94360NG but I get sketchy BT audio and no Airdrop. It's a real shame about the stock intel card because I get about 600 both ways....


CI fails when multiple NBs write to same module

Hi, I think I've found a bug during CI due to the # AUTOGENERATED! DO NOT EDIT! File to edit:... warning in modules being updated multiple times (or in an inconsistent order) when multiple notebooks write to the same module. This then gets caught as a change by nbdev_diff_nbs during CI:
Run if [ -n "$(nbdev_diff_nbs)" ]; then echo -e "$(nbdev_diff_nbs)" "!!! Detected difference between the notebooks and the library"; false; fi
  if [ -n "$(nbdev_diff_nbs)" ]; then echo -e "$(nbdev_diff_nbs)" "!!! Detected difference between the notebooks and the library"; false; fi
  shell: /bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.6.13/x64
diff -ru /tmp/tmpcx95vd8w/classifier.py /tmp/tmpdcz9c5nv/classifier.py
--- /tmp/tmpcx95vd8w/classifier.py	2021-02-24 13:27:15.000000000 +0000
+++ /tmp/tmpdcz9c5nv/classifier.py	2021-02-24 13:29:12.000000000 +0000
@@ -1,4 +1,4 @@
-# AUTOGENERATED! DO NOT EDIT! File to edit: nbs/02_feature_selection.ipynb (unless otherwise specified).
+# AUTOGENERATED! DO NOT EDIT! File to edit: nbs/05_feature_interdependence.ipynb (unless otherwise specified).

 __all__ = ['optimise_classifier_rf', 'evaluate_classifier_preds', 'plot_rf_feat_importance', 'plot_partials',
            'filtered_feats', 'ignore_feats']
diff -ru /tmp/tmpcx95vd8w/data.py /tmp/tmpdcz9c5nv/data.py
--- /tmp/tmpcx95vd8w/data.py	2021-02-24 13:27:15.000000000 +0000
+++ /tmp/tmpdcz9c5nv/data.py	2021-02-24 13:29:13.000000000 +0000
@@ -1,4 +1,4 @@
-# AUTOGENERATED! DO NOT EDIT! File to edit: nbs/00_data.ipynb (unless otherwise specified).
+# AUTOGENERATED! DO NOT EDIT! File to edit: nbs/09_regression_data.ipynb (unless otherwise specified).

 __all__ = ['PATH', 'get_data', 'get_df', 'split_df', 'cont_feats', 'create_data', 'reg_targ_feats', 'reg_cont_feats',
            'preproc_reg_inputs', 'preproc_reg_targets', 'create_reg_data']
diff -ru /tmp/tmpcx95vd8w/inference.py /tmp/tmpdcz9c5nv/inference.py
--- /tmp/tmpcx95vd8w/inference.py	2021-02-24 13:27:15.000000000 +0000
+++ /tmp/tmpdcz9c5nv/inference.py	2021-02-24 13:29:13.000000000 +0000
@@ -1,4 +1,4 @@
-# AUTOGENERATED! DO NOT EDIT! File to edit: nbs/12_dnn_on_systs.ipynb (unless otherwise specified).
+# AUTOGENERATED! DO NOT EDIT! File to edit: nbs/11_rf_on_systs.ipynb (unless otherwise specified).

 __all__ = ['get_syst_dfs', 'plot_systs', 'run_inference']
  !!! Detected difference between the notebooks and the library
Error: Process completed with exit code 1.

Strangely, though, I'm not able to reproduce this locally, where running nbdev_diff_nbs returns no error. The starting values of the AUTOGENERATED warnings are the same as those written in my modules, so at some point they get changed.
Which notebook gets listed in the warning also seems to change between modules; sometimes it is the first notebook to write to the module, and other times it is the last. However when I locally build the library the order is always opposite to when they are recreated during CI (note how nbs/02_feature_selection.ipynb and nbs/00_data.ipynb get changed for nbs/05_feature_interdependence.ipynb and nbs/09_regression_data.ipynb, i.e. swapping "first to write" for "last to write", but nbs/12_dnn_on_systs.ipynb gets swapped for nbs/11_rf_on_systs.ipynb, i.e. "last to write" for "first to write").
I assume that nbdev_diff_nbs works by re-exporting the notebooks to a seperate area and then comparing the resulting files with the existing ones to check for differences, so could the export order change somehow? Locally I'm using MacOS 10.15, but CI is on Ubuntu-latest.
Thanks in advance for any help.

👋🏽 @GilesStrong can you share a minimal reproducible example that would be helpful.  Can you provide a public repo with just two notebooks that reproduce this problem?  Thank you

Sure, will do!

Ok, I've setup a minimal repo that reproduces the problem: https://github.com/GilesStrong/nbdev_test
The two notebooks in nbs both write to the same module, but the file listed in the AUTOGENERATED warning changes during CI due to nbdev_diff_nbs, resulting in an error (see https://github.com/GilesStrong/nbdev_test/runs/1986723916?check_suite_focus=true).
This happens even if I run nbdev_build_lib on Ubuntu-16 and then run the CI on Ubuntu-16, so I don't think it's architecture dependent. Could there be a difference in the exporting between nbdev_diff_nbs and nbdev_build_lib?

Update on this, I think the problem stems from notebooks that write to the same module not having sequential name IDs:

In https://github.com/GilesStrong/nbdev_test, notebooks 00_data.ipynb and 09_regression_data.ipynb both write to data.py and nbdev_diff_nbs claims that there are differences between the notebooks and the modules: https://github.com/GilesStrong/nbdev_test/runs/2021243457?check_suite_focus=true
In https://github.com/GilesStrong/nbdev_test/tree/sequential_naming, I rename 09_regression_data.ipynb to 01_regression_data.ipynb and nbdev_diff_nbs no longer claims that there are differences between the notebooks and the modules and all checks pass: https://github.com/GilesStrong/nbdev_test/runs/2021255349?check_suite_focus=true

I'll begin looking into how nbdev_diff_nbs runs.


[BUG] error with terraform-provider-aws and terranova

hi, I run into the following error. maybe someone knows why.
Describe the bug
Build error with terraform-providers
To Reproduce
Steps to reproduce the behavior. Example:

I use go 1.14.
my go.mod file:

module github.com/blabla/testapp

go 1.14

require (
	github.com/johandry/terranova v0.0.4 // indirect
	github.com/terraform-providers/terraform-provider-aws v1.60.0 // indirect
	golang.org/x/crypto v0.0.0-20200604202706-70a84ac30bf9 // indirect
)

here the error:
$ go build main.go
# github.com/terraform-providers/terraform-provider-aws/aws
/Users/arnold1/go/pkg/mod/github.com/terraform-providers/terraform-provider-aws@v1.60.0/aws/resource_aws_glue_crawler.go:369:47: cannot convert *crawlerInput (type gl
ue.CreateCrawlerInput) to type glue.UpdateCrawlerInput
/Users/arnold1/go/pkg/mod/github.com/terraform-providers/terraform-provider-aws@v1.60.0/aws/structure.go:4874:7: spec.ServiceNames undefined (type *appmesh.VirtualRou
terSpec has no field or method ServiceNames)
/Users/arnold1/go/pkg/mod/github.com/terraform-providers/terraform-provider-aws@v1.60.0/aws/structure.go:4886:41: spec.ServiceNames undefined (type *appmesh.VirtualRo
uterSpec has no field or method ServiceNames)
/Users/arnold1/go/pkg/mod/github.com/terraform-providers/terraform-provider-aws@v1.60.0/aws/structure.go:4902:17: cannot use expandStringSet(vBackends) (type []*strin
g) as type []*appmesh.Backend in assignment
/Users/arnold1/go/pkg/mod/github.com/terraform-providers/terraform-provider-aws@v1.60.0/aws/structure.go:4969:7: unknown field 'ServiceName' in struct literal of type
 appmesh.DnsServiceDiscovery
/Users/arnold1/go/pkg/mod/github.com/terraform-providers/terraform-provider-aws@v1.60.0/aws/structure.go:4987:44: cannot use spec.Backends (type []*appmesh.Backend) a
s type []*string in argument to flattenStringSet
/Users/arnold1/go/pkg/mod/github.com/terraform-providers/terraform-provider-aws@v1.60.0/aws/structure.go:5028:64: spec.ServiceDiscovery.Dns.ServiceName undefined (typ
e *appmesh.DnsServiceDiscovery has no field or method ServiceName)

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots or logs
If applicable, add screenshots to help explain your problem.
For lengthy log-files, consider posting them as a gist (https://gist.github.com) or use services line Dropbox and share a link to the Zip file.
Don't forget to remove sensitive data from your logfiles before posting. You can replace those parts with "REDACTED".
Platforms
List the platforms Terraformer is building, modifying or destroying. Include version and the GitHub repo.
AWS:

v2.1.0
github.com/terraform-providers/terraform-provider-aws

Versions:

Terraformer: [e.g. v0.0.1]
Terraform: [e.g. v0.12.2]

Environment running Terraformer:

OS: [e.g. macOS]:

Darwin MacBook-Pro.local 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64

Version: [e.g. 10.14.6]

Additional context:
Add any other context about the problem here.
References:
List any other GitHub issues (open or closed) or Pull Requests that should be linked to this bug. For example:

#6017

@johandry ..any update please

In your go.mod file, try replacing the line github.com/terraform-providers/terraform-provider-aws v1.60.0 to:
github.com/terraform-providers/terraform-provider-aws v1.60.1-0.20191003145700-f8707a46c6ec

Then run go mod tidy to validate the modules version

If you use other modules, please, refer to the section Providers Version in the README: https://github.com/johandry/terranova#providers-version
Other good place to check compatible version and other samples, is the repo johandry/terranova-examples
Let me know if the version does not work or any other issue ... and sorry about the delay 😞

FYI: 01 and 02 examples did not build but 03 at least builds for now.
01_simple (master) $ make
go build .
go: downloading github.com/terraform-providers/terraform-provider-aws v1.60.1-0.20191003145700-f8707a46c6ec
go: downloading github.com/johandry/terranova v0.0.3
go: downloading github.com/zclconf/go-cty v1.1.0
go: downloading github.com/hashicorp/terraform v0.12.12
go: downloading github.com/aws/aws-sdk-go v1.25.4
go: downloading github.com/hashicorp/aws-sdk-go-base v0.4.0
go: downloading github.com/hashicorp/hcl/v2 v2.0.0
go: downloading golang.org/x/crypto v0.0.0-20190911031432-227b76d455e7
go: downloading github.com/hashicorp/go-getter v1.4.0
go: downloading github.com/mattn/go-isatty v0.0.8
go: downloading github.com/hashicorp/terraform-config-inspect v0.0.0-20190821133035-82a99dc22ef4
go: downloading github.com/mattn/go-colorable v0.1.2
go: downloading github.com/hashicorp/hcl2 v0.0.0-20190821123243-0c888d1241f6
# github.com/johandry/terranova-examples/01_simple/simple
./main.go:24:46: terranova.NewPlatform(code).SetMiddleware undefined (type *terranova.Platform has no field or method SetMiddleware)
make: *** [build] Error 2

go build -o ec2 .
go: downloading github.com/hashicorp/terraform v0.12.17
go: downloading github.com/xanzy/ssh-agent v0.2.1
go: downloading github.com/masterzen/winrm v0.0.0-20190223112901-5e5c9a7fe54b
go: downloading github.com/packer-community/winrmcp v0.0.0-20180102160824-81144009af58
go: downloading github.com/dylanmei/iso8601 v0.1.0
go: downloading github.com/nu7hatch/gouuid v0.0.0-20131221200532-179d4d0c4d8d
go: downloading github.com/Azure/go-ntlmssp v0.0.0-20180810175552-4a21cbd618b4
go: downloading github.com/ChrisTrenkamp/goxpath v0.0.0-20170922090931-c385f95c6022
go: downloading github.com/masterzen/simplexml v0.0.0-20160608183007-4572e39b1ab9
# github.com/johandry/terranova-examples/02_ec2/ec2
./main.go:56:46: terranova.NewPlatform(code).SetMiddleware undefined (type *terranova.Platform has no field or method SetMiddleware)
make: *** [build] Error 2

go build -o tf .
go: downloading github.com/hashicorp/terraform v0.11.13
go: downloading github.com/aws/aws-sdk-go v1.17.0
go: downloading github.com/mitchellh/go-homedir v1.0.0
go: downloading github.com/hashicorp/hcl v0.0.0-20170504190234-a4b07c25de5f
go: downloading github.com/hashicorp/hcl2 v0.0.0-20180308163058-5f8ed954abd8
go: downloading github.com/mitchellh/cli v0.0.0-20171129193617-33edc47170b5
go: downloading github.com/hashicorp/hil v0.0.0-20170627220502-fa9f258a9250
go: downloading github.com/hashicorp/go-cleanhttp v0.5.0
go: downloading github.com/hashicorp/go-uuid v1.0.0
go: downloading github.com/hashicorp/go-version v1.0.0
go: downloading github.com/zclconf/go-cty v0.0.0-20180302160414-49fa5e03c418
go: downloading github.com/posener/complete v0.0.0-20171219111128-6bee943216c8
go: downloading golang.org/x/crypto v0.0.0-20180211211603-9de5f2eaf759
go: downloading github.com/mattn/go-isatty v0.0.4
go: downloading gopkg.in/yaml.v2 v2.2.1
go: downloading github.com/apparentlymart/go-cidr v1.0.0
go: downloading golang.org/x/net v0.0.0-20171024115130-4b14673ba32b
go: downloading github.com/hashicorp/go-getter v0.0.0-20180327010114-90bb99a48d86
go: downloading github.com/mitchellh/mapstructure v1.0.0
go: downloading github.com/masterzen/winrm v0.0.0-20180224160350-7e40f93ae939
go: downloading github.com/xanzy/ssh-agent v0.2.0
go: downloading golang.org/x/text v0.0.0-20171024115504-6eab0e8f74e8
go: downloading github.com/agext/levenshtein v1.2.1
go: downloading github.com/ChrisTrenkamp/goxpath v0.0.0-20170625215350-4fe035839290
go: downloading github.com/ulikunitz/xz v0.5.4
go: downloading github.com/Azure/go-ntlmssp v0.0.0-20170803034930-c92175d54006
go: downloading github.com/masterzen/azure-sdk-for-go v0.0.0-20161014135628-ee4f0065d00c
go: downloading github.com/mitchellh/go-testing-interface v0.0.0-20171004221916-a61a99592b77

Update 01_simple builds with the changes:
01_simple (master) $ git diff main.go
diff --git a/01_simple/main.go b/01_simple/main.go
index 2d5ca7a..d2dc2df 100644
--- a/01_simple/main.go
+++ b/01_simple/main.go
@@ -22,11 +22,11 @@ func main() {
        defer logMiddleware.Close()
        platform, err := terranova.NewPlatform(code).
-               SetMiddleware(logMiddleware).
+               AddMiddleware(logMiddleware).
                AddProvider("aws", aws.Provider()).
                Var("c", count).
                Var("key_name", keyName).
-               PersistStateToFile(stateFilename)
+               WriteStateToFile(stateFilename)```

Thanks for let me know, I'll fix the examples.
I try to update them all every time there is a new release ... I'll add CI/CD with GH Actions to catch these errors

The examples were updated, validated and they are running.
If there is an issue with the examples, please, open an issue in the Terranova-examples repository.
I'm closing this issue as the solution it's documented in the README and here in the issue.


Allow user defined exceptions to have docstrings

For #2016

Sorry it took me a while to get to this, thanks for implementing! Got a few ideas and comments r.e. the new_type stuff

I have combined the functions and made it return Result<Py>. I think this is good to go after the 1.57 changes merge.

Sorry for the catastrophically slow reviewing. Looks great to me! Just needs rebasing and a CHANGELOG entry :)

Sorry for the catastrophically slow reviewing. Looks great to me! Just needs rebasing and a CHANGELOG entry :)

Should be good to go now :)

Any chance we could cut a release with this feature? I forked pyo3 to add this but it's already here, just unreleased.

@9999years please follow #2079 - we're getting close to last piece we want to complete before next release lands. You might be able to use a git dependency in the meanwhile?


Example of a OFX2DataSet feature

Removed some warnings on lib/ofxreader.pas;
Added the Example2.pas to show how to convert from OFX to a TDataSet;

Thanks @vstiebe


Jasper not recognizing its name

After days I finally got jasper.py to run (somewhat). Sadly Jasper ain't seem to hear its name when I'm calling it. The only word that's correctly recognized seems to be say (from what I can tell from the console output). Also there's lots of bad looking output before jasper starts up.
I've virtually no knowledge of alsa. My configs are just what I collected from the web.
Any help is appreciated. I'm close to dropping the whole thing again. Installation page is a mess and not a thing is working. But I feel so close now ...

        JASPER - THE TALKING COMPUTER           *

(c) 2015 Shubhro Saha, Charlie Marsh & Jan Holthuis *

ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.front.0:CARD=0'
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM front
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM surround21
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM surround21
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.surround40.0:CARD=0'
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM surround40
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM surround41
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM surround50
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.surround51.0:CARD=0'
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM surround51
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.surround71.0:CARD=0'
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM surround71
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM iec958
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM spdif
ALSA lib confmisc.c:1286:(snd_func_refer) Unable to find definition 'cards.bcm2835.pcm.iec958.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM spdif
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm_hw.c:1667:(_snd_pcm_hw_open) Invalid value for card
ALSA lib pcm_hw.c:1667:(_snd_pcm_hw_open) Invalid value for card
ALSA lib pcm_hw.c:1667:(_snd_pcm_hw_open) Invalid value for card
ALSA lib pcm_hw.c:1667:(_snd_pcm_hw_open) Invalid value for card
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
WARNING:client.notifier:gmail_address or gmail_password not set in profile, Gmail notifier will not be used
INFO:client.stt:Transcribed: ['']
INFO:client.stt:Transcribed: ['']
aplay -L:

null
Discard all samples (playback) or generate zero samples (capture)
default
sysdefault:CARD=ALSA
bcm2835 ALSA, bcm2835 ALSA
Default Audio Device
dmix:CARD=ALSA,DEV=0
bcm2835 ALSA, bcm2835 ALSA
Direct sample mixing device
dmix:CARD=ALSA,DEV=1
bcm2835 ALSA, bcm2835 IEC958/HDMI
Direct sample mixing device
dsnoop:CARD=ALSA,DEV=0
bcm2835 ALSA, bcm2835 ALSA
Direct sample snooping device
dsnoop:CARD=ALSA,DEV=1
bcm2835 ALSA, bcm2835 IEC958/HDMI
Direct sample snooping device
hw:CARD=ALSA,DEV=0
bcm2835 ALSA, bcm2835 ALSA
Direct hardware device without any conversions
hw:CARD=ALSA,DEV=1
bcm2835 ALSA, bcm2835 IEC958/HDMI
Direct hardware device without any conversions
plughw:CARD=ALSA,DEV=0
bcm2835 ALSA, bcm2835 ALSA
Hardware device with all software conversions
plughw:CARD=ALSA,DEV=1
bcm2835 ALSA, bcm2835 IEC958/HDMI
Hardware device with all software conversions
sysdefault:CARD=Device
Generic USB Audio Device, USB Audio
Default Audio Device
front:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Front speakers
surround21:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
2.1 Surround output to Front and Subwoofer speakers
surround40:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
4.0 Surround output to Front and Rear speakers
surround41:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
5.1 Surround output to Front, Center, Rear and Subwoofer speakers
surround71:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
7.1 Surround output to Front, Center, Side, Rear and Woofer speakers
iec958:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
IEC958 (S/PDIF) Digital Audio Output
dmix:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Direct sample mixing device
dsnoop:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Direct sample snooping device
hw:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Direct hardware device without any conversions
plughw:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Hardware device with all software conversions

arecord -L:

null
Discard all samples (playback) or generate zero samples (capture)
default
sysdefault:CARD=Device
Generic USB Audio Device, USB Audio
Default Audio Device
front:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Front speakers
surround21:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
2.1 Surround output to Front and Subwoofer speakers
surround40:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
4.0 Surround output to Front and Rear speakers
surround41:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
5.1 Surround output to Front, Center, Rear and Subwoofer speakers
surround71:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
7.1 Surround output to Front, Center, Side, Rear and Woofer speakers
iec958:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
IEC958 (S/PDIF) Digital Audio Output
dmix:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Direct sample mixing device
dsnoop:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Direct sample snooping device
hw:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Direct hardware device without any conversions
plughw:CARD=Device,DEV=0
Generic USB Audio Device, USB Audio
Hardware device with all software conversions

cat /etc/modprobe.d/alsa-base.conf

options snd-usb-audio index=1

cat ~/.asoundrc

pcm.!default {
type asym
playback.pcm {
type plug
slave.pcm "hw:1,0"
}
capture.pcm {
type plug
slave.pcm "hw:1,0"
}
}

cat /etc/asound.conf

#options snd-usb-audio index=-2
#options snd-hda-intel index=-2
#options bt87x index=-2
#options cx88_alsa index=-2
#options saa7134-alsa index=-2
#options snd-atiixp-modem index=-2
#options snd-intel8x0m index=-2
#options snd-via82xx-modem index=-2
#options snd-usb-caiaq index=-2
#options snd-usb-ua101 index=-2
#options snd-usb-us122l index=-2
#options snd-usb-usx2y index=-2
#options snd-pcsp index=0
pcm.usb
{
type hw
card sysdefault
}
pcm.internal
{
type hw
card sysdefault
}
pcm.!default
{
type asym
playback.pcm
{
type plug
slave.pcm "internal"
}
capture.pcm
{
type plug
slave.pcm "usb"
}
}
ctl.!default
{
type asym
playback.pcm
{
type plug
slave.pcm "internal"
}
capture.pcm
{
type plug
slave.pcm "usb"
}
}

Have you tested your audio with a record/playback?  If not I have a few tools for testing in the Tools repo. This is a configuration issue not a code issue.
Tools Repo
The script is audiotest.sh

@mattcurrycom sorry for hitting the wrong channel. What's the right place to ask for help? Your RPI-Jasper-Dependencies already helped a lot, thanks for that.
audiotest.sh never returns from recording:

Recording WAVE '/tmp/3secondtest.wav' : Signed 16 bit Little Endian, Rate 44100 Hz, Stereo

gotta stop by crtl+c

^CAborted by signal Interrupt...
playing back audio test
/tmp/3secondtest.wav: No such file or directory

yeah means your dev is not likely setup as default.  Use the forum for help; or the slack room I just created.  Someone will have to invite you.  I can if you send me your email.


GraphQL release

Coverage decreased (-0.1%) to 88.472% when pulling 8a272c38ef4335be5fee64f6b895d762403b8bdd on graphql-release into 0b133bf49ed54a4fec76631e6c12df1438cdf09c on preprod.

PR not ready for demo


language support

Please add Chinese to copilot.When I ask copilot in Chinese,he often answers me in English.

Thank you for taking my advice.Copilot now responses me in Chinese.


Improve Google PageSpeed score

At the moment, the inclusions of selectr.min.css and datepicker.min.css are affecting loading times.

The best way to do this would be to inline the CSS within the same pipeline that #4 would create. This is on halt until #4 is completed.

Fixed with the resolution of #4


Migration extension summary and dms page fixes

This PR fixes.
Summary label
DMS creation retry times
DMS dropdown resetting every time we enter the page.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 41.71%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
26419

Relevant Lines:
57475

💛  - Coveralls


FIP21- Add staking to FIO protocol

add new staking contract to core.

Is this just needed for people spinning up a new chain or...? I thought all the contracts were just defined on chain as contracts. This doesn't mean nodes will have to upgrade, does it? Thanks for clarifying.

@lukestokes these changes will provide a process whereby to add a new contract to the Fio Protocol the following steps will be performed by BPs via MSIG.. first, use addaction to add the new actions in the contract to the Fio Protocol. second use set code and setabi to perform the establishment of the new contract. Nodes will have to upgrade in order to eliminate the present need in the Fio Protocol to modify the core to add new system contracts to the core. After this upgrade NO core upgrades will be needed to add any new system contracts to the Fio Protocol

the changes completed here have been put into a new "fresh" dev branch, this pr can be closed, this branch can be removed


feat: Add optional "logging.enabled" flag to module logging input

Description

When iterating over a list of buckets to create in AWS S3 multiple module instantiations must be created with conditionals for buckets that desire logging and buckets that do not. Adding the the parameter logging.enabled will make this have a more DRY approach.
Fixes #153
Motivation and Context

This change solves the problem where someone is looping over a list of buckets and some of them require logging and others do not. Currently this requires two separate module instantiations with for_each loops that filter based on some parameter added to the list of inputs.
Breaking Changes

No breaking changes. Flag is optional and defaulted to true to keep existing behavior identical with no changes required from existing users.
How Has This Been Tested?

[X] I have updated at least one of the examples/* to demonstrate and validate my change(s)
[X] I have tested and validated these changes using one or more of the provided examples/* projects

[X] I have executed pre-commit run -a on my pull request

Adding 'enabled' flag to the module is unnecessary because it is simpler not to include logging map completely.
There is also a wrapper which can be useful for your use-case:
module "wrapper" {
  source = "terraform-aws-modules/s3-bucket/aws//wrappers"

  items = {
    bucket1 = {
      bucket = "my-random-bucket-1"
      logging = {
        target_bucket = "my-logs"
        target_prefix = "log/"
      }
    }
    bucket2 = {
      bucket = "my-random-bucket-2"
      tags = {
        Secure = "probably"
      }
    }
  }
}

If you don't want to use wrapper, you have to create input maps using for loops and include/exclude logging values.

@antonbabenko I would really appreciate if you would reconsider this PR.  I finally got around to trying your wrapper method and while it is really great it still does not work with a config drive model.  One would still have to use local variables to generate the content input into the wrapper items {} block and if attempting to use the DAG by specifying the logging bucket as output from a separate instantiation of this module it will fail with
│ The "count" value depends on resource attributes that cannot be determined
│ until apply, so Terraform cannot predict how many instances will be
│ created. To work around this, use the -target argument to first apply only
│ the resources that the count depends on.

Could you provide the value for items which does not work now?


RNSVGImage is not fabric compatible

Bug
I'm using images in my SVGs extensively, but upgrading to fabric gives the RNSVGImage is not fabric compatible.
return (

<Image
x={x - maxSize / 2}
y={y - maxSize / 2}
width={maxSize}
height={maxSize}
href={IMAGES[dsoName].img}
opacity="0.6"
/>
<Circle r={maxSize * 0.75} fill={dsoFill} cx={x} cy={y} />

);
Environment info
Run react-native info in your terminal and copy the results here. Also, include the precise version number of this library that you are using in the project
React native info output:
System:
OS: macOS 12.5.1
CPU: (10) arm64 Apple M1 Pro
Memory: 91.70 MB / 32.00 GB
Shell: 5.8.1 - /bin/zsh
Binaries:
Node: 18.13.0 - /usr/local/bin/node
Yarn: Not Found
npm: 8.19.3 - /usr/local/bin/npm
Watchman: 2<IP_ADDRESS> - /opt/homebrew/bin/watchman
Managers:
CocoaPods: 1.11.3 - /opt/homebrew/lib/ruby/gems/3.0.0/bin/pod
SDKs:
iOS SDK:
Platforms: DriverKit 22.2, iOS 16.2, macOS 13.1, tvOS 16.1, watchOS 9.1
Android SDK:
API Levels: 27, 28, 29, 30, 31, 32, 33
Build Tools: 28.0.3, 29.0.2, 30.0.2, 30.0.3, 31.0.0, 32.0.0, 33.0.0, 33.0.1
System Images: android-29 | ARM 64 v8a, android-30 | Google Play ARM 64 v8a, android-31 | Google APIs ARM 64 v8a, android-32 | Google APIs ARM 64 v8a, android-33 | Google APIs ARM 64 v8a
Android NDK: Not Found
IDEs:
Android Studio: 2021.1 AI-211.7628.21.2111.8092744
Xcode: 14.2/14C18 - /usr/bin/xcodebuild
Languages:
Java: <IP_ADDRESS> - /usr/bin/javac
npmPackages:
@react-native-community/cli: Not Found
react: 18.2.0 => 18.2.0
react-native: 0.71.2 => 0.71.2
react-native-macos: Not Found
npmGlobalPackages:
react-native: Not Found
Library version: 13.8.0
Steps To Reproduce
Issues without reproduction steps or code are likely to stall.

This project is already support fabric!!!

this is my react-native info
System:
OS: macOS 12.5
CPU: (8) arm64 Apple M2
Memory: 5.31 GB / 24.00 GB
Shell: 5.8.1 - /bin/zsh
Binaries:
Node: 16.10.0 - ~/.nvm/versions/node/v16.10.0/bin/node
Yarn: 1.22.19 - ~/.nvm/versions/node/v16.10.0/bin/yarn
npm: 7.24.0 - ~/.nvm/versions/node/v16.10.0/bin/npm
Watchman: 2<IP_ADDRESS> - /opt/homebrew/bin/watchman
Managers:
CocoaPods: 1.12.0 - /Users/jeong-yeonho/.rbenv/shims/pod
SDKs:
iOS SDK:
Platforms: DriverKit 22.2, iOS 16.2, macOS 13.1, tvOS 16.1, watchOS 9.1
Android SDK: Not Found
IDEs:
Android Studio: 2022.1 AI-221.6008.13.2211.9477386
Xcode: 14.2/14C18 - /usr/bin/xcodebuild
Languages:
Java: 11.0.18 - /usr/bin/javac
npmPackages:
@react-native-community/cli: Not Found
react: 18.2.0 => 18.2.0
react-native: 0.71.6 => 0.71.6
react-native-macos: Not Found
npmGlobalPackages:
react-native: Not Found

This is my code
<WithLocalSvg
    asset={require('@assets/icons/text/trade_list.svg')}
    width={18}
    height={22}
  />

hmm... If you include png in svg, it seems that this error occurs. Could you please confirm that you are using an svg that contains a png inside it?
@kevin-williams

@kr-yeon i'm getting this error when i'm trying this approach
`import * as React from 'react';
import { SvgXml } from 'react-native-svg';
const xml =   <svg width="32" height="32" viewBox="0 0 32 32">     <path       fill-rule="evenodd"       clip-rule="evenodd"       fill="url(#gradient)"       d="M4 0C1.79086 0 0 1.79086 0 4V28C0 30.2091 1.79086 32 4 32H28C30.2091 32 32 30.2091 32 28V4C32 1.79086 30.2091 0 28 0H4ZM17 6C17 5.44772 17.4477 5 18 5H20C20.5523 5 21 5.44772 21 6V25C21 25.5523 20.5523 26 20 26H18C17.4477 26 17 25.5523 17 25V6ZM12 11C11.4477 11 11 11.4477 11 12V25C11 25.5523 11.4477 26 12 26H14C14.5523 26 15 25.5523 15 25V12C15 11.4477 14.5523 11 14 11H12ZM6 18C5.44772 18 5 18.4477 5 19V25C5 25.5523 5.44772 26 6 26H8C8.55228 26 9 25.5523 9 25V19C9 18.4477 8.55228 18 8 18H6ZM24 14C23.4477 14 23 14.4477 23 15V25C23 25.5523 23.4477 26 24 26H26C26.5523 26 27 25.5523 27 25V15C27 14.4477 26.5523 14 26 14H24Z"     />     <defs>       <linearGradient         id="gradient"         x1="0"         y1="0"         x2="8.46631"         y2="37.3364"         gradient-units="userSpaceOnUse">         <stop offset="0" stop-color="#FEA267" />         <stop offset="1" stop-color="#E75A4C" />       </linearGradient>     </defs>   </svg>;
export default () => ;`

@kr-yeon i'm getting this error when i'm trying this approach
import * as React from 'react';
import { SvgXml } from 'react-native-svg';

const xml = `
  <svg width="32" height="32" viewBox="0 0 32 32">
    <path
      fill-rule="evenodd"
      clip-rule="evenodd"
      fill="url(#gradient)"
      d="M4 0C1.79086 0 0 1.79086 0 4V28C0 30.2091 1.79086 32 4 32H28C30.2091 32 32 30.2091 32 28V4C32 1.79086 30.2091 0 28 0H4ZM17 6C17 5.44772 17.4477 5 18 5H20C20.5523 5 21 5.44772 21 6V25C21 25.5523 20.5523 26 20 26H18C17.4477 26 17 25.5523 17 25V6ZM12 11C11.4477 11 11 11.4477 11 12V25C11 25.5523 11.4477 26 12 26H14C14.5523 26 15 25.5523 15 25V12C15 11.4477 14.5523 11 14 11H12ZM6 18C5.44772 18 5 18.4477 5 19V25C5 25.5523 5.44772 26 6 26H8C8.55228 26 9 25.5523 9 25V19C9 18.4477 8.55228 18 8 18H6ZM24 14C23.4477 14 23 14.4477 23 15V25C23 25.5523 23.4477 26 24 26H26C26.5523 26 27 25.5523 27 25V15C27 14.4477 26.5523 14 26 14H24Z"
    />
    <defs>
      <linearGradient
        id="gradient"
        x1="0"
        y1="0"
        x2="8.46631"
        y2="37.3364"
        gradient-units="userSpaceOnUse">
        <stop offset="0" stop-color="#FEA267" />
        <stop offset="1" stop-color="#E75A4C" />
      </linearGradient>
    </defs>
  </svg>
`;

export default () => <SvgXml xml={xml} width="100%" height="100%" />;

the react-native last version(0.71.8) is disable newArchitecture. kkkk
can u disable neweArchitecture in gradle.properties?
i think they know that this are not perfect yet.

How about subtracting the linearGradient from the code above?

Idk it's broken on react-native@next

This is my code
<WithLocalSvg
    asset={require('@assets/icons/text/trade_list.svg')}
    width={18}
    height={22}
  />

hmm... If u include png in svg, it seems that this error occurs. Could u please confirm that u r using an svg that contains a png inside it? I don't get an error for things that don't have an Image tag inside. Could u check this out and expand on this issue? @kevin-williams

Yes I am using PNG for my render and still get the issue.

@kr-yeon
Yeah confirmed it still fails if I use a PNG in my  tag.
I downloaded an SVG file and use that directly instead and I still get the issue on android.  Haven't tested IOS yet.

It works under IOS though!

I tested again on RN 72.6 (upgraded from 7.24) and now it works on android and ios!

I will close this issue since all components have been migrated to new arch already :rocket: If you spot any issues, please provide a small repo with reproduction included.


CheckWitness problem with invokeScript

Hey,
All invocations that require check witness fail if used with invokeScript looking at neo-python it adds a signed transaction which CheckWitness can use.
If I add signed transaction it successfully invokes the contract this comes at cost.
I think it would be nice if test invoke can handle check witness calls.

Hi Said,
We cannot do anything about this. You should file an issue on the C# node side instead as the JS SDK is an interaction package, we merely communicate the parameters across, we do not run our own VM internally. As such, we are unable to emulate what neo-python can do even if we want to.
To fix this, you will have to ask neo-project to change the specification of the test invoke call.

@Said007 are you sending your user's hash? Using the following formatting I got my checkwitness to pass.
const formattedAccount = u.reverseHex(
      wallet.getScriptHashFromAddress(this.account.address)
    );

@jeroenptrs Nope was sending the users address as a byteArray will try your approach of sending the users address hash

I had to add an extra step to the formatting for it to actually work. The neo-python prompt would pass it but in reality the Checkwitness wouldn't fire (thus throwing away my invocation).
I now use 1 function outside of neon-js to get my Checkwitness to actually fire and continue my application, along with the 2 previous functions:
const address = unhexlify(
  u.reverseHex(wallet.getScriptHashFromAddress(this.account.address))
);

unhexlify is a function from binascii
Post-formatting it's converted with sc.ContractParam.string before getting added to the args array. I do not use sc.ContractParam.array to cast args to a contract param.
If you need any more help, ping me a PM on discord 😄


Refactor ui

Codecov Report

Merging #89 into v0.3.0 will increase coverage by 2.37%.
The diff coverage is 50%.

@@            Coverage Diff             @@
##           v0.3.0      #89      +/-   ##
==========================================
+ Coverage   45.36%   47.74%   +2.37%
==========================================
  Files          30       31       +1
  Lines        1111     1108       -3
==========================================
+ Hits          504      529      +25
+ Misses        573      547      -26
+ Partials       34       32       -2

Impacted Files
Coverage Δ

pkg/clui/config.go
100% <ø> (ø)

pkg/clui/nop.go
0% <0%> (ø)

...kg/grapicmd/internal/module/generator/generator.go
0% <0%> (ø)
:arrow_up:

pkg/grapicmd/internal/module/generator/base.go
56.62% <100%> (ø)
:arrow_up:

pkg/grapicmd/internal/module/generator/command.go
100% <100%> (ø)
:arrow_up:

pkg/grapicmd/internal/module/generator/status.go
81.81% <100%> (ø)
:arrow_up:

pkg/grapicmd/internal/module/generator/service.go
95.55% <100%> (ø)
:arrow_up:

pkg/grapicmd/cmd/user_defined.go
85.18% <100%> (ø)
:arrow_up:

pkg/grapicmd/internal/module/generator/project.go
77.77% <100%> (ø)
:arrow_up:

pkg/clui/ui.go
100% <100%> (ø)

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f03676d...0163365. Read the comment docs.


chore(package): update mocha to version 2.4.2

http://greenkeeper.io/

Current coverage is 83.62%

Merging #15 into master will not affect coverage as of 3ef8317

@@            master     #15   diff @@
======================================
  Files            4       4
  Stmts          171     171
  Branches        41      41
  Methods          0       0
======================================
  Hit            143     143
  Partial          0       0
  Missed          28      28

Review entire Coverage Diff as of 3ef8317

Powered by Codecov. Updated on successful CI builds.


Feature Request - Jump to Top button on Website pages

When scrolling through the website sometimes I want to jump to the top of the page. I would like a button to appear after scrolling that returns my window to the top of the page when I click it.
edit relevant zulip discussion

When I've been going through the website tutorial/std lib docs most of the time I want to jump back to the top is to get the table of contents to jump to another section/function. It might be useful to make the table of contents always visible as you scroll (or tucked in a menu that is always visible as you scroll) to make this easier. This would be in addition the the top of page button, which I think is also a good idea.

Is this issue still exist??

It might be hard to place this button well when the site is in a narrower window or on mobile. I'm closing for now but we can reconsider if more people want it.


chore: list contributors in file

Related Issue or Design Document
https://github.com/ory-corp/general/issues/732
Checklist

[x] I have read the contributing guidelines
and signed the CLA.
[x] I have referenced an issue containing the design document if my change
introduces a new feature.
[x] I have read the security policy.
[x] I confirm that this pull request does not address a security
vulnerability. If this pull request addresses a security vulnerability, I
confirm that I got approval (please contact
<EMAIL_ADDRESS>from the maintainers to push
the changes.
[x] I have added tests that prove my fix is effective or that my feature
works.
[x] I have added the necessary documentation within the code base (if
appropriate).

Let's only list authors in popular open-source repos where it really adds value.


make flag debug work in manager

This PR fixed #2203
PR did:

add Flag debug in manager's context struct, making info api can get this boolean
add NFd and NGoroutines in info api
add /docker/pkg/fileutils in vendor to support NFd

After this pr, docker info shows like below when debug enabled:
root@ubuntu:/home/daocloud# docker info
Containers: 9
 Running: 0
 Paused: 0
 Stopped: 9
Images: 47
Server Version: swarm/1.2.1
Role: primary
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 1
 ubuntu: <IP_ADDRESS>:2376
  └ ID: RDEJ:ZCV6:CJH5:UCNB:P5RR:66LB:JZSO:YACX:TBL3:PYMV:2F77:KBUA
  └ Status: Healthy
  └ Containers: 9
  └ Reserved CPUs: 1 / 1
  └ Reserved Memory: 1 GiB / 2.052 GiB
  └ Labels: executiondriver=, kernelversion=3.19.0-25-generic, operatingsystem=Ubuntu 14.04.3 LTS, storagedriver=aufs
  └ Error: (none)
  └ UpdatedAt: 2016-05-03T12:44:18Z
  └ ServerVersion: 1.11.0
Plugins:
 Volume:
 Network:
Kernel Version: 3.19.0-25-generic
Operating System: linux
Architecture: amd64
CPUs: 1
Total Memory: 2.052 GiB
Name: ubuntu
Docker Root Dir:
Debug mode (client): false
Debug mode (server): true
 File Descriptors: 9
 Goroutines: 23
 System Time: 2016-05-03T20:44:22.605130289+08:00
 EventsListeners: 0
WARNING: No kernel memory limit support

Below is useful infomation added via this PR:
Debug mode (server): true
 File Descriptors: 9
 Goroutines: 23
 System Time: 2016-05-03T20:44:22.605130289+08:00
 EventsListeners: 0

Signed-off-by: Sun Hongliang<EMAIL_ADDRESS>

The command "GOOS=darwin go build" exited with 2.  failed
No compatible in darwin OS,  :(

ping @dongluochen @nishanttotla
I think docker/pkg/fileutils has already supported darwin, so I revendored those files to make this CI works. PTAL. Feel free to tell me if I could provide more details. Thanks.


Seal ArrowNativeType and OffsetSizeTrait (#1028)

Which issue does this PR close?
Closes #1028
Rationale for this change
ArrowNativeType is used to indicate "trivially safely transmutable" within the buffer implementations. If client can create their own implementations they can do potentially unwise things.
What changes are included in this PR?
Makes ArrowNativeType sealed
Are there any user-facing changes?
Technically yes, practically I can't think of any valid use-cases this will break.

Great! This could also seal the OffsetSizeTrait!

Good point, this will transitively also seal OffsetSizeTrait

This PR's description says "seal OffsetSizeTrait but I didn't see any change to that trait in

ArrowNativeType is sealed and so cannot be implemented for types outside of arrow-rs
OffsetSizeTrait can only be implemented for ArrowNativeType
Trait orphan rules prevent implementing OffsetSizeTrait for foreign types outside of arrow-rs

These three things together seal OffsetSizeTrait

ArrowNativeType is sealed and so cannot be implemented for types outside of arrow-rs
OffsetSizeTrait can only be implemented for ArrowNativeType
Trait orphan rules prevent implementing OffsetSizeTrait for foreign types outside of arrow-rs

These three things together seal OffsetSizeTrait

Sorry, I find that we should further seal the OffsetSizeTrait because Offset only support i32 and i64. I will file a followup PR.


Rerunning individual Cucumber scenario reruns entire feature file

This is not a problem with the rerun service itself, but due to an upstream issue somewhere in the main WDIO codebase:
https://github.com/webdriverio/webdriverio/issues/7735
The rerun.sh file specifies a line number like so:
rerun.sh
-----------
DISABLE_RERUN=true node_modules/.bin/wdio ./wdio-configs/local-chrome.conf.js   --spec=/Users/esaari/code/jwplayer-commercial/test/cucumber-features/basic/basic_mp4.feature:5

Due to the bug listed above, the entire feature file is being run which is causing unintended scenarios to be rerun.
Leaving this issue open for visibility in case anybody running Cucumber is experiencing the same behavior. A fix was attempted by the original issue reporter, but abandoned: https://github.com/webdriverio/webdriverio/pull/7752

Please post exact versions for 'webdriveri' dependencies.

I reproduced this issue with a basic project, here's the package.json
{
  "name": "my-new-project",
  "type": "module",
  "devDependencies": {
    "@wdio/allure-reporter": "^8.2.0",
    "@wdio/cli": "^8.2.4",
    "@wdio/cucumber-framework": "^8.2.3",
    "@wdio/junit-reporter": "^8.2.0",
    "@wdio/local-runner": "^8.2.4",
    "@wdio/spec-reporter": "^8.1.2",
    "chromedriver": "^109.0.0",
    "ts-node": "^10.9.1",
    "typescript": "^4.9.4",
    "wdio-chromedriver-service": "^8.0.1"
  },
  "scripts": {
    "wdio": "wdio run ./wdio.conf.ts"
  }
}

here's the command I use
npm run wdio -- --spec="/Users/carc/projects/wdio-nextgen/features/login.feature:9"

and this is the repo for that project
https://github.com/cesar-rivera/wdio-basicproject

@esaari @christian-bromann @cesar-rivera Found this issue occurs as of 7.20.6 last working version is 7.20.3, I am looking into what changed.

Hi just wondering if there's been any update to this issue or other fixes that may address it at all?
We recently started trying to upgrade to v8 of wdio packages, from v7 and have found this issue ^ with executing cucumber scenario via line number.

Fixed in https://github.com/webdriverio/webdriverio/pull/10772


LoggableTrait assumes numeric id

The fuctions in the LoggableTrait assumes numeric id's, but with uuid's and uniqid() it is not certain.

Resolved in https://github.com/KnpLabs/DoctrineBehaviors/pull/677

Resolved in https://github.com/KnpLabs/DoctrineBehaviors/pull/677


Avoid overlap with repokid-extras

Netflix open-sourced some of their internal work that shares strong similarities with CloudTracker:
https://github.com/Netflix-Skunkworks/repokid-extras
Keeping this here as a note so that project can be reviewed to avoid duplicate efforts.

Closing this as that project is not maintained (had only 3 commits, in a week in April 2018).


Remove non-constant time AES code and speed-up of legacy APIs

This PR contains some improvements from my openssl-111-features branch #11900
mostly remove almost all non-constant time AES code and significantly improve
the performance of the AES legacy APIs.
It consists of the following

Remove x86/x86_64 BSAES and AES_ASM support (which is non-constant time)
Add vpaes_ctr32_encrypt_blocks for x86_64 (author: @davidben)
Integrate vpaes_ctr32_encrypt_blocks into crypto
Enable legacy AES API to use AESNI or VPAES if available
Enable AES_cbc_encrypt to use AESNI or VPAES if available
Fix AES hardware acceleration for linux-x86 targets

Note: the last one does likely also affect win32 builds.
Previously constant time C code for AES was only used with no-asm builds,
but with this PR the AES assembler code will fall back to the constant time
C code when the CPU does not support AESNI or VPAES.

OTC: this is not for 3.0, we should get a performance comparison before accepting.


"export 'effect' was not found in 'deepsignal'

Hi,
I am new to deepsignals.
I face this issue when using effect as the example in Readme.md:

effect(() => {
console.log(state.counter);
});

Could you pls help to let me know where effect should i import from? Do i need to use effect from Signals along with Deepsignals?

Yes, if you need effect, you need to import it from @preact/signals 🙂
import { effect } from "@preact/signals";


[aad] Parse spring roles from azure ad roles

Currently, the spring library parses user groups out of the graph api and populates the spring roles. The expected behaviour is to parse jwt app roles and use that in the PreAuthorize filter.

This is a hack but seems to work for me  @PreAuthorize("principal.claims['roles'][0] == 'MyApplicationRole'")


[SP-1776] Regression of PDI-13633: Database Connections with JNDI mix up...

... JNDI name with Connection name (5.3 Suite)
@mattyb149, @brosander, review it please

Needs unit and/or integration tests

@ivanpogodin if we're going to make DatabaseUtil a singleton, we should make the constructor private to prevent accidentally having orphan instances used elsewhere in the codebase

@mattyb149, @brosander, review it please

@ivanpogodin where is connectUsingNamedDataSource called? do we still need this method? are there expected impacts from removing it?

I agree, we can remove private connectUsingNamedDataSource.
I kept it because it's the only point where rg.pentaho.di.core.database.DataSourceProviderFactory used.
As I see, removing connectUsingNamedDataSource makes [platform]PlatformKettleDataSourceProvider unused. As well as [kettle]DataSourceProviderFactory. And probably some other classes.
I haven't decided to remove them. The impact is too risky. This is a larger task - to remove classes or change datasource lookup algorithm.

see also fix https://github.com/pentaho/pentaho-kettle/pull/1264


In kubernetes when killing pod with goffys container, pod stuck in terminating. exit code 137.

We have an issue with pods running goofys as a sidecar in kubernetes. When we delete the pod, it takes time or stuck shutting down the goofys container. This is due to goofys connection to s3 endpoint not being terminated properly.
We weren't able to reproduce it in hyperscalers (aws,gcp,etc) .
The normal behavior of goofys when sent a sigkill , is to exit with exit code 137.
We're trying to figure out if there's a way to avoid having this exit code in goofys so that our pods will not be stuck on terminating and be deleted properly.
goofys version 0.23.1
running command: goofys -f --dir-mode 0777 --file-mode 0777 -o allow_other --debug_s3 --endpoint https://xxx.xxx.xxx bucket_name /data
The pod is running as privileged in kubernetes.
We can rule out OOM issues as this was checked. It also happens with empty buckets and with new pods that were just spun up.
Any assistance will be appreciated.

+1, I cant figure it out yet. Are you also running in the Alpine-based container @dpointk?

a process cannot handle SIGKILL, so if k8s is sending goofys SIGKILL there's nothing it can do


Fix package json

:tada: This PR is included in version 9.7.11 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Fix crash with auto qualification consequence

fixes a comparison between naive and aware datetime object

coverage: 89.369% (+0.02%) from 89.35% when pulling 275871f688a2b59a580234c6160c851e062c92a3 on fix-unaware-datetime-with-autoqualification into 829b669f2f785c06375c108dd0a100b2a6f842d5 on main.


Add an example using custom distance for nearest search

explore defining struct MyPoint { float x, y; };

The whole point is for a user to not define their own type. The only thing that's necessary is to have a separate type for disambiguation during the dispatch.

@dalg24 Do you have any blocking objections to merging the example in the current form?

I chose distances so that a user can see that the computed distances are indeed the custom ones. Computing just the found indices seems insufficient to me, as it still does not confirm that we can get the correct distance in the callback.

@dalg24 If you find complexity unnecessary, please provide a reply to this comment.


Documentation: Iteration example is incorrect (template <-> view)

Describe the bug
In the documentation under the section Iteration the property view is used in the example, which is however still named template in KeyedProps and IndexedProps.
Maybe we should rename the property for consistency.

yeah that's a typo in the docs. If you could send a PR, that would be great. Also just change the next docs, not the 0.7 docs because that would be updated by next release.


How to pre-configure hub for the users of our GitHub OAuth application?

Hello!
We run janitor.technology, a developer cloud service that makes it easy to contribute to well-known open source projects like Firefox, Servo, Chromium, Thunderbird, Discourse, etc. It works by letting you quickly spawn Docker containers that are fully pre-configured for developing a given software project, and since most of the projects we support are hosted on GitHub, all Janitor containers come with hub conveniently pre-installed.
Today, when a Janitor user want to submit a pull request, we tell them to run hub fork in their container, which signs them in to GitHub and generates a Personal Access Token, and then they need to manually create a ~/.netrc configuration file (by copy/pasting their PAT from ~/.config/hub, as explained in this gist).
However, this is very tedious to do manually for every new container you create, so we would like to fully automate developer authentication by pre-configuring hub for each user in each of their containers.
We already have Janitor registered as a GitHub OAuth application (i.e. users can enable Janitor's "GitHub integration" by "Signing in with GitHub", which generates a GitHub OAuth token for that user, allowing Janitor to use the GitHub API on behalf of the user), and we're hoping to use that API access to generate Personal Access Tokens for hub.

Can we use the GitHub API to generate Personal Access Tokens for our users?
If yes, which scopes and API handlers do we need to use? If no, how else can we generate PATs for our users?
Can we limit the scope of the generated Personal Access Tokens? (Ideally, for security reasons in shared infrastructure, we'd love PATs to only be able to open pull requests, but not merge arbitrary code).
Can we limit the lifetime or expire Personal Access Tokens? (We'd love to periodically rotate our user's PATs, so that if our infra is compromised and tokens are stolen, they're not harmful for very long).

Thanks for an amazing CLI tool! I look forward to making hub ultra-convenient for our users.

It works by letting you quickly spawn Docker containers that are fully pre-configured for developing a given software project

That sounds really cool :)

We already have Janitor registered as a GitHub OAuth application (i.e. users can enable Janitor's "GitHub integration" by "Signing in with GitHub", which generates a GitHub OAuth token for that user

It sounds like you already have a token for each user that, theoretically, can be used to create pull requests on behalf of that user, at least until they revoke Janitor from their approved apps list.
The scopes for the token need to be repo. There's currently no way to create an API token that can open pull requests but not merge them.
The only thing that would be required to configure hub at this point would be to somehow set the GITHUB_TOKEN environment variable within the container. This could be done in a global or a per-user bashrc (given that bash is the default login shell for the container). If GITHUB_TOKEN exists, it will be used for all API requests coming from hub without the need for the user to supply their username/password, even if ~/.config/hub doesn't exist within the container at all.
You could put the same token inside the user's ~/.netrc file as well.
However, be careful and transparent with your users. People typically like to share containers. If someone isn't aware that their GitHub API token is embeded within a container and they pass on that container to others, they've just compromised their account.

Read here for a background on an incident that happened when GitHub tokens leaked from Travis CI. You definitely don't want to find yourself mitigating this kind of incident.
For more information about programatically creating PATs using the GitHub API, see https://developer.github.com/v3/oauth_authorizations/
Does this answer all of your questions?

That sounds really cool :)

Thanks! 😄

It sounds like you already have a token for each user that, theoretically, can be used to create pull requests on behalf of that user, at least until they revoke Janitor from their approved apps list. I don't think there's a reason to create a separate Personal Access Token (PAT is just a fancy name for OAuth token).

This was just to separate concerns, e.g. one token that Janitor uses to authenticate and learn more about its users (e.g. real name, emails, public SSH keys) and another token that is responsible for sending Pull Requests from containers (this token should be rotated more often). But maybe it doesn't make sense, and we should just include the user's single Janitor OAuth token in all their configuration files and containers?

The scopes for the token need to be repo. There's currently no way to create an API token that can open pull requests but not merge them.

Thanks! The fact that we can't prevent arbitrary merges is unfortunate, but I guess more and more projects have a protected master branch (with no direct pushes, and only a few people are allowed to merge PRs, but first they need to pass tests / linting / code review) so hopefully stolen repo tokens won't actually be able to cause direct damage to important repositories.

The only thing that would be required to configure hub at this point would be to somehow set the GITHUB_TOKEN environment variable within the container. This could be done in a global or a per-user bashrc (given that bash is the default login shell for the container). If GITHUB_TOKEN exists, it will be used for all API requests coming from hub without the need for the user to supply their username/password, even if ~/.config/hub doesn't exist within the container at all.
You could put the same token inside the user's ~/.netrc file as well.

Cool, thanks! We do have automated container configurations (i.e. ~/.bashrc and any other user-specific file), so it will be easy to template a token into ~/.config/hub or set the GITHUB_TOKEN variable in ~/.bashrc (although we don't template that file yet), and ~/.netrc too.

However, be careful and transparent with your users. People typically like to share containers. If someone isn't aware that their GitHub API token is embeded within a container and they pass on that container to others, they've just compromised their account.

Great point! We do want to be careful and transparent with our users about the fact that their containers include sensitive tokens. Container sharing isn't possible on Janitor today (we auto-configure and host developer containers ourselves, and authenticate users to only grant access to container owners), but we'd like to implement some kind of port-specific sharing feature (see https://github.com/JanitorTechnology/janitor/issues/182) so there we should be very clear about what risks sharing a container implies.

For more information about programatically creating PATs using the GitHub API, see https://developer.github.com/v3/oauth_authorizations/

So if I want to generate a PAT for a user, I just POST to https://api.github.com/authorizations/ with my client_id, client_secret and scope=repo, authenticating via Basic Authentication with <username>:<janitor_token>, and get back a repo-scoped PAT with its authorization ID?
I suppose that for this to work, the original janitor_token also needs the repo scope? Any other scope needed, e.g. authorizations?
To revoke the generated PAT, I just DELETE https://api.github.com/authorizations/:id and that's it?

Thanks again for your very helpful replies! 👍

This was just to separate concerns, e.g. one token that Janitor uses to authenticate and learn more about its users (e.g. real name, emails, public SSH keys) and another token that is responsible for sending Pull Requests from containers (this token should be rotated more often).

It's better that you include the user's original Janitor token because that token will then be tied to Janitor's OAuth application. When a user is done using Janitor, they can simply revoke access to Janitor in their approved applications list on GitHub.
Also, you can't programatically create OAuth tokens for your users because the OAuth Authorizations API only supports Basic Authentication with real username/password/2FA token.
Again, be very careful when bundling users' OAuth tokens in the containers! A much better idea would be to generate some kind of Janitor token and bundle that instead. Then, scripts to open pull request could use the Janitor token to hit a Janitor server, which would then pull the user's OAuth token on file and use that to communicate with GitHub API.
Proxying all GitHub API operations through your servers, instead of bundling a tool like hub to do pull requests, makes it possible to exercise tighter security protocols and rotate tokens as often as you'd like.

It's better that you include the user's original Janitor token because that token will then be tied to Janitor's OAuth application. When a user is done using Janitor, they can simply revoke access to Janitor in their approved applications list on GitHub.

Thanks a lot for your advice, this does seem like the sanest approach here.
I also see the following advantages to Janitor OAuth app tokens over hub's default Personal Access Tokens:

hub can be pre-configured automatically, with no extra set-up step required before developers can use it in their containers
If there is ever a major security incident, a Janitor OAuth app admin can revoke all tokens in one click
Janitor OAuth tokens can be restricted to the public_repo scope, instead of having hub's PAT default repo scope

Proxying all GitHub API operations through your servers, instead of bundling a tool like hub to do pull requests, makes it possible to exercise tighter security protocols and rotate tokens as often as you'd like.

Thanks, this is an interesting idea to improve security even more, but I'm afraid it isn't practical for our system as we'd need to basically re-implement hub in containers and proxy all pull-request creation calls within our service (and maintain that forever in order to follow any hub or GitHub API changes).
I'm hoping that a combination of clear UI (e.g. "Warning: These tokens are dangerous, you can revoke them at any time here: https://github.com/settings/applications"), limited scope (just public_repo instead of repo and other powerful scopes) and protected master branches on most important repositories will lead to a design that's both convenient and safe enough.


Mac OSX index download significantly slower than Linux

I have TravisCI run tests for my Rust project on both and Ubuntu image and OSX.  When the process gets to getting the cartes.io index it's instantaneous on the Ubuntu image but for Mac OSX it sits about 3 to 5 minutes before downloading its crates.  The downloads are fast once they start so it's not a connection issue.
You can look at my project for the difference in time stamps: https://travis-ci.org/danielpclark/faster_path/builds
The point at which it stalls (only on Mac) is:
Updating registry `https://github.com/rust-lang/crates.io-index`

And the behavior is consistent.

I know that Travis has been having a difficult time recently with OSX, and historically we've seen that macs there are just slower (and we can't really explain why). Is this something you can reproduce locally? If not I'm tempted to chalk it up to "cloud infrastructure is weird sometimes"

I only have a PowerPC on the Mac end of things.  I'm not sure Rust supports that yet and I don't think that data sample would be fair in comparison to the Intel Mac OSX builds.

Ok, in that case I haven't experience this locally on my own Mac and otherwise I think this is a Travis issue, so closing.


chore(deps): upgrade dev dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-dev-dependencies-k8s-30-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 4715

Questions ?
Please refer to the Backport tool documentation


Apache httpd v2.4.12 not available from intended location

Currently apache.sh attempts to perform:
wget http://www.us.apache.org/dist//httpd/httpd-2.4.12.tar.gz

This fails since 2.4.12 is not available at that location.

As an emergency fix, bumping httpd version from 2.4.12 to 2.4.16. Ref #92.

While this is fixed from #92, it will just happen again when a new version comes out. See #94 for future plans.


Add support for 8-bit color format in ui.Image

It is recommended to add support for 8-bit color format in ui.Image, such as OPAQUE/APHLA8/GRAY8 anyway.
It provides the ability to input any data into the shader's sampler without memory waste.
for example:
Input Y/U/V components to the shader respectively, then you can play YUV stream on pure Flutter without use of any platform external texture.
pseudo code:
Future<ui.Image> decodeFrom8BitFormat(Uint8List bytes, int width, int height) {
  final completer = Completer<ui.Image>();
  decodeImageFromPixels(
    bytes.buffer.asUint8List(),
    width,
    height,
    PixelFormat.alpha8,  // 8bit pixel format
    completer.complete,
  );
  return completer.future;
}

Future<Shader> yuvShader() async {
  const width = 960;
  const height = 540;
  final yBytes = Uint8List(width * height);
  final uBytes = Uint8List(width * height ~/ 4);
  final vBytes = Uint8List(width * height ~/ 4);
  final yTextureImage = await decodeFrom8BitFormat(yBytes, width, height);
  final uTextureImage = await decodeFrom8BitFormat(uBytes, width ~/ 2, height ~/ 2);
  final vTextureImage = await decodeFrom8BitFormat(vBytes, width ~/ 2, height ~/ 2);

  const _spirv = ''; // base64 encoded compile from glsl
  final program = await FragmentProgram.compile(spirv: Uint8List.fromList(base64Decode(_spirv)).buffer);
  return program.shader(
    floatUniforms: Float32List.fromList([]),
    samplerUniforms: [
      ImageShader(
        yTextureImage,
        TileMode.clamp,
        TileMode.clamp,
        UmbraShader.identity,
      ),
      ImageShader(
        uTextureImage,
        TileMode.clamp,
        TileMode.clamp,
        UmbraShader.identity,
      ),
      ImageShader(
        vTextureImage,
        TileMode.clamp,
        TileMode.clamp,
        UmbraShader.identity,
      ),
    ],
  );
}

In my case, to render YUV420P images on pure Flutter, I must convert YUV data to fake RGBA data which intents to sink data to ImageShader.
So I converted YYYYUV to RGBA like, such as vec4(0, 0, 0, Y0),vec4(0, 0, 0, Y1),vec4(0, 0, 0, Y2),vec4(0, 0, 0, Y3),vec4(0, 0, 0, U0),vec4(0, 0, 0, V0). It's a much(3X) waste of memory. Follow is my code:
final yuvData = await rootBundle.load('assets/dash.yuv');
const yLen = 1920 * 1080;
const uvLen = 960 * 540;
final yData = Uint8List.view(yuvData.buffer, 0, yLen);
final uData = Uint8List.view(yuvData.buffer, yLen, uvLen);
final vData = Uint8List.view(yuvData.buffer, yLen + uvLen, uvLen);
final yRgbaData = Uint32List.fromList(yData);
final uRgbaData = Uint32List.fromList(uData);
final vRgbaData = Uint32List.fromList(vData);
yImage = await decodeRawRgba(yRgbaData.buffer.asUint8List(), 1920, 1080);
uImage = await decodeRawRgba(uRgbaData.buffer.asUint8List(), 960, 540);
vImage = await decodeRawRgba(vRgbaData.buffer.asUint8List(), 960, 540);

final result:

The source code is here: flutter_yuv_render_sample

This sounds like an issue that should be filed in the/a Flutter repository, maybe https://github.com/flutter/flutter.

I raised this question under flutter issues, but it is considered to be related to the dart:ui library.

The dart:ui library is a Flutter library, it's only provided by the Flutter SDK, and does not exist in the plain Dart SDK.

Back on track :)


Use of LocaleContextHolder vs. request.getLocale()

Hi,
In ErrorMessageRestExceptionHandler#resolveMessage, the active locale is determined using HttpServletRequest#getLocale. In Spring land, I believe LocaleContextHolder is the canonical way of retrieving the locale in the current context.

Hi, you're right, thanks for the suggestion! I'll fix it tomorrow.


don't change btax_betr_entity_Switch from default

Remove the "fixup" that forced btax_betr_entity_Switch to True.

cc @jdebacker @MattHJensen if this looks good, I will merge and deploy for testing.

@TJ, LGTM


Created endpoint to return a list of failed pickups

Closes #2471

:tada: This PR is included in version 5.6.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Deployment failed with Node 16 LTS

Please provide us with the following information:

This issue is for a: (mark with an x)
- [x] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Deploy the app using App service extension and select runtime as Node 16 LTS

Any log messages given by the failure

6:48:01 PM appservice41: Using Node version:
6:48:01 PM appservice41: v16.14.2
6:48:01 PM appservice41: Using Npm version:
6:48:01 PM appservice41: 8.5.0
6:48:01 PM appservice41: Running 'npm install --unsafe-perm'...
6:48:02 PM appservice41: npm ERR! Tracker "idealTree" already exists
6:48:02 PM appservice41: npm ERR!

Linux

Versions

Node 16

Mention any other details that might be useful
Deployment works with Node 14. So, need clarity why is Node 16 not supported.

Thanks! We'll be in touch soon.

Hello @aarthiem
We are looking into it and will get back to you soon.

Hi @aarthiem
We were able to deploy the app successfully to Azure App Service using the Deploy to Azure button  and following the steps.

I can't comment on why it's failing when you are trying to use the vscode plugin.
Maybe try creating a video showing what you are doing step by step so that we have more clarity on what could be wrong?

Under the hood. Azure App Service instances seem to use docker images. With Node 15+ It seems there is an issue with where commands are invoked in docker containers.  We are going to just suggest at this time customers use Node 14 in their dev environment and when they are deploying to Azure at this time.
https://stackoverflow.com/questions/57534295/npm-err-tracker-idealtree-already-exists-while-creating-the-docker-image-for

PR to update docs.
https://github.com/Azure-Samples/communication-services-web-chat-hero/pull/50
Thanks for logging this issue @aarthiem

Thanks for addressing the issue @alkwa-msft