Block import from command line uses wrong difficulty

Trinity version: 0.1.0a36 (157a9981c4f51702804e29144363fa8f9fadc6aa)
OS: linux
Python version (output of python --version): Python 3.7.9

Environment
aiohttp==3.6.0
alabaster==0.7.12
anyio==1.4.0
apipkg==1.5
appdirs==1.4.4
argcomplete==1.12.1
asks==2.4.10
asn1crypto==1.4.0
async-generator==1.10
async-lru==0.1.0
async-service==0.1.0a11
async-timeout==3.0.1
asyncio-cancel-token==0.2.0
asyncio-run-in-process==0.1.0a10
attrs==20.2.0
Babel==2.8.0
backcall==0.2.0
base58==2.0.1
bitarray==1.2.2
blake2b-py==0.1.3
bleach==3.2.1
bloom-filter==1.3
bump2version==1.0.1
bumpversion==0.6.0
cached-property==1.5.2
cachetools==3.1.1
certifi==2020.6.20
cffi==1.14.3
chardet==3.0.4
click==7.1.2
cloudpickle==1.6.0
coincurve==10.0.0
colorama==0.4.4
coverage==5.3
cryptography==3.1.1
cytoolz==0.11.0
decorator==4.4.2
distlib==0.3.1
docopt==0.6.2
docutils==0.16
entrypoints==0.3
eth-abi==2.1.1
eth-account==0.5.4
eth-bloom==1.0.3
eth-enr==0.3.0
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.3.3
eth-rlp==0.2.1
eth-tester==0.5.0b2
eth-typing==2.2.2
eth-utils==1.9.5
execnet==1.7.1
factory-boy==2.12.0
Faker==4.14.0
filelock==3.0.12
flake8==3.7.9
flake8-bugbear==19.8.0
h11==0.11.0
hexbytes==0.2.1
hypothesis==4.57.1
idna==2.10
ifaddr==0.1.7
imagesize==1.2.0
importlib-metadata==2.0.0
incremental==17.5.0
ipfshttpclient==0.6.1
ipython==7.9.0
ipython-genutils==0.2.0
jedi==0.17.2
jeepney==0.4.3
Jinja2==2.11.2
jsonschema==3.2.0
keyring==21.4.0
lahja==0.17.0
lru-dict==1.1.6
lxml==4.6.1
MarkupSafe==1.1.1
mccabe==0.6.1
more-itertools==8.5.0
multiaddr==0.0.9
multidict==4.7.6
mypy==0.782
mypy-extensions==0.4.3
netaddr==0.8.0
netdisco==2.8.2
netifaces==0.10.9
outcome==1.0.1
packaging==20.4
parsimonious==0.8.1
parso==0.7.1
pathtools==0.1.2
pexpect==4.8.0
pickleshare==0.7.5
pkginfo==<IP_ADDRESS>
pluggy==0.13.1
plyvel==1.2.0
prometheus-client==0.7.1
prompt-toolkit==2.0.10
protobuf==3.13.0
psutil==5.7.2
ptyprocess==0.6.0
py==1.9.0
py-ecc==4.1.0
py-evm==0.3.0a19
pycodestyle==2.5.0
pycparser==2.20
pycryptodome==3.9.8
pyethash==0.1.27
pyflakes==2.1.1
pyformance==0.4
Pygments==2.7.1
pyparsing==2.4.7
pyrsistent==0.17.3
pysha3==1.0.2
pytest==5.4.3
pytest-cov==2.8.1
pytest-forked==1.3.0
pytest-mock==1.12.1
pytest-randomly==3.1.0
pytest-timeout==1.4.2
pytest-watch==4.2.0
pytest-xdist==1.29.0
python-dateutil==2.8.1
python-snappy==0.5.4
pytz==2020.1
readme-renderer==27.0
requests==2.24.0
requests-toolbelt==0.9.1
rfc3986==1.4.0
rlp==2.0.0a1
rusty-rlp==0.1.15
SecretStorage==3.1.2
semantic-version==2.8.5
six==1.15.0
sniffio==1.2.0
snowballstemmer==2.0.0
sortedcontainers==2.2.2
Sphinx==1.7.9
sphinx-rtd-theme==0.5.0
sphinxcontrib-asyncio==0.2.0
sphinxcontrib-serializinghtml==1.1.4
sphinxcontrib-websupport==1.2.4
SQLAlchemy==1.3.20
sqlalchemy-stubs==0.3
termcolor==1.1.0
text-unidecode==1.3
toml==0.10.1
toolz==0.11.1
towncrier==19.2.0
tox==2.7.0
tqdm==4.50.2
traitlets==5.0.5
trie==2.0.0a4
-e git+https://github.com/ethereum/trinity@157a9981c4f51702804e29144363fa8f9fadc6aa#egg=trinity
trio==0.16.0
trio-typing==0.5.0
twine==3.2.0
typed-ast==1.4.1
typing-extensions==<IP_ADDRESS>
upnp-port-forward==0.1.1
uPnPClient==0.0.8
urllib3==1.25.10
uvloop==0.14.0
varint==1.0.2
virtualenv==20.0.35
watchdog==0.10.3
wcwidth==0.2.5
web3==5.12.2
webencodings==0.5.1
websockets==8.1
yarl==1.6.2
zeroconf==0.28.6
zipp==3.3.1

What is wrong?
I'm trying to run some tests against trinity in hive. In my test, I first try to import some blocks into the Trinity database using the following command:
trinity --log-level INFO --data-dir /dd --network-id 19763 --genesis /newconfig.json --disable-tx-pool import /chain.rlp

Trinity reports:
    INFO  2020-10-19 18:00:00,219           BlockImport  Importing 10 blocks
 WARNING  2020-10-19 18:00:00,234                 Chain  Proposed Block #1-0x5f49..f2e8 doesn't follow EVM rules, rejecting...
   ERROR  2020-10-19 18:00:00,235           BlockImport  Mismatch between block and imported block on 1 fields:
 - header.difficulty:
    (actual)  : 16
    (expected): 131072
   ERROR  2020-10-19 18:00:00,235           BlockImport  Import failed

My genesis file is:
{
  "accounts": {
    "0x71562b71999873db5b286df957af199ec94617f7": {
      "balance": "0xf4240"
    }
  },
  "genesis": {
    "author": "0x0000000000000000000000000000000000000000",
    "difficulty": "0x10",
    "extraData": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "gasLimit": "0x8000000",
    "nonce": "0xdeadbeefdeadbeef",
    "timestamp": "0x0"
  },
  "params": {
    "miningMethod": "ethash",
    "chainId": "0x19763",
    "homesteadForkBlock": "0x0",
    "EIP150ForkBlock": "0x0",
    "EIP158ForkBlock": "0x0",
    "byzantiumForkBlock": "0x0"
  },
  "version": "1"
}

The chain.rlp file I'm trying to import is this file.
This file can be imported with go-ethereum, besu and other clients using an equivalent genesis configuration, so I assume something is not working in Trinity.

Hey @fjl - The trinity genesis config is implemented so that [All numbers except for the version are written in hexadecimal.](https://github.com/ethereum/trinity/blob/27f877f3444efe5a0bb34c16db1a6e5503599797/docs/guides/creating_a_custom_testnet.rst#creating-our-own-configuration-file), which is a subtle difference from geth.
I noticed that the chainId param is set to "0x19763" and the network-id cli argument is also set to 19763.  So, I'd recommend updating the chainId param in the genesis config to the hex representation of your desired network id.
But, that won't fix the bug that's happening here. It seems as though the problem is with the difficulty param in your genesis config (which needs to be a hexadecimal representation as well). Do you intend a difficulty of 10 decimal or hex? With difficulty: "0x10" (which trinity interprets as the integer 16), the hash trinity calculates for block 0 matches the parent hash of block 1 from chain.rlp - but from the error we can see that their difficulties don't match. And with the difficulty in the genesis config to 0x20000 then the calculated hash of block 0 is no longer a match with the parent hash of block 1 from chain.rlp.

Thanks for debugging it! It looks like I can resolve this issue by regenerating the test blockchain with the correct difficulty. I'm still unsure why it worked with all other clients though.

It seems that the issue might be with trinity after all. I'm generating my test chain using go-ethereum tools. It implements the ethash difficulty algorithm correctly (I think) and applies a minimum difficulty of 131072 (see it here) after other parts of the formula. In Trinity, the difficulty formula applies the minimum as well, but has an extra term that uses the parent difficulty if it's lower than the minimum (see here).

Who is right? I'm not sure. In other clients, importing my original test chain works because they compute the difficulty of block 1 as 131072 (0x20000). I can make this specific test work with Trinity by adjusting the genesis difficulty to 0x20000 and regenerating the test chain, but don't really want to do it for all tests.

@njgheorghita the test I'd try first would be disabling the override here and seeing if the state tests still pass.  I assume that we wrote that code for a reason, and the first reason I'd look at is whether our implementation still passes tests if we remove it.

    "difficulty": "0x10",

Having a starting difficulty at below minimum difficulty leads into undefined territory, where the formula is ambiguous. The YP difficulty-formula is undefined when this occurrs. So it doesn't surprise me if trinity does something different than geth or other clients does

I can make this specific test work with Trinity by adjusting the genesis difficulty to 0x20000 and regenerating the test chain, but don't really want to do it for all tests.

IMO that's the proper way to fix it

If you don't want to mine, use the noproof-engine, which has the same difficulty-calculation as ethash, but clients are expected not to verify the PoW against it. I'm not sure if trinity supports it though?

Yes, trinity supports it. But I already started mining a new test chain, will use that one.

We have resolved this in Hive by generating a new test chain at exactly minimum difficulty.
There seems to be another issue in the blockchain import command: for the chain sync
tests, we use chainID 0x4d33. The chain in the test contains some transactions signed using the
EIP155 scheme. Trinity refuses to import these transactions, printing

  |    ERROR  2020-11-03 08:31:28,326           BlockImport  Transaction.v 39562 is not less than or equal to 28
-- | --

(full log here)
In all other clients, this test chain can be imported correctly. And Trinity has no problem syncing this
chain from another peer, it's only the CLI block import which is failing.

If you want to reproduce this yourself, find the test chain here: https://github.com/ethereum/hive/blob/master/simulators/ethereum/sync/simplechain/chain.rlp
We are using the following genesis configuration:
{
  "accounts": {
    "0x71562b71999873db5b286df957af199ec94617f7": {
      "balance": "0xffffffff"
    }
  },
  "genesis": {
    "author": "0x0000000000000000000000000000000000000000",
    "difficulty": "0x20000",
    "extraData": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "gasLimit": "0x80000000",
    "nonce": "0xdeadbeefdeadbeef",
    "timestamp": "0x0"
  },
  "params": {
    "EIP150ForkBlock": "0x0",
    "EIP158ForkBlock": "0x0",
    "byzantiumForkBlock": "0x0",
    "chainId": "0x4d33",
    "frontierForkBlock": "0x0",
    "homesteadForkBlock": "0x0",
    "miningMethod": "ethash"
  },
  "version": "1"
}

Thanks for the follow-up report, I was able to reproduce this. At first glance, it seems the problem might be how we generate the v value when applying transactions. We factor in various chain-id's as detailed in the spec. Though, I'm not very familiar with this part of the codebase, so I'll have to dig a little deeper to confirm, but maybe @cburgdorf has some thoughts?


itch: init at version v1.24.0

Add setup for itch.io desktop client. From issue #90233

Motivation for this change
Attempting to add the itch.io desktop client via the itch-setup utility. See issue #90233.
Things done
Tried to base it on other electron app expressions without success.
Currently it is failing, but I'm hoping someone knows how to fix it.

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ x ] NixOS (fails)
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

any updates on this?

This PR is superseded by https://github.com/NixOS/nixpkgs/pull/157921.


build: bump version to 0.9.0

Update package.json and CHANGELOG
because we have some new files under dist, I also add dist into this commit, if this not needed, I can remove the changes under dist.

@IgorMinar, @mhevery, please review, thanks!

We should either cut a beta first or create a PR in the cli repo that
verifies that this unreleased build will work well in a cli app. JiaLi
could you do that please? This might require differential loading in cli
(feature that we are currently working on) to be somewhat usable in CLI.
@manfredsteyer is leading that effort. Manfred could you please help JiaLi
verify that we've got all we need for differential loading before we cut
the 0.9 zone.js release? Thank you.
On Wed, Feb 27, 2019, 9:35 PM JiaLiPassion<EMAIL_ADDRESS>wrote:

Update package.json and CHANGELOG
because we have some new files under dist, I also add dist into
this commit, if this not needed, I can remove the changes under dist.

@IgorMinar https://github.com/IgorMinar, @mhevery
https://github.com/mhevery, please review, thanks!
You can view, comment on, or merge this pull request online at:
https://github.com/angular/zone.js/pull/1193
Commit Summary

build: bump version to 0.9.0

File Changes

M CHANGELOG.md
https://github.com/angular/zone.js/pull/1193/files#diff-0 (13)
M dist/jasmine-patch.js
https://github.com/angular/zone.js/pull/1193/files#diff-1 (24)
M dist/jasmine-patch.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-2 (2)
M dist/long-stack-trace-zone.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-3 (2)
M dist/mocha-patch.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-4 (2)
M dist/proxy.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-5 (2)
M dist/task-tracking.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-6 (2)
M dist/webapis-media-query.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-7 (2)
M dist/webapis-notification.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-8 (2)
M dist/webapis-rtc-peer-connection.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-9 (2)
M dist/webapis-shadydom.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-10 (2)
M dist/wtf.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-11 (2)
M dist/zone-bluebird.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-12 (2)
M dist/zone-error.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-13 (2)
A dist/zone-evergreen-testing-bundle.js
https://github.com/angular/zone.js/pull/1193/files#diff-14 (4658)
A dist/zone-evergreen.js
https://github.com/angular/zone.js/pull/1193/files#diff-15 (2961)
A dist/zone-evergreen.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-16 (1)
A dist/zone-legacy.js
https://github.com/angular/zone.js/pull/1193/files#diff-17 (1509)
A dist/zone-legacy.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-18 (1)
M dist/zone-mix.js
https://github.com/angular/zone.js/pull/1193/files#diff-19 (450)
M dist/zone-node.js
https://github.com/angular/zone.js/pull/1193/files#diff-20 (27)
A dist/zone-patch-canvas.js
https://github.com/angular/zone.js/pull/1193/files#diff-21 (31)
A dist/zone-patch-canvas.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-22 (1)
M dist/zone-patch-cordova.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-23 (2)
M dist/zone-patch-electron.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-24 (2)
A dist/zone-patch-fetch.js
https://github.com/angular/zone.js/pull/1193/files#diff-25 (116)
A dist/zone-patch-fetch.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-26 (1)
M dist/zone-patch-jsonp.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-27 (2)
M dist/zone-patch-promise-test.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-28 (2)
M dist/zone-patch-resize-observer.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-29 (2)
M dist/zone-patch-rxjs.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-30 (2)
M dist/zone-patch-socket-io.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-31 (2)
M dist/zone-patch-user-media.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-32 (2)
M dist/zone-testing-bundle.js
https://github.com/angular/zone.js/pull/1193/files#diff-33 (1120)
M dist/zone-testing-node-bundle.js
https://github.com/angular/zone.js/pull/1193/files#diff-34 (51)
M dist/zone-testing.js
https://github.com/angular/zone.js/pull/1193/files#diff-35 (24)
M dist/zone.js
https://github.com/angular/zone.js/pull/1193/files#diff-36 (1166)
M dist/zone.min.js
https://github.com/angular/zone.js/pull/1193/files#diff-37 (3)
M package.json
https://github.com/angular/zone.js/pull/1193/files#diff-38 (2)

Patch Links:

https://github.com/angular/zone.js/pull/1193.patch
https://github.com/angular/zone.js/pull/1193.diff

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/angular/zone.js/pull/1193, or mute the thread
https://github.com/notifications/unsubscribe-auth/AANM6I2HnOXBN8fkzGmGVAANOV_OedDeks5vR2qdgaJpZM4bWDBj
.

@IgorMinar, Sure, will do, thanks!


refactor: remove unnecessary argument - isRoot

Current dependencies on/for this PR:

master

PR #9

PR #10   👈

PR #8

PR #11

PR #12

This comment was auto-generated by Graphite.

Current dependencies on/for this PR:

master

PR #9

PR #10   👈

PR #8

PR #11

PR #12

This comment was auto-generated by Graphite.

⏳ This pull request is set to merge as part of a Graphite merge job
Job ID: Mudo7G0eRYbNUVVj4Ct1.
See details on graphite.dev


Add a command to enable a Travis build to sign & publish an extension

I want to publish my Elm language extension, from a Travis build.
Need a Rug to enable this please, for either a Rug language extension or a Rug function.

@russmiles worked on a manual implementation of this process today. It was hard.
For later reference, here was our process: https://github.com/jessitron/kats-parser/compare/17db7d1...11f1b84

checklist:
[ ] Copy these files from this repository:
[ ] .travis.yml
[ ] src/main/scripts/travis-build.bash
[ ] settings.xml
[ ] Add to pom.xml:
[ ] property ${maven.build.timestamp}
[ ] property <maven.build.timestamp.format>yyyyMMddHHmmss</maven.build.timestamp.format>
[ ] Enable the repository for builds on travis-ci.org

signed-release profile: +

       <id>signedRelease</id>

       <activation>

           <activeByDefault>false</activeByDefault>

       </activation>

       <build>

           <plugins>

               <plugin>

                   <groupId>org.apache.maven.plugins</groupId>

                   <artifactId>maven-gpg-plugin</artifactId>

                   <version>1.6</version>

                   <executions>

                       <execution>

                           <id>sign-artifacts</id>

                           <phase>verify</phase>

                           <goals>

                               <goal>sign</goal>

                           </goals>

                       </execution>

                   </executions>

               </plugin>

           </plugins>

       </build>

   </profile>

distributionManagement repository public-atomist-release:
<distributionManagement>
        <repository>
            <id>public-atomist-release</id>
            <name>Atomist Release</name>
            <url>https://atomist.jfrog.io/atomist/libs-release-local</url>
        </repository>
    </distributionManagement>

this is not going to be relevant in the future


puppet-module-cron info for SLES10

SLE[S|D] 10 machines need an empty /etc/cron.deny file or the cron daemon shuts down.
Fixed locally by adding this to hiera for SLE[S|D] machines, it just creates the empty file.
cron::cron_deny_users:

@kividiot Thanks for the report. This was fixed with https://github.com/Ericsson/puppet-module-cron/commit/83d92d21c729a4fbced90fb59debaa0c09e92fae and is available in master branch.
Will be released with v2.0.0 soon


fix(proxy-server): redirects with relative URLs in the Location header, fix #3668

When the proxy receives a response containing a Location header with a relative URL (e.g. Location: /foobar), it redirects to the proxy itself.
With this PR this is fixed and the relative URL is prefixed with the original URL, that we’d like to fetch.
This PR also adds some tests (finally) to the proxy server.

awesome work @hanspagel
gunna wait for 1 more review before merging


Alert-Modul Error after automatic hide.

"notificationFx.js:147 Uncaught NotFoundError: Failed to execute 'removeChild' on 'Node': The node to be removed is not a child of this node.onEndAnimationFn @ notificationFx.js:147
Could we have a "ns-custom.css"? I´m having issues showing my motioneye snapshot.

Using:
this.sendNotification("SHOW_ALERT", {title: "Motion", imageHeight: "300px", timer: "6000", imageUrl: "http://localhost:8080/modules/cam/cam.png"});

There is a css/custom.css file which might help. For for the other issues: @paviro :)

I already tried that. My css/custom.css-file is overridden by "ns-default.css".

Ah, that makes sense ... This is something I need to change. The module css files are currently loaded after the custom.css file. Could you make a separate issue for this? We can then keep this issue for the bug mentioned in the openings post.

done. New seperate issue #284

This issue should be fixed. Can you check it?

fixed with: #284;

Really? Wanted to check today, well glad it works again :D


Consider using SFINAE for g::c::i::ComparePaginationValues()

@devjgm suggested
(optional): I might have written this overload sfinae'ing on the existence of ==. Maybe like:
template <typename T>
auto ComparePaginationValues(T const& a, T const& b) -> decltype(a == b) {
return a == b;
}

Not sure which way is really "preferred", but something for you to consider. Happy with whatever you decide.

To flesh out my suggestion a little, the question is what to SFINAE on: the existence of op==, or that T is not a subclass of Message. Since this overload of ComparePaginationValues() is implemented by calling op== on the two arguments, it makes sense that we'd SFINAE on the existence of this operator as well.
The difference with the existing implementation of
template <typename T, typename std::enable_if<
                          !std::is_base_of<google::protobuf::Message, T>::value,
                          int>::type = 0>
bool ComparePaginationValues(T const& lhs, T const& rhs) {
  return lhs == rhs;
}

is that the existing implementation advertises that it works for any T that is not a subclass of google::protobuf::Message, but it actually requires that the T supports op==.
These are both part of an internal overload set, so whether we change anything or not is not a high priority, IMO. I just wanted to clarify my suggestion since it's in a longer-living issue now.

Obsolete. This code no longer exits.
Related to https://github.com/googleapis/google-cloud-cpp/pull/5545


"Vârvoru de Jos" is listed in the drop-down menu

Description:
"Vârvoru de Jos" is listed in the drop-down menu alongside its corresponding county and commune, according to the law.
Precondition:
The website is up an running.

Step 1
Type "Vârvoru de Jos" in the search bar.

Expected results
The location is listed in the drop-down menu as "Comuna Vârvoru de Jos, DOLJ".


PSR-15 support (http-interop/http-server-middleware)

Requires https://github.com/zendframework/zend-expressive-session/pull/14 to be merged and released first.

Thanks, @webimpress


add AKS container images to VHD build script

What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
If applicable:

[ ] documentation
[ ] unit tests
[ ] tested backward compatibility (ie. deploy with previous version, upgrade with this branch)

Release note:

add AKS container images to VHD build script

https://msazure.visualstudio.com/One/_build/results?buildId=18402607&_a=summary&view=logs


[release-1.3] Fix Pilot Agent to correctly generate bootstrap config for control plane auth

This is an automated cherry-pick of #16460

@istio-testing: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

integ-new-install-k8s-presubmit-tests-release-1.3
857e64594ca68cec49e6c6928beafef5df9ee908
link
/test integ-new-install-k8s-presubmit-tests-release-1.3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Problem with Binary classification

If we make the fine_grain False, it performs the binary classification. But with binary, I can see there are three labels 0, 1, 2. So is it really a binary task?

because in non-root node (which is part of the sentence), it is OK to be neutral (so does the dataset https://nlp.stanford.edu/sentiment/treebank.html)
We only filter out neutral sentence in Binary Task, but keep full sentence with 0, 2 (including neutral part of its).
So, the label 1 is necessary for the non-root node. At root node, it will ignore label 1 and consider whether 0 or 2 is more "score" to decide as sentiment of the sentence.

ok, I get it now. You can have a softmax over 3 classes in the internal nodes of the tree but at the root, rather than keeping the softmax and overlooking the neutral class, it is possible to have a sigmoid layer and BCELoss as the loss function.


Routing failing when ACC is in a subfolder

This isn't a bug on the main server because of the domain, but going to https://accounts-dev.wmflabs.org/rc will give you an error vs. https://accounts-dev.wmflabs.org/rc/ which will work

I have a strong suspicion this isn't a tool issue, more likely an issue with Apache's underlying routing.

Yeah, this is definitely not a problem with the tool itself, it's a problem with our Apache config somewhere.

OK, I know what's going on here, and it's a side-effect of several bits of awkward configuration.
accounts-dev runs as a port-based virtual host behind the Labs openstack proxy. As such, Apache thinks it's running on port 8080 (which is correct) but that's not the actual port that it's accessed on publicly.
Apache sees the request for /rc, and due to the DirectorySlash directive, it issues a redirect to /rc/, but puts the entire URL in there (as far as apache sees it) instead of making it relative.
Fixes for this include disabling DirectorySlash and acknowledging the security warning; spinning up a new instance just for the dev tools; or possibly switching to name-based vhosts (if that even works behind the labs proxy - which I think it doesn't hence why we're using port-based vhosts).
I'm actually very tempted to "won't fix" this.

if that even works behind the labs proxy - which I think it doesn't hence why we're using port-based vhosts

It should work just fine, the Host header is passed through to the backend server.

Will be fixed by enwikipedia-acc/tf@46c38c079f5c1f20d40114ac68d55408b800a359 once the move to bullseye is completed.


[Bug] BD is installed but not usable in Discord.

I dont know if this is issue from BD or from Discord app, just want to make sure. I did install the BD completely but somehow it's not in the Discord. All the functions in BD like Themes, Plug-in, Settings,.... dont appear in Discord. Maybe BD  doesnt hook my Discord yet?!

Please make sure you are using the latest installer from https://betterdiscord.app


DEPRECATION: validators is being installed using 'setup.py install'

Getting this warning when installing the package:
DEPRECATION: validators is being installed using the legacy 'setup.py install' method, because it does not have a 'pypnt is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559

ref: #267 & #268


allChunks options seems to have no effect

Ideally allChunks: true should produce a css file for each System.imported resource. But it doesn't.
In my routes, I have
        {
            name: 'A',
            path: '/a',
            getComponent: () => __CLIENT__ ?
              System.import('/a').then(p => p.default) :
              Promise.resolve(require('./a').default),
          },
          {
            name: 'B',
            path: '/b',
            getComponent: () => __CLIENT__ ?
              System.import('/b').then(p => p.default) : Promise.resolve(require('./a').default),
          },

In webpack config,
{
        test: /\.scss$/,
        exclude: [/stylesheets/],
        loader: ExtractTextPlugin.extract({
          fallbackLoader: 'style-loader',
          loader: [{
              loader: 'css-loader',
              query: {
                modules: true,
                importLoaders: 2,
                localIdentName: '[hash:base64:5]',
                sourceMap: true,
              }},
            'postcss-loader',
            {
              loader: 'sass-loader',
              query: { sourceMap: true},
            }],
        }),
      },

In plugins,
  new ExtractTextPlugin({
      filename: '[name].[contenthash:5].css',
      allChunks: true,
    }),

However, I only get a single css file, although three different JS splits are produced.
What's interesting is extract text plugin logs the following:
 Child extract-text-webpack-plugin:
        [FZ+f] ./~/css-loader/lib/css-base.js 1.51 kB {0} [built]
        [S2/2] ./~/css-loader?{"modules":true,"importLoaders":2,"context":"/Users/abc/dev/","localIdentName":"[hash:base64:5]","sourceMap":true}!./~/postcss-loader!./~/sass-loader?{"sourceMap":true}!./src/A.scss 787 bytes {0} [built]

 Child extract-text-webpack-plugin:
        [FZ+f] ./~/css-loader/lib/css-base.js 1.51 kB {0} [built]
        [S2/2] ./~/css-loader?{"modules":true,"importLoaders":2,"context":"/Users/abc/dev/","localIdentName":"[hash:base64:5]","sourceMap":true}!./~/postcss-loader!./~/sass-loader?{"sourceMap":true}!./src/B.scss 787 bytes {0} [built]

"webpack": "2.2.0-rc.7",
"extract-text-webpack-plugin": "2.0.0-beta.4",
"style-loader": "^0.13.1",
"css-loader": "^0.26.1",

allChunks: true does not produce a separate file for each async import, it just extracts the css to the parent chunk css file(might be wrong on the parent part).
By default it's false and it just doesn't extract css for async imports. That's why you need to specify the fallbackLoader, that handles the css for these async imports.

@kostasmanionis I wasn't aware of that. Is there a way to split css for async imports then? I've seen a couple of websites use it.

@pastelsky This isn't supported atm, the feature request is here #455


[WIP] armdev: fix sys_clone

partially resolves #1099
sys_clone now successfully returns, but the child process hits access flag faults and permission faults. I'm working on it.

Any progress, If we run nginx, it will clone some subprocesses, and will panic after clone.

Any progress? If we run nginx, it will clone some subprocesses, and will panic after clone.

You could apply this PR and see what happens.  The sys_clone already works and the remaining problem is  memory mapping.
Since you have a fix for mprotect, this may work differently. @CharlyYu

Also include #1118, and see if you get a PF for the MMIO page.

together with the newer changes merged into armdev, this fixes (at least partially) the sys_clone issue.
testing program:
test_fork.c
#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>

int main() {
        pid_t wpid;
        int status = 0;
        printf("testing fork; init pid = %d\n", getpid());
        pid_t p = fork();
        if (p<0){
                printf("fork failed!");
                exit(1);
        }
        int mypid = getpid();
        if (p == 0) {
                while ((wpid = wait(&status)) > 0);
                printf("[PARENT]. my pid = %d\n", mypid);
                printf("\n");
        }else {
                printf("[CHILD]. my pid = %d, childpid = %d\n", mypid, p);
                printf("\n");
                exit(0);
        }
        // should not reach.
        return 0;
}

$ docker run --rm -it --runtime=quark_d test_fork

testing fork; init pid = 1
[CHILD]. my pid = [1PARENT]. my pid = , childpid = 22

(the outputs are scrambled, this is expected because of race.)
Possible dependencies on unmerged commits:
8d00808e aarch64: do not set hwcap of x86 to auxv
c92d31ef aarch64: return cntfrq as vcpu frequency
6f25e2da aarch64: fix PageOpts definition missing MT_NORMAL

known issue:
It is not working when there are more tasks than vcpu count: scheduling on the same processor seems broken atm. e.g. when setting. In this case the child is successfully forked, however the scheduler never switches back to the parent.

testing program:
pid_t p = fork();
if (p == 0) {
while ((wpid = wait(&status)) > 0);
printf("[PARENT]. my pid = %d\n", mypid);
printf("\n");
}else {
printf("[CHILD]. my pid = %d, childpid = %d\n", mypid, p);
printf("\n");
exit(0);
}

Shouldn't this be the other way around?

p == 0 => child else parent; call wait() on parent

Seems to work good!


DET-1578: bash probe kills tests after 60

just curious, what have you tested this on?

just curious, what have you tested this on?

Mostly my mac. I did throw it on an amazon container for giggles, but i was mostly testing hanging tests, not a mix of hanging / not hanging, hence missing the code reset.

just curious, what have you tested this on?

Mostly my mac. I did throw it on an amazon container for giggles, but i was mostly testing hanging tests, not a mix of hanging / not hanging, hence missing the code reset.

👍 was more curious about platform (mac/al2/ubuntu/what-have-you)


Sonar is conflicting with auto1111 webui

My extra-networks tab recently stopped working, I tried everything to get it working again, it was all fine one day, and not the other, I tried disabling extensions, and it worked, so I started to look for which extension was causing the conflict, and it was this one, when I disable it, the extra-networks tab starts working again, when I enable it, I can't access extra-networks through the tab/button, and its the only way to easily access it on Automatic1111, so I figured it was worthy of note.

I uninstalled, reinstalled the extension, and the same problem persists.

🤔 Let me check it why...

Well, this issue was once reported here #10, but nothing seems broken on my machine... I could still open the extras network tab and add any hypernets etc...
Or, could you please check is there any console error logs in your browser (press F12 and turn to console tab) when you click the dead buttons?

No console errors, nothing happens, its like I'm not even pressing the button.

Still not know why, could not reproduce this error and have no way to debug.
Leave it dangling for the moment.

Still not know why, could not reproduce this error and have no way to debug. Leave it dangling for the moment.

webui now uses gradio 3.28.1

Could this approach still have any advantages or perhaps stuff like refiners and adefiner made this obsolete? I liked the process behind this though...

Could this approach still have any advantages or perhaps stuff like refiners and adefiner made this obsolete? I liked the process behind this though...

since SDXL and more samplers came out, sonar (the momentum mechanism) is not that notable now...

as for a1111's updates, maybe it's not a problem now


TableView Resize Does not Recompute Width

Provide a general summary of the issue here
There is an unwanted behavior where the TableView does not resize with the browser. The problem is limited when the browser reduces in width. When the browser increases in width the TableView is responsive and grows.
Looking at the generated HTML, it looks like widths are dynamically applied to some DOM. However, it does not happen when reducing the browser.
Here is a full CodeSandbox
But in a nutshell, the issue appears when you have flex display:
<Provider theme={defaultTheme}>

Menu Side

Content Top

            Content Bottom
            <TableView overflowMode="wrap" selectionMode="none">
              <TableHeader>
                <Column>Date</Column>
                <Column>Name</Column>
                <Column>Region</Column>
                <Column align="end">Status</Column>
              </TableHeader>
              <TableBody>
                <Row>
                  <Cell>todo</Cell>
                  <Cell>todo</Cell>
                  <Cell>todo</Cell>
                  <Cell>todo</Cell>
                </Row>
                <Row>
                  <Cell>todo</Cell>
                  <Cell>todo</Cell>
                  <Cell>todo</Cell>
                  <Cell>todo</Cell>
                </Row>
                <Row>
                  <Cell>todo</Cell>
                  <Cell>todo</Cell>
                  <Cell>todo</Cell>
                  <Cell>todo</Cell>
                </Row>
                <Row>
                  <Cell>todo</Cell>
                  <Cell>todo</Cell>
                  <Cell>todo</Cell>
                  <Cell>todo</Cell>
                </Row>
              </TableBody>
            </TableView>

    </Provider>

With the css:
.App {
  background-color: aqua;
  display: flex;
  height: 100vh;
  width: 100vw;
}

body {
  padding: 0;
  margin: 0;
}
.side {
  background-color: blue;
  width: 200px;
  flex: 0 0 200px;
}

.content {
  flex: 1;
  display: flex;
  flex-direction: column;
}

.content-top {
}

.content-bottom {
  flex: 1 0 auto;
}

You can fix the issue by adding to the content overflow: hidden;
Issue:
Video
🤔 Expected Behavior?
I expect the TableView to be responsive inside a div with flex without needing to apply any custom css to fix the responsiveness when reducing the width of the browser.
😯 Current Behavior
It does not reduce the size of the grid when the browser is reduced in width.
💁 Possible Solution
Currently, adding to the parent container overflow: hidden; fix the issue.
🔦 Context
No response
🖥 Steps to Reproduce
Here is a CodeSandbox
Version
3.33.1
What browsers are you seeing the problem on?
Chrome
If other, please specify.
No response
What operating system are you using?
MacOS
🧢 Your Company/Team
Adobe Express
🕷 Tracking Issue
No response

This actually has to do with how flex works, you just need to change content's CSS to have
min-width: 0;

This will allow it to shrink and then the table will resize to fit. I'll close this as it's working when the CSS is changed.
The internet can explain why flex works this way better than I can. https://stackoverflow.com/questions/36247140/why-dont-flex-items-shrink-past-content-size


Add nuspec file

Currently on PusherClient nuget site we can see that Microsoft is author or that package.
This can be fixed by adding .nuspec file, just like Newtonsoft JSON package (https://github.com/JamesNK/Newtonsoft.Json/blob/master/Build/Newtonsoft.Json.nuspec)
This would allow to add project url, license url and other information to nuget package page.

Hi,
We do have one in the root of the solution :)
Thanks!
John


Assign Item Ids automatically

We need a method in some common interface(which needs to be created) to assign ids.
Ids for flowers, jewelry, and bouquets must be differentiable somehow (for example flower 0 - 100, bouquet 100 - 200, jewelry 200 - 300)
This is just an example in reality we would need more ids for flowers than for bouquets cuz bouquets consist of flowers.
U can add some constants in the options class for max num of ids for each type.
Also, u would need to keep track of the last id for each type
Ids should be reusable if u reached max, start from zero.
Constructors should be changed:

Ids in parameter lists should be deleted.
Call the method to assign id

Ps. Bouquets do not have ids in our implementation but I think we might need them for orders in the future. This is debatable. But we definitely need to assign ids for all items.

Done caption!


Bug: Floating link editor doesn't show up for AutoLinkNode

https://user-images.githubusercontent.com/70709113/227732021-9372e8c6-9ab8-4084-88bb-9ffcdb33b592.mp4
Lexical version: 0.9.1

Just checked the behaviour on the playground and saw that this issue is still there. @zurfyx, I would like to work on this issue if it's not already being fixed by someone.


Support URLs

Please add URLs to the item get and update methods.
We are a power user of the Connect SDK and would need this capability to look at moving across to this new SDK.
Moving away from hosting own infra to run Connect has huge appeal.

Hey @jessewindebank1809.
Thanks for sharing your use-case for the 1Password SDKs!
We'll look into adding support for URLs in future versions of the SDKs

Closing this ticket since support for URL fields was added since v0.1.0-beta.12.

@jessewindebank1809 quick note that unfortunately URL type fields are separate from the built-in website field on Login items. The built-in website field on Login items is not yet supported. Would you need support for that too? If so, could you file another issue for that?


Connect Vote to Agenda

Adding some visuals to User views
Adding Agenda reference to Vote
Adding :selected to VotePost

Just some small comments

@navro Fixed said comments 😄

@navro Fixed said comments :smile:

Merge! :smile:


Issue with SQL table name generation

I'm new to Entity Framework. I'm seeing what I believe to be a bug when it generates the table name. I have inherited a codebase that is using EF Designer for its models (EDMX). I have an Entity Type named "DbGame", which has an Entity Set Name of "DbGames". The Model Viewer in Visual Studio shows that the table for this entity should be called "DbGame":

However, the generated table name seems to be "DbGames":

The MSL file for the entity looks like this. Again suggesting the table should be called "DbGame", not "DbGames":

The end result is that at runtime, EntityFramework tries to look for a table named "DbGame", while it has tried to generate "DbGame".
Steps to reproduce
Unfortunately, I cannot share any source code other than the above screenshots. However, I will happily try troubleshooting steps.
Further technical details
EF version: 6.2.0
Database Provider: EntityFramework.SqlServer
Operating system: Windows 10
IDE: 15.6.6

The setting you are looking for is
entity framework plural table names - Stack ...
https://stackoverflow.com › questions › e...
Sent from my Samsung Galaxy smartphone.
-------- Original message --------From: "Jasper St. Pierre"<EMAIL_ADDRESS>Date: 2018/04/27  06:39  (GMT+02:00) To: aspnet/EntityFramework6<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: [aspnet/EntityFramework6] Issue with SQL table name generation (#528)
I'm new to Entity Framework. I'm seeing what I believe to be a bug when it generates the table name. I have inherited a codebase that is using EF Designer for its models (EDMX). I have an Entity Type named "DbGame", which has an Entity Set Name of "DbGames". The Model Viewer in Visual Studio shows that the table for this entity should be called "DbGame":
However, the generated table name seems to be "DbGames":
The MSL file for the entity looks like this. Again suggesting the table should be called "DbGame", not "DbGames":
The end result is that at runtime, EntityFramework tries to look for a table named "DbGame", while it has tried to generate "DbGame".
Steps to reproduce
Unfortunately, I cannot share any source code other than the above screenshots. However, I will happily try troubleshooting steps.
Further technical details
EF version: 6.2.0
Database Provider: EntityFramework.SqlServer
Operating system: Windows 10
IDE: 15.6.6
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/aspnet/EntityFramework6","title":"aspnet/EntityFramework6","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/aspnet/EntityFramework6"}},"updates":{"snippets":[{"icon":"DESCRIPTION","message":"Issue with SQL table name generation (#528)"}],"action":{"name":"View Issue","url":"https://github.com/aspnet/EntityFramework6/issues/528"}}}

EF Team Triage: This issue is lacking enough information for us to be able to effectively triage it. In particular, it is missing the following information requested in the new issue template. Can you please provide this information?

Steps to reproduce

Ideally include a complete code listing that we can run to reproduce the issue.
Alternatively, you can provide a project/solution that we can run.
BTW we're not just doing this to be mean :smile:... we get a lot traffic on this project and it takes time to attempt to reproduce an issue based on fragments of information. In addition, our attempt is often unsuccessful as the exact conditions required to hit the issue are often not explicitly included in the code provided. To ensure we maximize the time we have to work on fixing bugs, implementing new features, etc. we ask that folks give us a self-contained way to reproduce an issue.
For a guide on submitting good bug reports, read Painless Bug Tracking.
BTW this is a canned response and may have info or details that do not directly apply to this particular issue. While we'd like to spend the time to uniquely address every incoming issue, we get a lot traffic on the EF projects and that is not practical. To ensure we maximize the time we have to work on fixing bugs, implementing new features, etc. we use canned responses for common triage decisions.

EF Team Triage: Closing this issue as the requested additional details have not been provided and we have been unable to reproduce it.
BTW this is a canned response and may have info or details that do not directly apply to this particular issue. While we'd like to spend the time to uniquely address every incoming issue, we get a lot traffic on the EF projects and that is not practical. To ensure we maximize the time we have to work on fixing bugs, implementing new features, etc. we use canned responses for common triage decisions.


TypeError: init() got an unexpected keyword argument 'load_in_8bit'

Getting below issue when running with load_in_8bit:
TypeError: __init__() got an unexpected keyword argument 'load_in_8bit'
My Env details:
Running the model in Amazon SageMaker Notebook Instances.
CUDA Version: 11.7
Python version: 3.9
bitsandbytes-cuda117

Let me know if any other details are required.

Please show the line of code where you set this

@srowen I've set it in training.trainer.py
def load_model(
    pretrained_model_name_or_path: str = DEFAULT_INPUT_MODEL, *, gradient_checkpointing: bool = False
) -> AutoModelForCausalLM:
    logger.info(f"Loading model for {pretrained_model_name_or_path}")
    model = AutoModelForCausalLM.from_pretrained(
        pretrained_model_name_or_path, trust_remote_code=True, use_cache=False if gradient_checkpointing else True,
        **load_in_8bit=True, device_map='auto'**
    )
    return model

I'm using it as I want to train the model with limited GPU memory of 16 GB.
Any suggestion, how can I train the model EleutherAI/pythia-2.8b on 16 GB memory GPU?
FYI, I've already tried reducing batch size = 3. But that's giving OOM error.

Reduce batch size to 1. You may have longer inputs

Try param offload - this really isn't specific to this model but to your data and deepspeed settings

With below settings inside "zero_optimization" section in ds_z3_bf16_config.json?
"offload_optimizer": {
  "device": "cpu",
  "pin_memory": true
},

and NOT using (commenting out) # load_in_8bit=True, device_map='auto'
with above setup, getting new issue now:
2023-05-16 19:27:44 INFO [torch.distributed.distributed_c10d] Added key: store_based_barrier_key:2 to store for rank: 0
2023-05-16 19:27:44 INFO [torch.distributed.distributed_c10d] Rank 0: Completed store-based barrier for key:store_based_barrier_key:2 with 1 nodes.
 [WARNING]  cpu_adam cuda is missing or is incompatible with installed torch, only cpu ops can be compiled!
Using /home/users/abc/.cache/torch_extensions/py38_cu118 as PyTorch extensions root...
Creating extension directory /home/users/abc/.cache/torch_extensions/py38_cu118/cpu_adam...
Emitting ninja build file /home/users/abc/.cache/torch_extensions/py38_cu118/cpu_adam/build.ninja...
Building extension module cpu_adam...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/2] c++ -MMD -MF cpu_adam.o.d -DTORCH_EXTENSION_NAME=cpu_adam -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/ops/csrc/includes -isystem /home/users/abc/venv/lib/python3.8/site-packages/torch/include -isystem /home/users/abc/venv/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/users/abc/venv/lib/python3.8/site-packages/torch/include/TH -isystem /home/users/abc/venv/lib/python3.8/site-packages/torch/include/THC -isystem /usr/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -march=native -fopenmp -D__AVX256__ -D__DISABLE_CUDA__ -c /home/users/abc/venv/lib/python3.8/site-packages/deepspeed/ops/csrc/adam/cpu_adam.cpp -o cpu_adam.o
[2/2] c++ cpu_adam.o -shared -fopenmp -L/home/users/abc/venv/lib/python3.8/site-packages/torch/lib -lc10 -ltorch_cpu -ltorch -ltorch_python -o cpu_adam.so
Loading extension module cpu_adam...
Time to load cpu_adam op: 75.48912215232849 seconds
Using /home/users/abc/.cache/torch_extensions/py38_cu118 as PyTorch extensions root...
Creating extension directory /home/users/abc/.cache/torch_extensions/py38_cu118/utils...
Emitting ninja build file /home/users/abc/.cache/torch_extensions/py38_cu118/utils/build.ninja...
Building extension module utils...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/2] c++ -MMD -MF flatten_unflatten.o.d -DTORCH_EXTENSION_NAME=utils -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /home/users/abc/venv/lib/python3.8/site-packages/torch/include -isystem /home/users/abc/venv/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /home/users/abc/venv/lib/python3.8/site-packages/torch/include/TH -isystem /home/users/abc/venv/lib/python3.8/site-packages/torch/include/THC -isystem /usr/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -c /home/users/abc/venv/lib/python3.8/site-packages/deepspeed/ops/csrc/utils/flatten_unflatten.cpp -o flatten_unflatten.o
[2/2] c++ flatten_unflatten.o -shared -L/home/users/abc/venv/lib/python3.8/site-packages/torch/lib -lc10 -ltorch_cpu -ltorch -ltorch_python -o utils.so
Loading extension module utils...
Time to load utils op: 26.54496741294861 seconds
Parameter Offload: Total persistent parameters: 1070080 in 258 params
2023-05-16 19:29:44 ERROR [__main__] main failed
Traceback (most recent call last):
  File "/home/users/abc/dolly/training/trainer.py", line 334, in <module>
    main()
  File "/home/users/abc/venv/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/users/abc/venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/users/abc/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/users/abc/venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/users/abc/dolly/training/trainer.py", line 326, in main
    train(**kwargs)
  File "/home/users/abc/dolly/training/trainer.py", line 281, in train
    trainer.train()
  File "/home/users/abc/venv/lib/python3.8/site-packages/transformers/trainer.py", line 1664, in train
    return inner_training_loop(
  File "/home/users/abc/venv/lib/python3.8/site-packages/transformers/trainer.py", line 1741, in _inner_training_loop
    deepspeed_engine, optimizer, lr_scheduler = deepspeed_init(
  File "/home/users/abc/venv/lib/python3.8/site-packages/transformers/deepspeed.py", line 378, in deepspeed_init
    deepspeed_engine, optimizer, _, lr_scheduler = deepspeed.initialize(**kwargs)
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/__init__.py", line 165, in initialize
    engine = DeepSpeedEngine(args=args,
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 308, in __init__
    self._configure_optimizer(optimizer, model_parameters)
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 1173, in _configure_optimizer
    self.optimizer = self._configure_zero_optimizer(basic_optimizer)
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 1463, in _configure_zero_optimizer
    optimizer = DeepSpeedZeroOptimizer_Stage3(
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/runtime/zero/stage3.py", line 256, in __init__
    self._create_fp16_partitions_with_defragmentation(self.trainable_param_groups)
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/runtime/zero/stage3.py", line 600, in _create_fp16_partitions_with_defragmentation
    device_buffer = __class__.defragment(parameter_partitions)
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/runtime/zero/stage3.py", line 436, in defragment
    assert len(set(t.device for t in tensors)) == 1
AssertionError
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/users/abc/dolly/training/trainer.py", line 334, in <module>
    main()
  File "/home/users/abc/venv/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/users/abc/venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/users/abc/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/users/abc/venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/users/abc/dolly/training/trainer.py", line 326, in main
    train(**kwargs)
  File "/home/users/abc/dolly/training/trainer.py", line 281, in train
    trainer.train()
  File "/home/users/abc/venv/lib/python3.8/site-packages/transformers/trainer.py", line 1664, in train
    return inner_training_loop(
  File "/home/users/abc/venv/lib/python3.8/site-packages/transformers/trainer.py", line 1741, in _inner_training_loop
    deepspeed_engine, optimizer, lr_scheduler = deepspeed_init(
  File "/home/users/abc/venv/lib/python3.8/site-packages/transformers/deepspeed.py", line 378, in deepspeed_init
    deepspeed_engine, optimizer, _, lr_scheduler = deepspeed.initialize(**kwargs)
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/__init__.py", line 165, in initialize
    engine = DeepSpeedEngine(args=args,
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 308, in __init__
    self._configure_optimizer(optimizer, model_parameters)
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 1173, in _configure_optimizer
    self.optimizer = self._configure_zero_optimizer(basic_optimizer)
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 1463, in _configure_zero_optimizer
    optimizer = DeepSpeedZeroOptimizer_Stage3(
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/runtime/zero/stage3.py", line 256, in __init__
    self._create_fp16_partitions_with_defragmentation(self.trainable_param_groups)
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/runtime/zero/stage3.py", line 600, in _create_fp16_partitions_with_defragmentation
    device_buffer = __class__.defragment(parameter_partitions)
  File "/home/users/abc/venv/lib/python3.8/site-packages/deepspeed/runtime/zero/stage3.py", line 436, in defragment
    assert len(set(t.device for t in tensors)) == 1
AssertionError
[2023-05-16 19:29:48,876] [INFO] [launch.py:428:sigkill_handler] Killing subprocess 76847
[2023-05-16 19:29:48,876] [ERROR] [launch.py:434:sigkill_handler] ['/home/users/abc/venv/bin/python3.8', '-u', '-m', 'training.trainer', '--local_rank=0', '--deepspeed', '/home/users/abc/dolly/config/ds_z3_bf16_config.json', '--epochs', '1', '--local-output-dir', './dolly_2023-05-16_19-26-31', '--dbfs-output-dir', '', '--per-device-train-batch-size', '3', '--per-device-eval-batch-size', '3', '--lr', '1e-5', '--warmup-steps', '50', '--input-model', 'EleutherAI/pythia-2.8b', '--logging-steps', '1000', '--test-size', '10'] exits with return code = 1

@srowen Downgraded deepspeed to pip install deepspeed==0.8.3.
Now new issue:
Exception: >- DeepSpeed Op Builder: Installed CUDA version 9.1 does not match the version torch was compiled with 11.8, unable to compile cuda/cpp extensions without a matching cuda version.
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/users/chintan/dolly/training/trainer.py", line 334, in <module>
    main()
  File "/home/users/chintan/venv/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/users/chintan/venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/users/chintan/venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/users/chintan/venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/users/chintan/dolly/training/trainer.py", line 326, in main
    train(**kwargs)
  File "/home/users/chintan/dolly/training/trainer.py", line 281, in train
    trainer.train()
  File "/home/users/chintan/venv/lib/python3.8/site-packages/transformers/trainer.py", line 1664, in train
    return inner_training_loop(
  File "/home/users/chintan/venv/lib/python3.8/site-packages/transformers/trainer.py", line 1741, in _inner_training_loop
    deepspeed_engine, optimizer, lr_scheduler = deepspeed_init(
  File "/home/users/chintan/venv/lib/python3.8/site-packages/transformers/deepspeed.py", line 378, in deepspeed_init
    deepspeed_engine, optimizer, _, lr_scheduler = deepspeed.initialize(**kwargs)
  File "/home/users/chintan/venv/lib/python3.8/site-packages/deepspeed/__init__.py", line 125, in initialize
    engine = DeepSpeedEngine(args=args,
  File "/home/users/chintan/venv/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 340, in __init__
    self._configure_optimizer(optimizer, model_parameters)
  File "/home/users/chintan/venv/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 1283, in _configure_optimizer
    basic_optimizer = self._configure_basic_optimizer(model_parameters)
  File "/home/users/chintan/venv/lib/python3.8/site-packages/deepspeed/runtime/engine.py", line 1354, in _configure_basic_optimizer
    optimizer = DeepSpeedCPUAdam(model_parameters,
  File "/home/users/chintan/venv/lib/python3.8/site-packages/deepspeed/ops/adam/cpu_adam.py", line 96, in __init__
    self.ds_opt_adam = CPUAdamBuilder().load()
  File "/home/users/chintan/venv/lib/python3.8/site-packages/deepspeed/ops/op_builder/builder.py", line 485, in load
    return self.jit_load(verbose)
  File "/home/users/chintan/venv/lib/python3.8/site-packages/deepspeed/ops/op_builder/builder.py", line 500, in jit_load
    self.build_for_cpu = not assert_no_cuda_mismatch(self.name)
  File "/home/users/chintan/venv/lib/python3.8/site-packages/deepspeed/ops/op_builder/builder.py", line 105, in assert_no_cuda_mismatch
    raise Exception(
Exception: >- DeepSpeed Op Builder: Installed CUDA version 9.1 does not match the version torch was compiled with 11.8, unable to compile cuda/cpp extensions without a matching cuda version.
Exception ignored in: <function DeepSpeedCPUAdam.__del__ at 0x7f4a40638b80>
Traceback (most recent call last):
  File "/home/users/chintan/venv/lib/python3.8/site-packages/deepspeed/ops/adam/cpu_adam.py", line 110, in __del__
    self.ds_opt_adam.destroy_adam(self.opt_id)
AttributeError: 'DeepSpeedCPUAdam' object has no attribute 'ds_opt_adam'

I'm guessing you did not install the NVIDIA libs showing in the script

@srowen any specific libs? That would help me fix the issue quickly.

@srowen Already did all of the below installations:
# MAGIC %md
# MAGIC Install these additional NVIDIA libraries for Databricks Runtime 12.2 ML:

# COMMAND ----------

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/libcusparse-dev-11-3_<IP_ADDRESS>-1_amd64.deb -O /tmp/libcusparse-dev-11-3_<IP_ADDRESS>-1_amd64.deb && \
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/libcublas-dev-11-3_<IP_ADDRESS>-1_amd64.deb -O /tmp/libcublas-dev-11-3_<IP_ADDRESS>-1_amd64.deb && \
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/libcusolver-dev-11-3_<IP_ADDRESS>-1_amd64.deb -O /tmp/libcusolver-dev-11-3_<IP_ADDRESS>-1_amd64.deb && \
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/libcurand-dev-11-3_<IP_ADDRESS>-1_amd64.deb -O /tmp/libcurand-dev-11-3_<IP_ADDRESS>-1_amd64.deb && \
dpkg -i /tmp/libcusparse-dev-11-3_<IP_ADDRESS>-1_amd64.deb && \
dpkg -i /tmp/libcublas-dev-11-3_<IP_ADDRESS>-1_amd64.deb && \
dpkg -i /tmp/libcusolver-dev-11-3_<IP_ADDRESS>-1_amd64.deb && \
dpkg -i /tmp/libcurand-dev-11-3_<IP_ADDRESS>-1_amd64.deb

Still the same issue.

Oh, it's this, mismatched CUDA versions:
Exception: >- DeepSpeed Op Builder: Installed CUDA version 9.1 does not match the version torch was compiled with 11.8, unable to compile cuda/cpp extensions without a matching cuda version.
You need CUDA 11.x


Minimal buttons use CSS vars for intent colors in light mode

Signed-off-by: Petra Laohakul<EMAIL_ADDRESS>Hoist P/R Checklist
Pull request authors: Review and check off the below. Items that do not apply can also be
checked off to indicate they have been considered. If unclear if a step is relevant, please leave
unchecked and note in comments.

[x] Caught up with develop branch as of last change.
[N/A] Added CHANGELOG entry, or determined not required.
[N/A] Reviewed for breaking changes, added breaking-change label + CHANGELOG if so.
[N/A] Updated doc comments / prop-types, or determined not required.
[N/A] Reviewed and tested on Mobile, or determined not required.
[N/A] Created Toolbox branch / PR, or determined not required.

If your change is still a WIP, please use the "Create draft pull request" option in the split
button below to indicate it is not ready yet for a final review.

Pull request reviewers: when merging this P/R, please consider using a squash commit to
collapse multiple intermediate commits into a single commit representing the overall feature
change. This helps keep the commit log clean and easy to scan across releases. PRs containing a
single commit should be rebased when possible.

Haven't run this yet but looks good.
We should support for dark mode as well.  Ideally the intent variables would have defaults for dark mode that would work well for both the non-minimal buttons (background color) and these minimal/outlined versions (text/border color). I know you mentioned that the dark mode looked too bright - could you check that they are getting the colors we expect from:
https://github.com/xh/hoist-react/blob/develop/styles/vars.scss#L374
Maybe something there is not applying, or maybe these "muted" variations are still pretty bright.

Hi Anselm! I changed the color defaults for the dark mode intent vars to make them more visible. Bright was maybe not the right adjective -- they just weren't 'light/pastel-ly' enough.


Changes to OptionsParser breaks homebridge-ws locations.

44996a76f246068340c86f243eaf07db5bf842e5 added a RangeError for paths that do not start with "/".
The homebridge-ws allows one to pass "RedactedCity,GB" as an argument in to "locations". However these are not passed to OpenWeatherMap as resource paths, but as URL parmeters ("q=City,GB"). WSAccessory calls get(this.resource) with the location passed as an argument.
The changes to OptionsParser might be valid, but if so then homebridge-ws shouldn't be assuming it can pass locations as REST resources.
Sample Homebridge log:
[Weather] RedactedCity,GB: warning: resource: RedactedCity,GB: not a valid path

Did you just (re-)install homebridge-ws?

Release<EMAIL_ADDRESS>if you re-install homebridge-ws it should work again.

I’ll be able to test this out tonight. I just happened to reinstal homebridge-ws after your changed was pushed and saw the error.
Thanks for sending out a fix!

I've just reinstalled homebridge-ws and it's all working again. Thanks!


Scan COUNT results are not aggregated

Describe the bug
When passing the option count: true to find()/scan() the count results are not aggregated when the record set exceeds DB limits and requires paging to get all record counts. Instead, the count result of the last page is returned.
To Reproduce
Run any find/scan with the option count: true on a data set large enough that it will require paging in order count all records.
Fix provided: https://github.com/sensedeep/dynamodb-onetable/pull/375

Thanks for the PR. Merged.


Allow setting the tcp host

The tcp host is currently hard coded to <IP_ADDRESS>. This makes it impossible to use if PHP is in a docker environment with a loopback.
Adding a php.tcpHost option to the user settings to override this default.
Fixes #305.

Are you trying to run the language server in a Docker container while connecting to it from vscode running on the host?

Yep, you guessed it!

That is not a supported scenario (there are other issues about this). The language server needs access to the project on the file system. Even if you mounted the workspace into the container, the file paths that the language server sees and the file paths VS Code sends would not match.

Running PHP in docker does entail setting up a bash script that mounts the workspace properly (I mount my home folder identically in the docker container). I can verify that this works since I have this fix running on my machine. Other developers I work with wanted to use the same functionality, hence the PR :)
Example bash script:
#!/bin/sh

docker run --rm -i \
  -v /Users/thoom:/Users/thoom \
  -v $PWD:/code \
  -w /code \
  php:alpine php "$@"

Could it be that it only works for open files? I see no way how the language server would return the correct URIs for VS Code and vice versa.

I swear I searched before, but searching again, I did find a few mentions to this type of issue (so I apologize for logging another one).
As to your question, if you mean that it won't autocomplete for files that are not in the open VSCode workspace that does appear to be the case.

Pretty neat idea here. The extension "host" that the language server connects to doesn't necessarily need to be hardcoded, and in this case macOS (and probably Windows) has an issue with connecting to containers running on the local loopback device.
@felixfbecker in my personal use case, it works because the language server is accessing the files relative to the current working directory and I have mounted the current folder into the docker container and set it as the working directory.
For example, I have something like -v $PWD:/code -w /code in the docker command my php bash script runs within. This allows the language server to parse files correctly that are within the current workspace.
Out of curiosity, what are the potential downsides of allowing the host IP to be configurable?

LSP has always used absolute file URIs. The only scenario in which this would work is indeed if the file paths are 100% the same in the container, which means it is impossible to work if the OS are different.
I don't think it is a good idea to add a setting to configure the host if it is not properly working. It just creates the impression that it should work.

For this particular PR, I'm not sure what the downside of supporting changing the tcpHost IP address is. It could potentially have applications outside of using Docker to run the language server. I wouldn't suggest claiming that the purpose of allowing this is explicitly for supporting Docker, it just supports edge cases that users might have without needing to individually modify the extension code for their particular use case, one of which could be Docker.

What other use case would that be? Correct me if I am wrong, supporting to change the host to something other than localhost means supporting a scenario where the language server runs on a different host, which is exactly the scenario that won't work properly because of limitations in LSP.

While I was researching the initial problem, I learned that the language server supported a --tcp argument that wasn't exposed in the VSCode extension. By changing the host to my loopback IP, I was able to get it working in Docker for Mac.
It seemed like such a simple, non-breaking change that I figured I'd contribute back to the official project, especially since other developers I work with were looking for the same solution.

I appreciate that, but it won't work as you (or a regular user) would expect

Based on the discussion, putting the Language Server in a container wouldn't work, but I'm just not too sure why? There are a few hints here and there (in this thread) which seem to indicate that if the file structure on the host and in the container are identical, then the LS would work. Or have I misinterpreted?
I'm not too keen on installing PHP just for code Intellisense. I mean, I already use PHP containers to test my code, so why not take advantage of them even more?

It does work fine but since felix doesn't want to support it and didn't design it for that, and there is nothing in the LSP spec for it, he doesn't want to add it. Okay, that's his right. I just moved on the Inteliphense instead.


Batchloader and Caching

Hi Etienne,
Could it be, that the results of the batchloading are cached?
The second call doesn't stop by in the batchloader function of my delegate.
Is there an annotation I can switch it off with?
Sometimes caching makes sense but in other cases my calls get "Live"-data.
Thanks,
Markus

Yes, I've seen that.
And I'm not that sure.
I don't manage the data loader. The generated code uses an existing library.
For more information, you should take a look at the GraphQL-Java, as the generated code is just what you would have developed, to use it.
I'll also dig into it, to give some hints on how to do this, or what doc should be read.
Étienne

Yes, I've seen that.
And I'm not that sure.
I don't manage the data loader. The generated code uses an existing library.
For more information, you should take a look at the GraphQL-Java, as the generated code is just what you would have developed, to use it.
I'll also dig into it, to give some hints on how to do this, or what doc should be read.
Étienne

First of all I hacked around it by using random ids and managing an internal hashmap from real id to that random id.
But if we want to have it nice, we need to let GraphQLInvocation handle it.
https://blog.softwaremill.com/graphql-dataloader-in-spring-boot-singleton-or-request-scoped-16699436f680
If there would be two DataLoaderRegistries. One for the application scoped dataloaders and one "registry" that holds the information, so that the GraphQLInvocation can create new DataLoaders with each request and adds the application-scoped from the original DataLoaderRegistry.

First of all I hacked around it by using random ids and managing an internal hashmap from real id to that random id.
But if we want to have it nice, we need to let GraphQLInvocation handle it.
https://blog.softwaremill.com/graphql-dataloader-in-spring-boot-singleton-or-request-scoped-16699436f680
If there would be two DataLoaderRegistries. One for the application scoped dataloaders and one "registry" that holds the information, so that the GraphQLInvocation can create new DataLoaders with each request and adds the application-scoped from the original DataLoaderRegistry.

@etienne-sf
I saw, that you changed something in 1.12.1.
Now all dataloader are request-scoped.
But i was using my own @Primary GraphQLInvocation in order to log the calls with sleuth.
Do you know, how I can override/hook into it?

@etienne-sf
I saw, that you changed something in 1.12.1.
Now all dataloader are request-scoped.
But i was using my own @Primary GraphQLInvocation in order to log the calls with sleuth.
Do you know, how I can override/hook into it?

But I see the problem. There are already 2 and there can only be one Primary. So when I come in as 3rd.
So the only option would be, if you provide a switch so that you don't generate the GraphQLInvocation at all.
Right?
Maybe only a commented version of it...

There are other ways. I'll exclude the default one. So mine won't need to be Primary.
And I'll suggest the graphql-java-spring another way to do this that would not need any such component on the plug-in side.
In the meanwhile, can you test without as is, for instance by removing this component from the generated code ?
(Dont't forget to copy/paste the code that creates a new DataLoaderRegistry for each request invocation)

I just commit an update, in which I exclude the DefaultGraphQLInvocation. So the customGraphQLInvocation created by the plugin is no more marked as @Primary.
I'll submit a PR to the graphql-java-spring project, that allows to have a DataLoaderRegistry par request as it's done here, in the generated code.
I didn't create a new release yet.
Can you checkout the plugin, and test from a local build ?
If you do that, the plugin version is local-SNAPSHOT, instead of 1.12.1

Thanks Etienne,
It works. It stopped at the breakpoint in my GraphQLInvocation 👍
Merci,
Markus

Markus,
I'm about to release a new (minor) version.
Can you confirm to me that all your opened issues are solved, and just waiting for a new release ?

Hi Etienne,
Actually I am really working productive right now :)
So yes, you can build a new minor.
There is one "smal"l thing we like to see and that I might look into myself when I have time left.
You remember the Batchloader ID/value discussion?
It would be nice, if we could force the Batchloading with an annotation. Or define the return value of the returning object within that ResponseObject.
Thanks,
Markus


Geany support

:smiley:
But of course #13 should be fixed first :wink:

Geany support is not in our near term plans unfortunately!


'amplify init' UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'value' of undefined

Describe the bug
yarn variant of @aws-amplify/cli does not render type: string[] options of amplify init command and throws  exception UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'value' of undefined
Amplify CLI Version 4.44.0
To Reproduce
> amplify init
Note: It is recommended to run this command from the root of your app directory
? Enter a name for the project amplifycliaction
? Enter a name for the environment prod
? Choose your default editor: Visual Studio Code
? Choose the type of app that you're building       # List is not propagated to the terminal

(node:540213) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'value' of undefined
    at ListPrompt.getCurrentValue (/usr/local/share/.config/yarn/global/node_modules/inquirer/lib/prompts/list.js:148:53)
    at MapSubscriber._next (/usr/local/share/.config/yarn/global/node_modules/rxjs/internal/operators/map.js:49:35)
    at MapSubscriber.Subscriber.next (/usr/local/share/.config/yarn/global/node_modules/rxjs/internal/Subscriber.js:66:18)
    at TakeSubscriber._next (/usr/local/share/.config/yarn/global/node_modules/rxjs/internal/operators/take.js:54:30)
    at TakeSubscriber.Subscriber.next (/usr/local/share/.config/yarn/global/node_modules/rxjs/internal/Subscriber.js:66:18)
    at Interface.handler (/usr/local/share/.config/yarn/global/node_modules/rxjs/internal/observable/fromEvent.js:22:28)
    at Interface.emit (events.js:326:22)
    at Interface.EventEmitter.emit (domain.js:483:12)
    at Interface._onLine (readline.js:329:10)
    at Interface._line (readline.js:658:8)
(node:540213) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:540213) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Expected behavior
? Choose the type of app that you're building       # List is not propagated to the terminal
CLI option should render list of application types for selection but throws an unhandled exception. Issue is only replicable when installed with yarn global.
Desktop (please complete the following information):

OS: ubuntu, kernel 5.8.18
Node Version. v12.20.2

Log output
2021-02-19T17:38:18.211Z|info : amplify status core
2021-02-19T17:42:14.779Z|info : amplify init core
2021-02-19T17:46:49.339Z|info : amplify version core  {"version":true,"yes":false}

@sar is the Amplify CLI installed globally by using the following yarn command
yarn global add @aws-amplify/cli

@sar is the Amplify CLI installed globally by using the following yarn command
yarn global add @aws-amplify/cli

Correct. I was able to replicate on multiple new containers, installed with yarn (clean caches) but not npm. Seems like a dependency is getting missed.

Could you share the plugin.json file which should be located in <home-dir>/.amplify/plugins.json

Could you share the plugin.json file which should be located in <home-dir>/.amplify/plugins.json

{
  "pluginDirectories": [
    "cli-local-node-modules",
    "cli-parent-directory",
    "global-node-modules"
  ],
  "pluginPrefixes": [
    "amplify-"
  ],
  "userAddedLocations": [],
  "lastScanTime": "2021-02-18T14:35:35.277Z",
  "maxScanIntervalInSeconds": 86400,
  "plugins": {
    "core": [
      {
        "packageName": "@aws-amplify/cli",
        "packageVersion": "4.44.0",
        "packageLocation": "/usr/local/share/.config/yarn/global/node_modules/@aws-amplify/cli/",
        "manifest": {
          "name": "core",
          "type": "core",
          "commands": [
            "categories",
            "configure",
            "console",
            "delete",
            "env",
            "help",
            "init",
            "logout",
            "migrate",
            "plugin",
            "publish",
            "push",
            "pull",
            "run",
            "status",
            "uninstall",
            "upgrade",
            "version"
          ],
          "commandAliases": {
            "h": "help",
            "serve": "run",
            "ls": "status"
          }
        }
      }
    ]
  },
  "excluded": {}
}

From the plugin.json it looks like the CLI is not able to find the plugins needed by CLI. What version of yarn do you have running?

While waiting on a response I found the root cause with @yuth, which is the CLI cannot find any registered frontend plugins, which probably means a corrupt or unsupported installation of the CLI. Creating a PR for the better DX for this case.

@attilah I tried your code changes in my environment, but they don't work for me. It just stops with a different error now.

@volkanunsal the PR was solving the problem of giving a better error message instead of the stacktrace hell as it comes from a dependency. Your problem is something local.
As @yuth asked can you tell what version of yarn are you using? We only testing/supporting yarn classic (1.x) at the moment.
Could you remove the CLI from yarn global and try to give it a go and tell us what is the result:
$ npm install -g @aws-amplify/cli
$ amplify plugin scan

And then check what is in `plugin.json` and if amplify works or not.

Ah, ok. I figured the problem might be with yarn. I'll just use npm for the time being. Thanks.

Closing the issue as it is solved for customer.


Make wait return_when args available.

Fix #174
>>> import pywren
>>> pywren.ALL_COMPLETED
1

Before we had to do
>>> import pywren
>>> pywren.wait.ALL_COMPLETED
1

@apengwin pylint is failing with
C: 18, 0: Import "from pywren.wait import ALL_COMPLETED, ANY_COMPLETED, ALWAYS" should be placed at the top of the module (wrong-import-position)

Also @apengwin can you edit the PR title ? It currently reads Make wait return_when args available


fix: precompute search option keys, add metadata

:tada: This PR is included in version 1.2.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Block until file lock is released by second JVM

I've a question about file locking: is there an option to have a DBMaker.fileDB().make() wait until the lock held by another jvm is released?

+1

There is fileLockWait() to wait until file becomes unlocked by other application. There is also optional timeout parameter.


Refactor Framework-independent storybook code

why
Currently there is lots of copy-paste code for each framework (React, Vue, Angular, etc.)
what

[ ] Refactor this into a core library
[ ] Add documentation and tests

I'd be interested in contributing to this, as I would love to get storybook integration for a framework I'm working on. I see #2241 in progress. Anything else I could do to help push this forward?

@mAAdhaTTah Let's chat in our slack, it's actually quite easy already, the PR just addresses a bit of duplicated code.

Can we close this when https://github.com/storybooks/storybook/pull/3261 gets merged?

There is also #2636 this one.

Can someone gather a list of remaining duplications?

I'm going to start refactoring out html.ejs templates and error / nopreview views

Can anyone find anything else left to refactor? I can't =)


remove report a problem button

Canada.ca removed the share this page button. Might want to replace it with the "Did you find what you were looking for? " option.

Can be found on the Employment Insurance benefits - Canada.ca page which comes before (or to the left of) the Share this page button.

Closes #552


feat(optimizer): eliminate between statement

Rewrite Y BETWEEN X AND Z as X <= Y AND Y <= Z. And due to the support of this optimization rule, limbo should now be able to execute the BETWEEN AND statement.

I simply tested it on my laptop and the result is as follow:

When performing insertion, I found that inserting multiple values at the same time will prompt me unique conflict. I want to know if this is a bug or does limbo currently not support inserting multiple tuples at the same time?

When performing insertion, I found that inserting multiple values at the same time will prompt me unique conflict.

This is a known bug https://github.com/tursodatabase/limbo/issues/407 the write path in limbo is still in very early stages

I like the idea on paper, however this actually produces bytecode that is a bit less efficient. See sqlite implementation of BETWEEN:
sqlite> explain select first_name from users where first_name between 'Jack' and 'Jill';
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     10    0                    0   Start at 10
1     OpenRead       0     2     0     2              0   root=2 iDb=0; users
2     Rewind         0     9     0                    0
3       Column         0     1     1                    0   r[1]= cursor 0 column 1
4       Lt             3     8     1     BINARY-8       82  if r[1]<r[3] goto 8
5       Gt             4     8     1     BINARY-8       82  if r[1]>r[4] goto 8
6       Column         0     1     5                    0   r[5]= cursor 0 column 1
7       ResultRow      5     1     0                    0   output=r[5]
8     Next           0     3     0                    1
9     Halt           0     0     0                    0
10    Transaction    0     0     3     0              1   usesStmtJournal=0
11    String8        0     3     0     Jack           0   r[3]='Jack'
12    String8        0     4     0     Jill           0   r[4]='Jill'

The Column instruction appears only once. So there's two reasons to retain the BETWEEN expression:

It keeps sqlite bytecode compatibility and
Is a bit more efficient.

Would you like to take a crack at implementing BETWEEN the regular way?

Would you like to take a crack at implementing BETWEEN the regular way?

Sqlite between optimization
SQLite seems to perform related optimization when selecting indexes (converting between can help select index), I agree that it can be done better here. Since I am not familiar with sqlite enough, I can study the implementation of sqlite and come back later.

Sqlite between optimization

Yes, good point -- the key is this part I think:
Thus if the BETWEEN term is not used as an index constraint and instead must be used to test input rows, the expr1 expression is only evaluated once.

It keeps sqlite bytecode compatibility and
Is a bit more efficient.

I checked the implementation of sqlite. In fact, there is no problem with rewriting the rule itself. SQLite treats BETWEEN in the same way.   Sqlite exprCodeBetween
But what causes our opcode to be different from sqlite?

The opcode of sqlite is Lt, Gt, this is actually the same as what we did, the difference is:
sqlite convert Y between x and z to Y >= x AND Y <= z Sqlite src/expr.c L5798
And we are currently converting Y between x and z into x <= Y AND Y <= z. Obviously, this is logically equivalent, but if we want to pursue unification with SQLite in this regard, I can change the order, which does no harm.

The Column instruction appears only once.

This is because sqlite puts Y in the pDel variable (still using the above example), and pDel is a pointer, sqlite chooses to make the comp expr on both sides of AND point to pDel.
https://github.com/sqlite/sqlite/blob/master/src/expr.c#L5793
So back to Rust, our Column appears twice here because lhs.clone() is performed here. Why? Because lhs is a Box<Expr>. In other words, it is std::unique_ptr<T> in C++. We cannot let two Box point to the same address, otherwise we will have a double free problem.
https://github.com/tursodatabase/limbo/pull/490/files#diff-f81df8fcf98287b6b1303e97df2e09e7603dcf01efd1fdd672ef56ac03aef4eeR859
What if we want two elements to point to the same address at the same time? We can choose Rc, reference, or even choose unsafe. But no matter which implementation is implemented, there will be propagation (such as reference lifetime issues). In other words, we have sacrificed the versatility and neatness of the entire expr system for a very small optimization and may bring other potential risks.
For simplicity of implementation, I think it is acceptable to call lhs.clone() here.

As far as I know, all databasesrewrite the BETWEEN statement. No one will process BETWEEN alone. For example, this is running the above SQL in Duckdb.
Duckdb shell
And after studying the implementation of sqlite, I suspect that there is an error in the documentation of sqlite Sqlite The BETWEEN Optimization.
Eliminate between can be considered as part of simple expression rewriting. Regardless of different vendors, this should happen early in the optimizer's execution. In other words, the step of simple expression rewriting should have been completed when performing index selection, and no one will rewrite simple expressions repeatedly throughout the entire optimization process. The above SQLite processing process of between also reflects this point.
Duckdb Expression rewriter
Datafusion logic optimization order

Yeah I agree expression rewriting should happen before other optimizations, but is there a way for us to rewrite between as binary comparisons without generating extra instructions?

Yeah I agree expression rewriting should happen before other optimizations, but is there a way for us to rewrite between as binary comparisons without generating extra instructions?

Let's not talk about lifecycle issues for now, just look at it from the perspective of the type system. Binary(Box<Expr>, Operator, Box<Expr>), , this is the definition of Binary expression, which requires two Box. If we change the definition of lhs to something like Rc, we do not satisfy the definition of Binary.
What if we change the definition of Binary? Note that we want to share Y because we know that the expression we rewrite is X <= Y AND Y <= Z. But Binary itself does not have such a feature. In other words, we are damaging the rest of the system for a very small exception.
This is Datafusion's conversion for between statement. They also call expr.clone(). I still think it is worth to call lhs.clone here.
https://docs.rs/datafusion-optimizer/latest/src/datafusion_optimizer/simplify_expressions/expr_simplifier.rs.html#1452

Or we can change the entire definition of Between and Binary, changing the definition of Binary to Binary(Rc<Expr>, op, Rc<Expr>). Then make relevant changes in all places where Binary and Between appear. This is the quickest way, but if we want to keep the definition of Box and want the two Binaries to use the same memory address, there is no good way.

Yeah we can probably live with the extra bytecode instruction for now and optimize it away later. I'll review this properly when I have a better moment

rock


drivers: ssd1306: Support GPIO reset function

Add the reset function to the SSD1306 driver.
128x64 Display Mode requires a reset.
https://github.com/zephyrproject-rtos/zephyr/blob/master/dts/bindings/display/solomon%2Cssd1306fb.yaml#L68-L71
    reset-gpios:
      type: compound
      category: optional
      generation: define, use-prop-name

Related to reset in the SSD1306 Datasheet

Test environment
Board: stm32f4_disco
SSD1306 Module: Monochrome 1.3" 128x64 OLED graphic display
The test was done by creating the ssd1306_gpio_reset_with_stm32f4-disco branch.
west init zephyrproject -m https://github.com/KwonTae-young/zephyr/ --mr ssd1306_gpio_reset_with_stm32f4-disc
cd zephyrproject/zephyr
source zephyr-env.sh
mkdir samples/display/cfb/build
cd samples/display/cfb/build/
cmake -DBOARD=stm32f4_disco ..
make -j8 flash

console output

***** Booting Zephyr OS zephyr-v1.13.0-5042-geb9edf5a49 *****
initialized SSD1306
font width 20, font height 32
font width 15, font height 24
font width 10, font height 16
x_res 128, y_res 64, ppt 8, rows 8, cols 128
[00:00:00.014,389] <dbg> cfb.cfb_framebuffer_init: number of fonts 3

Before & After

Before
After

@KwonTae-young thanks. It is a issue that affects display boards without reset circuit.

Codecov Report

Merging #13663 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #13663   +/-   ##
=======================================
  Coverage   52.41%   52.41%
=======================================
  Files         322      322
  Lines       46596    46596
  Branches    10772    10772
=======================================
  Hits        24422    24422
  Misses      17267    17267
  Partials     4907     4907

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c2d5e7b...86cf9d6. Read the comment docs.


Refactor vpplink

This PR contains refactored of vpplink. It is still WIP and I will be updating the PR gradually when group of methods is done with refactor.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.5 out of 6 committers have signed the CLA.:white_check_mark: sknat:white_check_mark: onong:white_check_mark: ondrej-fabry:white_check_mark: AloysAugustin:white_check_mark: hedibouattour:x: hexoticYou have signed the CLA already but the status is still pending? Let us recheck it.


A mistake in "blend.py"

I think there is something wrong with the codes in 'blend.py' in latest version. That is on the 137th row. There is:
c._gamma(pos, self._img.shape, offset_int=source.center_int) for c in self.components

While in the gamma definition, there is:
def __init__(self, psfs=None, center=None, dy=0, dx=0):

The "pos" is correspongding to the "center", but the input shape of dy and dx does not match

There's indeed something wrong there, but not what you think. The first line doesn't call the constructor of Gamma, it calls its __call__ method. However, due to a change we made, this has a different signature now, so that line 137 is indeed incorrect and will raise an error.
The problem ties in with more sweeping changes we plan to make soon. I will therefore not fix it now, but replace the incorrect line with a proper error message.


Schema override for JSON

Add a schema_override option to daft.read_json to allow users to pass in a definitive schema.

I'm on it :)


Board Review: Distributed Tracing Preview 2

https://gist.github.com/SuyogSoti/3b1d8b22f40eb3c1f34300a0565db55e
This gist tries to clear some up confusion up. There are 2 main confusions to clear up.

When is the tracing enabled for each languages
What spans should be generated when the user writes certain code.

Regular requests vs Paging vs Polling?
How does Authentication and caching show up?

Related to: #501 #507

We just did a distributed tracing meeting with the arch board where this was covered.  Can you please work with @johanste to get specific answers to your questions?

I believe all these issues were covered in the meeting. Tracing requires opt in. Paging and polling, each outgoing request is traced. No relationship between requests (links) for preview 2. Authentication, each individual request is traced.