Tools for deterministic builds of standalone binaries and distribution archives

This PR adds several scripts and tools for making standalone binaries of the hwi.py script and for creating distribution archives that will go on pypi.org.
To achieve deterministic builds, the dependencies used must be locked to specific versions and hashes. To do this, I have added configuration files for using the Poetry dependency manager. Because Poetry uses a pyproject.toml file instead of setup.py, I have created a helper script which will automatically generate the proper setup.py file from pyproject.toml. The reason I chose Poetry instead of Pipenv for this task is because it has the ability to do deterministic builds of the distribution archives (python wheel and source tar for pypi.org) which Pipenv does not have.
Additionally scripts have been added to the newly created contrib/ folder which will perform the deterministic builds of the binaries and distribution archives. The builds of the binaries are done using pyinstaller. In order to build for different platforms, the contrib/build_bin.sh script needs to be run on each of the platforms we wish to release for. It can also be run in wine to do windows builds (see contrib/build_wine.sh). The configuration file that pyinstaller needs has also been added.
Lastly the pyenv version for this project has been bumped to 3.6.8 since using python 3.5.x produced standalone binaries that did not work.
This PR is built on #120 as reducing the number of dependencies fixed several issues with the standalone binary builds.

Concept ACK. Will review after 120 is merged.

The best way to review this is probably to just have everyone do the build and check that their hashes match. All of the builds can be done with contrib/build_bin.sh && contrib/build_dist.sh && contrib/build_wine.sh. Note that contrib/build_bin.sh builds the binary for your type of system, so it will differ between e.g. linux and MacOS.
These are what I got:
$ sha256sum dist/*
e5009335aa1a73edaf209182af271c812c686aee6da6831d4ccb442bdade42a4  dist/hwi
ceae5bfc684f3f17cf25e901055893734b69a3c93ea131e1292c7eabdf6c14b4  dist/hwi-0.0.5-py3-none-any.whl
b250bff5c054ffcc08163ac42d0879b9e098b57bf287c4287ffe72118c96d88c  dist/hwi-0.0.5.tar.gz
167b96ed351bddefa00440f8f9be0453ea46c91aa7dee1e2feffbddb571ac688  dist/hwi.exe

I had to jump through some hoops on macOS, including PYTHON_CONFIGURE_OPTS="--enable-framework" pyenv install 3.6.8 (see https://github.com/pyenv/pyenv/issues/1095). Hopefully that doesn't impact using the binaries.
For macOS maybe point to brew install libfaketime wine p7zip
The wine part doesn't work for me, see logs.
I'm get the same checksum for dist/hwi-0.0.5-py3-none-any.whl, but the others are different.
ef74d3df7a63876fa6cb5db8026294eacdb60a3909dc9b65f7c1274269ac8406  dist/hwi
ceae5bfc684f3f17cf25e901055893734b69a3c93ea131e1292c7eabdf6c14b4  dist/hwi-0.0.5-py3-none-any.whl
bdc1a36230a99a2e5c222ef16831ddef3fb9301ffc7a450a8c69eed161c3fcc6  dist/hwi-0.0.5.tar.gz

For macOS, I had issues getting faketime to work. I have a fix for poetry that removes the need to use faketime but is waiting to be merged.

Nice. Getting setup was easy.
Building with 88ce6ab03e6e51b33d09e8293a65bb86c9a00f70, I'm getting these hashes:
976354df3090a8b910fea64889481528be5db5ae75ec6b7771521aa90d7a2c8b  HWI/dist/hwi
ceae5bfc684f3f17cf25e901055893734b69a3c93ea131e1292c7eabdf6c14b4  HWI/dist/hwi-0.0.5-py3-none-any.whl
a4179a11c38cb7e6f84a1b2e2d93c912a3eb035b661a7a3172cb48316649cdbd  HWI/dist/hwi-0.0.5.tar.gz

When I try and do the windows build contrib/build_wine.sh, I'm seeing a page fault.
Backtrace and log prior:
backtrace.txt
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
003b:fixme:ntdll:NtQueryInformationJobObject stub: 0x48 9 0x22f960 144 0x22f8d0
003d:fixme:msvcrt:_configure_wide_argv (1) stub
003d:fixme:msvcrt:_initialize_wide_environment stub
003d:fixme:winsock:set_dont_fragment IP_DONTFRAGMENT for IPv6 not supported in this platform
003d:fixme:ntdll:server_get_file_info Unsupported info class e
003d:fixme:ntdll:server_get_file_info Unsupported info class e
0040:fixme:ntdll:server_get_file_info Unsupported info class e
0042:fixme:ntdll:NtQueryInformationJobObject stub: 0x20 9 0x23e420 144 0x23e390
0044:fixme:msvcrt:_configure_wide_argv (1) stub
0044:fixme:msvcrt:_initialize_wide_environment stub
0044:fixme:ntdll:server_get_file_info Unsupported info class e
0044:fixme:ntdll:server_get_file_info Unsupported info class e
0044:fixme:ntdll:server_get_file_info Unsupported info class e
wine: Unhandled page fault on read access to 0xffffffffffffffff at address 0x14003301d (thread 0042), starting debugger...

I've investigated the non-determinism and I think I've finally fixed it. Since for linux and macOS we are using pyenv which self compiles python, the python installation that each person was using was non-deterministic and thus introducing non-determinism. There are now instructions in docs/release-process.md which explain how to create a deterministic python install with pyenv. To test this, you'll need to uninstall Python 3.6.8 from pyenv first, and then you can install the deterministic one.
I've also added more to the build_bin.sh script which makes the python library files included in the build to be deterministic as building python deterministically is not enough.
Lastly I've added another travis job that builds the binary and checks that it works.
Below are the hashes of the result of the complete build.
14126ca063b72e357b36b52a3e44b413c48ed707c3ffb92af93da209feac1898  dist/hwi
7050ab9b4c50c3735d64ed02834b4528b4bf44b6c90cf5f7316f8bce045b03f2  dist/hwi-0.0.5-py3-none-any.whl
e74a8a333deb1aabcee23f17bda063edeb68de93f0bdd9ffd8c92f66817229d8  dist/hwi-0.0.5.tar.gz
2bea2fd3e3ea55ec75a28300eefafa8fd58e31b175226c330ab1fe650693b9c4  dist/hwi.exe

I haven't done the macOS build yet. I will do so once I get to a mac. I will also investigate the wine issue on macOS.

Tested building at 464c51db26ecced6624f7dcac86174fd64853687:
zipped /dist - dist.zip.
c59c643acd2c3ed2a75ba6229674e9e24743ab8b76689baef2f283dff3d4b286 dist/hwi
7050ab9b4c50c3735d64ed02834b4528b4bf44b6c90cf5f7316f8bce045b03f2 dist/hwi-0.0.5-py3-none-any.whl
77bad3308ed5bb9a3c85cc24076abd37759b57e68a00eba50a0baa52b6646639 dist/hwi-0.0.5.tar.gz

contrib/build_wine.sh fails with:
Extracting archive: libusb.7z
--
Path = libusb.7z
Type = 7z
Physical Size = 980895
Headers Size = 682
Method = LZMA2:23 BCJ
Solid = +
Blocks = 2

Everything is Ok

Folders: 18
Files: 32
Size:       6294517
Compressed: 980895
0032:fixme:msvcrt:_configure_wide_argv (1) stub
0032:fixme:msvcrt:_initialize_wide_environment stub
Traceback (most recent call last):
  File "c:\python3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\python3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "c:\python3\lib\site-packages\pip\__main__.py", line 16, in <module>
    from pip._internal import main as _main  # isort:skip # noqa
  File "c:\python3\lib\site-packages\pip\_internal\__init__.py", line 4, in <module>
    import locale
  File "c:\python3\lib\locale.py", line 16, in <module>
    import re
  File "c:\python3\lib\re.py", line 283, in <module>
    _pattern_type = type(sre_compile.compile("", 0))
  File "c:\python3\lib\sre_compile.py", line 579, in compile
    groupindex, indexgroup
TypeError: compile() argument 6 must be tuple, not list

It looks like faketime isn't working on mac which is why dist/hwi-0.0.5.tar.gz have different hashes.

@fanquake I got something different for the Mac build:
7050ab9b4c50c3735d64ed02834b4528b4bf44b6c90cf5f7316f8bce045b03f2  hwi-0.0.5-py3-none-any.whl
3083c8788120e4b4dd06a4471c29981ec690bbe6dd59ed1c7da5b3f282a94abf  hwi-0.0.5.tar.gz
53890b06a5e09c48ec0e6019b9d514dba27239df6933615befd7ab05622fa342  hwi.exe
7c6dac10288edc4623ebe31d708949d1a44f929bbaea0a05be86f3738f32d5dc  hwi

Can you sent me the result of
PYTHONHASHSEED=42 poetry run pyinstaller --additional-hooks-dir='contrib/pyinstaller-hooks/' hwi.py

This will create a folder named hwi in dist/ instead of the executable file. It's easier to debug with that than the single executable file.
When you ran contrib/build_bin.sh, did you get a bunch of errors about touch and strip not recognizing arguments?
Also I had no issues with getting wine to work. I was using macOS 10.12.6 with wine installed via homebrew.

PYTHONHASHSEED=42 poetry run pyinstaller --additional-hooks-dir='contrib/pyinstaller-hooks/' hwi.py

dist-hwi-folder.zip

When you ran contrib/build_bin.sh, did you get a bunch of errors about touch and strip not recognizing arguments?

Yep:
touch: illegal option -- d
usage:
touch [-A [-][[hh]mm]SS] [-acfhm] [-r file] [-t [[CC]YY]MMDDhhmm[.SS]] file ...

Also I had no issues with getting wine to work on a Mac. I was using macOS 10.12.6 with wine installed via homebrew.

I'm still seeing the TypeError: compile() argument 6 must be tuple, not list from above.
macOS 10.14.3, wine also installed via brew:
michael$ brew info wine
wine: stable 4.0 (bottled), HEAD
Run Windows applications without a copy of Microsoft Windows
https://www.winehq.org/
/usr/local/Cellar/wine/4.0 (8,989 files, 656.4MB) *
  Poured from bottle on 2019-03-01 at 11:08:06
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/wine.rb
==> Dependencies
Build: cmake ✔, makedepend ✘, pkg-config ✔
==> Requirements
Build: macOS <= 10.12 ✘
Required: macOS >= 10.11 ✔
==> Options
--HEAD
	Install HEAD version
==> Caveats
You may also want winetricks:
  brew install winetricks

Do we need winetricks installed?

Do we need winetricks installed?

No
I've just tried with a macOS 10.14.2 system and did not receive any errors with the wine build.. I honestly have no idea why it doesn't work.
I'm thinking getting deterministic builds across different machines is not going to be possible. It seems that Python itself is not deterministically built and because pyinstaller packages Python itself into the resulting binary, it will be non-deterministic so long as Python is non-deterministic. The best that I have been able to do is deterministic on the same machine.
The only one that will probably be deterministic is the Windows build as that uses Python's prebuilt binaries. I suppose the same could be done for macOS. However no such binaries exist for linux.

I've added a dockerfile and instructions to build the linux, windows, and pypi distributions using docker. This should be deterministic. Unfortunately we can't build macOS deterministically so that will remain non-deterministic.
$ sha256sum dist/*
22772246a469ddf5943bf2b1e8f415d754c4f1358015361dbbf425026a28555d  dist/hwi
7050ab9b4c50c3735d64ed02834b4528b4bf44b6c90cf5f7316f8bce045b03f2  dist/hwi-0.0.5-py3-none-any.whl
f450ecd14dcbb508321bc3d7151a516f94b6ba3876767a8615314d2ff6410279  dist/hwi-0.0.5.tar.gz
4e7130b4eef7a76d262a4a0a4139baf9304b10b42390b6ad89f50e9ee3cf0cbb  dist/hwi.exe

I've updated the dockerfile to use debian stretch-slim as the base image. There's also a travis job that runs the determinstic build, tests the linux binary, and outputs the hashes of the result. Another travis job does the macOS build but does not run any tests.
The hashes I got were:
503f375e671264dd6ab83f33d06568209e0a6a50773c12694f9d5b9df552383b  dist/hwi
7050ab9b4c50c3735d64ed02834b4528b4bf44b6c90cf5f7316f8bce045b03f2  dist/hwi-0.0.5-py3-none-any.whl
f450ecd14dcbb508321bc3d7151a516f94b6ba3876767a8615314d2ff6410279  dist/hwi-0.0.5.tar.gz
4e7130b4eef7a76d262a4a0a4139baf9304b10b42390b6ad89f50e9ee3cf0cbb  dist/hwi.exe

I'm seeing the same hashes as you:
root@e10a4cdc76f4:/opt/hwi# sha256sum dist/*
503f375e671264dd6ab83f33d06568209e0a6a50773c12694f9d5b9df552383b  dist/hwi
7050ab9b4c50c3735d64ed02834b4528b4bf44b6c90cf5f7316f8bce045b03f2  dist/hwi-0.0.5-py3-none-any.whl
f450ecd14dcbb508321bc3d7151a516f94b6ba3876767a8615314d2ff6410279  dist/hwi-0.0.5.tar.gz
4e7130b4eef7a76d262a4a0a4139baf9304b10b42390b6ad89f50e9ee3cf0cbb  dist/hwi.exe

As mentioned, sometimes the Windows build hangs (using contrib/build_wine.sh), particularly when installing dependencies via poetry. However running the commands manually inside the container works fine..

I've made a few changes to .travis.yml to build the macOS binary with an older version for better compatibility. That travis job will also upload the resulting binary to an AWS S3 bucket. That upload will be what I will use for releases.
Can someone with a mac test that this binary works and actually communicates with devices? It can be downloaded from https://s3.amazonaws.com/hwi-travis-builds/achow101/hardware-wallet-interface/114/114.4/hwi. This was built in a travis job on my personal fork of this repo: https://travis-ci.com/achow101/hardware-wallet-interface/jobs/183298042.

wget https://s3.amazonaws.com/hwi-travis-builds/achow101/hardware-wallet-interface/114/114.4/hwi
sha256sum hwi
d9acc15b507ce2079134b372e68278f71780ff0c762249c906778aefdf47f5f4  hwi

enumerate and getkeypool work

Merging as there is agreement on the produced binary hashes and that they all work.


Bump dex version

This will fix CI.
Depends on: https://build.opensuse.org/request/show/557306 and its request for IBS.

CI is green.


feat: Adjust Network Fees From Swap Amout

Description
This pull request is designed to modify the maximum balance input to account for the fees incurred. Specifically, if the gas fee is to be paid in the same token type as the transaction, it adjusts the transaction amount to ensure there is sufficient balance left to cover the gas fees. It's important to note, however, that due to fluctuations in the gas limit dust may remain. The primary goal of this update is to optimize transactions involving the maximum balance, rather than to minimize these dust amounts.
Error state when there is an error computing the fee or max balance quote:

Brief Changelog

Add 'isLoading' state to UI button
Subtract max balance tx fees to prevent tx failures.

Testing and Verifying

[ ] Can perform max balance transaction with OSMO as in token in swap

@Amosel Thank you for the review 🙌  Addressed the feedback on my latest commits


Update minimum required versions: 3.8

Make the project compatible with Python 3.8+
CI Add Tox and workflow tox-gh

Thanks.  There were a couple of casts in astview.py that seemed to be problematic too, causing an exception to be raised. I've shuffled things around a little (in the develop branch at the moment) and testing it looks fine with 3.8 now.
I'll pass on the additional build tooling for now thanks; I don't really know tox, but I'll be sure to take a look at some point.


$xdid(dialog,id).seltext comboex error

Having code:
xdialog -c themes 2 comboex 95 46 180 120 dropdown
xdid -a themes 2 0 0 0 0 0 Standard
xdid -a themes 2 0 0 0 0 0 Dark
xdid -c themes 2 2

When I execute:
//echo -a $xdid(themes,2).sel returns 2
//echo -a $xdid(themes,2,$xdid(themes,2).sel).text returns Dark
//echo -a $xdid(themes,2).seltext returns the following error:
D_ERROR $xdid ("themes 2 seltext" error: Invalid Item)
$xdid([dialog],[ID](,options)).prop
[dialog] = the dialog containing the control
[ID] = the ID of the control to affect
(options) = depends on the prop used
[prop] = depends on the control type

Tested on mIRC 7.48 with the latest 3.1-162. On 2.0.0 Build 1120 works fine.

fixed.
Small change in code messed it up at some point :/


Support Visual Studio Development

Hi,
I've been trying to build Material.Avalonia from Visual Studio 2022 and it fails because of the source generator. So I tried to build the generator itself and it also fails; more accurately the sample projects fail because the source generator is simply... not running.

TLDR: target netstandard2.0 in the source generator project in order for it to be compatible with Visual Studio.

I've been digging a bit and found that anything besides netstandard2.0 is unsupported by Microsoft for source generators or analyzers: https://github.com/dotnet/roslyn/issues/45162#issuecomment-645052356... and judging by the discussion in this issue and similar ones, it' unlikely this constraint will be relaxed any time soon. What's worse is that building with dotnet build or Rider works just fine.
Here are the error and the warning Visual Studio spits when trying to build:
error CS1061: 'AppBuilder' does not contain a definition for 'UseXamlDisplay' and no
accessible extension method 'UseXamlDisplay' accepting a first argument of type 'AppBuilder'
could be found (are you missing a using directive or an assembly reference?)

warning CS8032: An instance of analyzer ShowMeTheXaml.Avalonia.ShowMeTheXamlGenerator
cannot be created from D:\repositories\ShowMeTheXaml.Avalonia\ShowMeTheXaml.Avalonia.Generator\bin\Debug\netstandard2.1\ShowMeTheXaml.Avalonia.Generator.dll :
Could not load file or assembly 'netstandard, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies.
Le fichier spécifié est introuvable..

By looking at your source code, I see two ways (none of which is optimal) for the generator to target netstandard2.0:

Target netstandard2.0 instead of netstandard2.1 in ShowMeTheXaml.Avalonia.Generator.csproj then
Either remove the dependency to ShowMeTheXaml.Avalonia.csproj from ShowMeTheXaml.Avalonia.Generator.csproj: this is the easiest way, but will prevent users from automaticallty taking a dependence on ShowMeTheXaml.Avalonia when they reference the generator (it might still be possible if the nuget package is handcrafted but this is a hassle...) and thus, they'll have to explicitely reference both packages.
Or refrain from using RuntimeFeature in XamlDisplay.xaml.cs and target netstandard2.0 in ShowMeTheXaml.Avalonia.csproj as well: the downside here is that IsEditable won't be initialized to a suitable value by default (I don't know how important this is)

Thanks for reading me

Ehh, again sufferings from VS studio. I'll think what i can do.

Should be fixed in 2.5.1

A pity that Microsoft's own flagship IDE is lagging behind Rider and even dotnet build... Anyway, thanks for this!


Fix array fields in audio data

Signed-off-by: Olga Bulat<EMAIL_ADDRESS>Fixes

Fixes #161
Description
This PR makes required changes to the new audio data to make the CI tests pass.
Checklist

[x] My pull request has a descriptive title (not a vague title like Update index.md).
[ ] My pull request targets the default branch of the repository (main or master).
[x] My commit messages follow best practices.
[x] My code follows the established code style of the repository.
[x] I added or updated tests for the changes I made (if applicable).
[x] I added or updated documentation (if applicable).
[x] I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

Same, I'm going to go ahead and merge since it's night time for @dhruvkb. Thank you!


Proposed fiz for #1703 timewarp pull-exif timezone fails to update Ph…

…otos Timezone info when offset_seconds

Conflicted changes. Cancelled. Manual fixed on this commit


Lockdown Threadpool implementation

This needs to work so the test cases dont fail.

This was added today, its probably not the most intuitive way to just copy and paste the method but it will work for now since its just checking a boolean.


Cucumber 2.x

Jus a question and not a bug per se, but, are there any plans to support Cucumber.js 2.x?

No plans at the moment. What value does it add?

At a high level, essentially nothing, but 2.x is the actively developed branch.
There are a number of major coding changes, though (see changelog), which means it also not really a trivial change either.
I just worry about things going stale and unsupported.

Yes, that's a good point. I've not had chance to look over v2. I wonder if it's possible to support both versions, allowing users the choice of syntax and removing the need to force them to upgrade.

Again, I don't have any good answers here, but, what about the latest Cucumber.js 1.x version in the interim? Shouldn't be a huge jump from 1.2.2 to 1.3.3, I wouldn't think?


Correct the test semantics of the test case in ItemSetControllerTest

What's the purpose of this PR
Correct the test semantics of the test case in ItemSetControllerTest.
motivation

Reduce the code for constructing RestTemplate.
As only three places are expected to catch exceptions, there is no need to use noErrorHandlerRestTemplate in other places.

How about we use restTemplate and add try catch block in the test codes?


Says update is available when not

Description
As the image below show. The update sensor is "on" and the installed version is newer than what is "latest version"

Reproduction Steps
Dont know how to reproduce this
Expected behavior
The update sensor should be "off"
If applicable, add error logs.
No response
Device Firmware
1.3.1
Using stream component
Yes
Does camera work via official integrations?
N/A
Camera has all attributes filled out in developer tools
Yes
HASS Environment
HA OS
Search for similar issues
Yes
Additional information
No response

Update sensor shows as on when the version as latest reported is different to the currently installed version. I am not sure how this could happen. Maybe they rolled back the released version?
Could your try to reboot home assistant to see if it resolves?

A reboot solved it


find triggers insert (find({...}, populate:[...]) flushes pending changes under certain conditions.)

find({...}, populate:[...]) flushes pending changes under certain conditions.
  expect(await ItemRepo.count()).toBe(2)

  ItemRepo.create({user: 1, id: 3 }) // NO FLUSH

  // 🔴 the `find` triggers an insert:
  // |[query] select `u0`.* from `user` as `u0` where `u0`.`id` = 1 [took 0 ms]
  // |[query] begin
  // |[query] insert into `item` (`id`, `user_id`) values (3, 1) returning `id` [took 0 ms]
  // |[query] commit
  // |[query] select `i0`.* from `item` as `i0` where `i0`.`user_id` in (1) order by `i0`.`user_id` asc [took 0 ms]
  await UserRepo.find(1,{populate: ['items']})

  // [query] select count(*) as `count` from `item` as `i0` [took 0 ms]
  expect (await ItemRepo.count()).toBe(2) // ❌ 3

repro : https://stackblitz.com/edit/mikro-orm-repro-gl8t4t?file=test.ts
or https://github.com/rvion/mikro-orm-repro-gl8t4t/blob/main/test.ts
Expected behavior
no flush happening
Versions
latest

This is called auto flushing, you can disable it if you dont like it. It is a feature, not bug.
https://mikro-orm.io/docs/unit-of-work#flush-modes

Thanks, and sorry for the noise ! 🙏


Add zsh completion for 'docker service logs' command

ref. #28089
This can be planned for 1.13.1.
Signed-off-by: Steve Durrheimer<EMAIL_ADDRESS>

Should this be gated by an experimental flag?


update packages

:white_check_mark: Build TheIdServer 2.1.1-fix-dependencies1-0094 completed (commit https://github.com/Aguafrommars/TheIdServer/commit/2acd247fa7 by @)


Entities.ClientSecret to Core.Models.Secret mapping configuration

idsvr's Secret type sets its Type property to "SharedSecret" in the constructor; "SharedSecret" can be thought of as a default value. This is not reflected in the EF project. The EF project's mapper configuration is set in such a way that regardless of what the constructor does with the properties, automapper will come in afterward and reset everything to the database values. This is a source of insidious bugs when the value in the db for Secret Type is null.
Unfortunately EF6 has poor support for default values using code first at this time so instead of changing the Type column to 'Required' with a default value of "SharedSecret" I propose changing the mapper configuration to the following:
Mapper.CreateMap<Entities.ClientSecret, IdentityServer3.Core.Models.Secret>(MemberList.Destination)
            .ForMember(dest => dest.Type, opt => opt.Condition(srs => !srs.IsSourceValueNull));

This will change the mapping so if the value of Type is null in the db, the mapper will ignore it and the intended "SharedSecret" value set in the constructor will persist for the newly created object.
If this is acceptable I can submit a PR, but I would also recommend changing the Type column to non-nullable with a default value of "SharedSecret" in the next schema change as well.

Related? https://github.com/IdentityServer/IdentityServer3.EntityFramework/issues/69

Yes，I think we should keep all fields "NOT NULL"

It's not really related. I think the Clients table is set up appropriately and idsvr isn't making any assumptions on its values. Idsvr code is making an assumption that ClientSecrets.ClientSecretTypes will never be null however.

Done on dev -- take a look, please.


Do we still need a horizontal divider line now that PF4 Namespace Selector has one?

The lines kind of look redundant now that the PF4 controls have horizontal lines underneath them?

@mtho11, @serenamarie125 I think the divider has been removed and that this can be closed.

@jshaughn right, it doesn't exist anymore.


Add updates page

This page lists all the changes from the CHANGELOG in major and minor releases since 4.0.0, a similar way to the Service manual updates page.
The main aim is to be more easily findable than the changelog, and in a more accessible form.
The changelog would remain as the more exhaustive and developer-focused documentation.
It’s only linked to from the footer initially. We could have a separate "What’s new" page above it (like the Service manual What’s new page) which includes the most recent release notes, as well as any other recent major updates to the documentation site (eg new guides), links to blog posts, links to NHS frontend updates, and anything else we might want to call out?
Part of #101
Screenshot

This looks good, my main questions will be:

do we also include change to the document site here (I assume so) and if so should we make this clear in the lede for example: 'Changes made in each release of the prototype kit and the documentation'?
will we need to decide what counts as version changes for the document site to justify notes?

do we also include change to the document site here (I assume so) and if so should we make this clear in the lede for example: 'Changes made in each release of the prototype kit and the documentation'?
will we need to decide what counts as version changes for the document site to justify notes?

I was thinking not - that this page would purely be about releases of the kit (like the CHANGELOG but more accessible and within the site rather than on GitHub).
Changes to the documentation site aren’t versioned in the same way, and probably usually don’t need to be called out (continuous improvement and all that). But for major changes or new features we could mention them on a "What’s new" page which sits above the Updates page?

Maybe add notes on this page about what is counted and not (for example we won't include typos or entirely internal facing)

@vickytnz sounds good. I’ll add a note to the release guide about updating this page separately.


[$1000] [HOLD for payment 2023-05-03] Double-check we have offline feedback for sign-in page

Problem
Jason reported here that he didn't encounter a message indicating he was offline.

Do we not have any offline feedback on the sign-in page? Just ran into an issue on this slow wifi where I later discovered I was offline, but at the time it seemed like the site was down and/or I ran into a passwordless bug.

Creating an issue to check whether offline feedback has been enabled for sign in page.
Solution

Disable the button entirely
Move the offline message to be immediately below the chat input

This is a duplicated issue for this one which was created in the wrong repo.
 You'll only have to do this once, and this is how we'll hire you on Upwork.
Please follow these steps:

Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.

Format:
Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>

Proposal
Please re-state the problem that we are trying to solve in this issue.
User reported that he didn't encounter a message indicating he was offline, while trying to sign in. Also states he was in a slow connection that later went off.
What is the root cause of that problem?
Checked all platforms (iOS, Android, web, webmobile, desktop) and in all of them the offline message appears at the bottom of the page when going offline. The button also goes disabled.
The listener to show/hide the message is also working in all platforms. If you come back online, the message disappears and the button enables again.
I also tested with Network Link Conditioner to simulate a very bad connection:

Bandwith: 1kbps
Packets dropped: 95%
Delay: 5000

This also made the offline message appear and button disable, which is good because it's a very unreliable connection.
So it seems like either this is not an issue anymore, or maybe we have a UX problem (user did not see the message). It could be too small or too low in the page making it unnoticeable.
Indeed, on web in very tall screens, the message can get really low (see image)

Tall and wide

But that does not happen if the screen is narrow:

Narrow

What changes do you think we should make in order to solve the problem?
In SignInPageContent.js, adjust the positioning of OfflineIndicator in tall and wide screens so it stays closer to the form, below the button. This would also make it match more closely the narrow screen designs

Thank you! Great proposal @thiagobrez! Go ahead with the implementation 🚀

PR is now ready for review :)

Reviewing label has been removed, please complete the "BugZero Checklist".

The solution for this issue has been :rocket: deployed to production :rocket: in version 1.3.5-6 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

https://github.com/Expensify/App/pull/17705

If no regressions arise, payment will be issued on 2023-05-03. :confetti_ball:
After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

[ ] External issue reporter
[ ] Contributor that fixed the issue
[ ] Contributor+ that helped on the issue and/or PR

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

Merged PR within 3 business days of assignment - 50% bonus
Merged PR more than 9 business days after assignment - 50% penalty

Bug0 Triage Checklist (Main S/O)

[ ] This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
[ ] This bug is not a duplicate report (check E/App issues and #expensify-bugs)

If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead

[ ] This bug is reproducible using the reproduction steps in the OP. S/O

If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
If the reproduction steps aren't clear and you determine the correct steps, please update the OP.

[ ] This issue is filled out as thoroughly and clearly as possible

Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.

[ ] I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

Sorry @fedirjh !  Added Bug label to get a bug zero team member assigned to the issue. I'll also assign you to the issue. I saw no C+ assigned and that's why I forgot about the C+ review.

@joekaufmanexpensify Please help us with the C+ payment 🙏

Job added to Upwork: https://www.upwork.com/jobs/~012892ebc537d91822

Current assignee @joekaufmanexpensify is eligible for the External assigner, not assigning anyone new.

Current assignee @cristipaval is eligible for the External assigner, not assigning anyone new.

It's been a week with no regressions, issuing payment!

@fedirjh $1,000 sent and contract ended!

Upwork job closed.

Closing as this is all set!


compiler api: jsdoc parse optional params

Suggestion
Optional jsdoc parameters are currently lost on @param: https://jsdoc.app/tags-param.html
🔍 Search Terms
jsdoc optional params
⭐ Suggestion
Please allow for optional params parsing
  /**
  * @param {string} [somebody] - Somebody's name.
  */
  export function sayHello(somebody) {
  }

currently parses to tags:
   {
      name: "param",
      text: [
        {
          text: "somebody",
          kind: "parameterName",
        },
        {
          text: " ",
          kind: "space",
        },
        {
          text: "- Somebody's name.",
          kind: "text",
        },
      ],
    },

Possible suggestions:

add a node  kind: optional,
or keep the square brackets around the name so users have a way to parse the optional parameter name:

 {
    text: "[somebody]",
    kind: "parameterName",
  },

📃 Motivating Example
example test to reproduce https://github.com/atanasster/ts-issues/blob/master/tests/optional-param.test.ts

The services part of the Typescript API is not intended for semantic analysis, but for display. Parameter optionality is displayed on signature help, not on the parameter tag.
I'd recommend using the compiler part of the Typescript API, like so:
// existing code ..
const symbol = exports[0]
const params = ts.getALLJSDocTagsOfKind(
  symbol.valueDeclaration,
  ts.SyntaxKind.JSDocParameterTag
) as ts.JSDocParameterTag[]
for (const p of params) {
  p.isBracketed
  // inspect other properties of the tag here...
}

The compiler API exposes the parse tree and is intended to answer syntactic and semantic questions. It's more appropriate if you're not building an editor.

thank you very much, the compiler API it seems much more appropriate for what I am building. Will try to switch and will close all my jsdoc-related suggestions.
Any pointers for docs on the jsdoc compiler API?

Our documentation isn't very good, but the API isn't too complicated? The main wrinkle is that, say, getJSDocTags takes a node that you're interested in and the API provides you with all related tags, even those that aren't directly attached to that node.


[TECH-77] footer upgrade

Reformat the website footer according to Rev's suggestions:

give more breathing room for content
stack some of the info to make room for logo

i dont like that the logo isnt exactly in the middle but ive spent way too long trying so itll have to do for now 😠


feature to set AOAI API endpoint by environment variable, instead of AZURE_OPENAI_NAME

What would you like to be added?
Add feature to set AOAI API endpoint by environment variable, instead of AZURE_OPENAI_NAME.
Why is this needed?
To call AOAI via APIM, this feature is needed.

Simply, we can define API_BASE variable, instead of AZURE_OPENAI_NAME
e.g. https://github.com/ks6088ts-labs/azure-openai-playground/commit/736586b12869077e5eb039610cc50ec08ad93e3a
But this fix has breaking change, so we should find a safe way to fix.


Build failure: zulu8 with JavaFX

Steps To Reproduce
Attempt to build zulu8 with JavaFX: zulu8.override { enableJavaFX = true; }.
The build fails due to auto-patchelf being unable to satisfy dependencies.
The issue seems to occur for other versions of zulu as well.
Build log
nffvs8x2f4049m93j4aiap7kvxgi1ya4-zulu-ca-fx-jdk-8.0.402.drv.log
Additional context
Build was attempted in a flake with Nixpkgs pinned to f945939fd679284d736112d3d5410eb867f3b31c,
Notify maintainers

@wegank
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
- system: `"x86_64-linux"`
 - host os: `Linux 6.7.6-xanmod1, NixOS, 24.05 (Uakari), 24.05.20240228.9099616`
 - multi-user?: `yes`

 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.1`
 - nixpkgs: `/nix/store/cbbsvzjcqvzlylq9s7p6d2gyxh64q88p-source`

Add a :+1: reaction to issues you find important.

I remember encountering this during a last bump, but eventually abandoned it because something like libavcodec.so.53 point to obsolete versions of ffmpeg that no longer exist in Nixpkgs. I'll probably mark zulu with enableJavaFX = true on Linux as broken at some time this week.

Oh, there's actually autoPatchelfIgnoreMissingDeps for that. Would you please test if #294241 works?


UI: Accordion

Title not in a Text component

size doest change if changing global size

how to add clickable icons in the title?

Primary: double blue border on focus

Secondary: focus border has an offset

Title is the Text component.
Default value for Text component is s, so if you want to see how will be looks title of accordion, you can change s size in the root styles.
Basically we implemented default behavior for iconLeft and iconRight props (you can check how it will be) + icons property gives you the opportunity to add as many icons as you want.

Primary: set normalizer in your application or set rule for all buttons outline: none
Secondary: need to check.


Add Telemetry (Sentry)

Summary
We should add some sort of crash telemetry (Sentry) and performance monitoring! The specifics of how this works in mobile can be discovered after an initial integration.
Need to check:

Error exception catching (plus feedback?)
React component profiling
Release mapping
Sourcemaps

i'm largely satisfied with the implementation, however I think we're still seeing duplicate releases per release with different naming schemes... need to sort this out, but will keep an eye on.


Namespaced xml

I prepared this modification in order to handle the reading of an OEM file in XML format that has a namespace in it. The modification will remove from the tag the namespace, it is transparent for XML files without namespace.

Apologies for losing track of this -- I think I understand now. If you revert the changes to setup.py I can merge this in.

Hello @bradsease. As asked, I reverted the setup.py file.
Have a nice day.

Just a question, I encountered a problem installing the package directly from the repository due to the setup.py line you asked me to revert. It is possible to generalize the setup.py to avoid this problem. A "setup.cfg" could be useful in that sense

Sorry about that -- I'll work on making that more general. I was focused on getting it to work with my CI/CD and didn't think about local use.


Add Deno JS Indonesia

A secure runtime for JavaScript and TypeScript built with V8, Rust, and Tokio

Thanks
On 13 May 2020 13.56 +0800, Laode Muhammad Al Fatih<EMAIL_ADDRESS>wrote:

A secure runtime for JavaScript and TypeScript built with V8, Rust, and Tokio
You can view, comment on, or merge this pull request online at:
https://github.com/hendisantika/List-All-Programming-Telegram-Group/pull/165
Commit Summary
• Add Deno JS Indonesia
File Changes
• M README.md (1)
Patch Links:
• https://github.com/hendisantika/List-All-Programming-Telegram-Group/pull/165.patch
• https://github.com/hendisantika/List-All-Programming-Telegram-Group/pull/165.diff
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.


Some of JavaScript Blocks for API don't match translation file's sentences

Describe the bug
Some of JavaScript Sprite blocks (Image as below) don't match sentence of translation file.
For example, in the image below,  the first block shows the text "on sprite of kind  [kind] overlaps [otherKind] of kind [handler]", but in the translation file (https://crowdin.com/translate/kindscript/2958/en-ja) , the sentence is
on $sprite of kind $kind=spritekind overlaps $otherSprite of kind $otherKind=spritekind

The blocks in the  "Lifecycle" section don't match neither.
To Reproduce
Steps to reproduce the behavior:

Access to beta version (https://arcade.makecode.com/beta)
Create a project
Select "JavaScript" and  "Sprite" toolbox drawer

Expected behavior
Considering what the api is actually doing, the translation file should be like "on $sprite overlaps $otherSprite run $handler".
Screenshots

Desktop (please complete the following information):

OS: [e.g. iOS]   macOS Catalina 10.15.3
Browser [e.g. chrome, safari] Safari 13.0.5 (15608.5.11)

Hm, yeah, those snippets are showing up incorrectly / the translation looks correct actually; the text for on overlap should be showing up as
on [sprite] of kind [kind] overlaps [otherSprite] of kind [otherKind] [handler]

It's currently dropping [otherSprite] / replacing it with the next param, weird..


chore(Python): add graph

What kind of change does this PR introduce? (check at least one)

[ ] Bugfix
[x] New algorithm
[ ] Optimization in previous algorithms
[ ] Code style update
[ ] Refactor
[ ] Documentation
[ ] Other, please describe:

Briefly describe the changes in this PR
Added Graph Algorithm and its simple implementation
Language: Python

Other information:

I had re-forked MakeContributions/DSA. Should I again pull request?

I had re-forked MakeContributions/DSA. Should I again pull request?

Hi @hashfx,
You should make a marge from our main branch to your main branch


For #465: split dokku repository reset

Last lines of 434aa056 rultor release execution are failing due to a settings.xml file left on the repository, because the reset command is not being executed when there is a failure to push the repository to dokku.
+ rm -rf /home/r/.ssh
+ mkdir /home/r/.ssh
+ mv ../id_rsa ../id_rsa.pub /home/r/.ssh
+ chmod 600 /home/r/.ssh/id_rsa /home/r/.ssh/id_rsa.pub
+ echo -e 'Host *\n  StrictHostKeyChecking no\n  UserKnownHostsFile=/dev/null'
+ git remote add dokku dokku@dokku.jpeek.org:jpeek
+ git add settings.xml
+ git commit -m settings.xml
[__rultor a153e96] settings.xml
 1 file changed, 33 insertions(+)
 create mode 100644 settings.xml
++ git symbolic-ref --short HEAD
+ git push -f dokku __rultor:master
Warning: Permanently added 'dokku.jpeek.org,<IP_ADDRESS>' (ECDSA) to the list of known hosts.
Enumerating objects: 10077, done.
Counting objects: 100% (10077/10077), done.
Delta compression using up to 12 threads
Compressing objects: 100% (3266/3266), done.
Writing objects: 100% (10077/10077),712.31MMiB||113.07MMiB/s, done.
Total 10077 (delta 4817), reused 10068 (delta 4812)
remote: Resolving deltas: 100% (4817/4817),Kdone.K
remote: Reinitializing monit daemonK
remote: 1G-----> Cleaning up...K
remote: 1G-----> Building jpeek from herokuish...K
remote: 1GReinitializing monit daemonK
remote: 1G-----> Adding BUILD_ENV to build environment...K
remote: 1G1G       1G-----> Java app detectedK
remote: 1G1G-----> Installing JDK 1.8... doneK
remote: 1G1G-----> Installing Maven 3.6.2... doneK
remote: 1G1G-----> Executing MavenK
remote: 1G1G       $ mvn -DskipTests -Dinvoker.skip clean dependency:list installK
remote: Picked up JAVA_TOOL_OPTIONS: -Xmx1g -Xss512k -XX:CICompilerCount=2 -Dfile.encoding=UTF-8K
remote: 1G1G---------------------------------------------------K
remote: 1G1G       constituent[0]: file:/cache/.maven/conf/logging/K
remote: 1G1G       constituent[1]: file:/cache/.maven/lib/maven-resolver-spi-1.4.1.jarK
remote: 1G1G       constituent[2]:#K
remote: 1G1G       # There is insufficient memory for the Java Runtime Environment to continue.K
remote: 1G1G       # Native memory allocation (malloc) failed to allocate 32744 bytes for ChunkPool::allocateK
remote: 1G1G       file:# An error report file with more information is saved as:K
remote: 1G1G       # /tmp/build/hs_err_pid344.logK
remote: 1G1G       /cache/.maven/lib/commons-lang3-3.8.1.jarK
remote: 1G1G       constituent[3]: file:/cache/.maven/lib/org.eclipse.sisu.inject-0.3.3.jarK
remote: 1G1G       constituent[4]: file:/cache/.maven/lib/maven-builder-support-3.6.2.jarK
remote: 1G1G       constituent[5]: file:/cache/.maven/lib/javax.inject-1.jarK
remote: 1G1G       constituent[6]: file:/cache/.maven/lib/jsr250-api-1.0.jarK
remote: 1G1G       constituent[7]: file:/cache/.maven/lib/maven-embedder-3.6.2.jarK
remote: 1G1G       constituent[8]: file:/cache/.maven/lib/plexus-utils-3.2.1.jarK
remote: 1G1G       constituent[9]: file:/cache/.maven/lib/maven-shared-utils-3.2.1.jarK
remote: 1G1G       constituent[10]: file:/cache/.maven/lib/commons-cli-1.4.jarK
remote: 1G1G       constituent[11]: file:/cache/.maven/lib/org.eclipse.sisu.plexus-0.3.3.jarK
remote: 1G1G       constituent[12]: file:/cache/.maven/lib/wagon-http-3.3.3-shaded.jarK
remote: 1G1G       constituent[13]: file:/cache/.maven/lib/maven-settings-3.6.2.jarK
remote: 1G1G       constituent[14]: file:/cache/.maven/lib/jcl-over-slf4j-1.7.25.jarK
remote: 1G1G       constituent[15]: file:/cache/.maven/lib/maven-resolver-util-1.4.1.jarK
remote: 1G1G       constituent[16]: file:/cache/.maven/lib/wagon-file-3.3.3.jarK
remote: 1G1G       constituent[17]: file:/cache/.maven/lib/plexus-component-annotations-2.0.0.jarK
remote: 1G1G       constituent[18]: file:/cache/.maven/lib/slf4j-api-1.7.25.jarK
remote: 1G1G       constituent[19]: file:/cache/.maven/lib/maven-slf4j-provider-3.6.2.jarK
remote: 1G1G       constituent[20]: file:/cache/.maven/lib/maven-artifact-3.6.2.jarK
remote: 1G1G       constituent[21]: file:/cache/.maven/lib/maven-resolver-api-1.4.1.jarK
remote: 1G1G       constituent[22]: file:/cache/.maven/lib/plexus-cipher-1.7.jarK
remote: 1G1G       constituent[23]: file:/cache/.maven/lib/jansi-1.17.1.jarK
remote: 1G1G       constituent[24]: file:/cache/.maven/lib/maven-resolver-impl-1.4.1.jarK
remote: 1G1G       constituent[25]: file:/cache/.maven/lib/guava-25.1-android.jarK
remote: 1G1G       constituent[26]: file:/cache/.maven/lib/maven-core-3.6.2.jarK
remote: 1G1G       constituent[27]: file:/cache/.maven/lib/plexus-sec-dispatcher-1.4.jarK
remote: 1G1G       constituent[28]: file:/cache/.maven/lib/maven-resolver-provider-3.6.2.jarK
remote: 1G1G       constituent[29]: file:/cache/.maven/lib/maven-repository-metadata-3.6.2.jarK
remote: 1G1G       constituent[30]: file:/cache/.maven/lib/maven-resolver-#K
remote: 1G1G       # Compiler replay data is saved as:K
remote: 1G1G       # /tmp/build/replay_pid344.logK
remote: 1G1G       transporK
remote: 1G1G       !     ERROR: Failed to build app with MavenK
remote: 1G1G       We're sorry this build is failing! If you can't find the issue in application code,K
remote: 1G1G       please submit a ticket so we can help: https://help.heroku.com/K
remote: 1G1G       K
To dokku.jpeek.org:jpeek
 31m! [remote rejected]m __rultor -> master (pre-receive hook declined)
31merror: failed to push some refs to 'dokku@dokku.jpeek.org:jpeek'
m+ curl -f --connect-timeout 15 --retry 5 --insecure --retry-delay 30 https://i.jpeek.org
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 14661    0 14661    0     0  16736      0 --:--:-- --:--:-- --:--:-- 16717
+ mv /home/r/repo .
++ whoami
+ chown -R root repo
+ '[' -n '' ']'
++ whoami
+ sudo chown -R rultor repo
+ cd repo
+ for f in '"${sensitive[@]}"'
+ '[' -e settings.xml ']'
+ echo 'Sensitive file settings.xml is present, can'\''t release'
Sensitive file settings.xml is present, can't release
+ exit - 1
./run.sh: line 170: exit: -: numeric argument required
container a7aa8afc19d1b6718df5f3ade7ea650f8aecf1ffa48b7b3b5c784bc111401966 is dead
Sun Apr 26 06:51:15 CEST 2020

This fix the reset of the current repository even if we fail to publish to dokku server

@paulodamaso/z this pull request is too small, just 5 lines changed (less than 10), there will be no formal code review, see §53 and §28; in the future, try to make sure your pull requests are not too small; @paulodamaso/z please review this and merge or reject

@rultor release tag is 0.30.21

@rultor release tag is 0.30.21

@paulodamaso OK, I will release it now. Please check the progress here

Job gh:yegor256/jpeek#466 is not assigned, can't get performer

@rultor release tag is 0.30.21

@paulodamaso Oops, I failed. You can see the full log here (spent 18min)
remote: \u001b[1G\u001b[1G       [INFO] Finished at: 2020-04-26T14:29:18Z\u001b[K
remote: \u001b[1G\u001b[1G       [INFO] ------------------------------------------------------------------------\u001b[K
remote: \u001b[1G\u001b[1G       \u001b[1G-----> Discovering process types\u001b[K
remote: \u001b[1G\u001b[1G       Procfile declares types -> web\u001b[K
remote: \u001b[1G-----> Releasing jpeek (dokku/jpeek:latest)...\u001b[K
remote: \u001b[1G-----> Deploying jpeek (dokku/jpeek:latest)...\u001b[K
remote: \u001b[1G-----> App Procfile file found (/home/dokku/jpeek/DOKKU_PROCFILE)\u001b[K
remote: \u001b[1G       DOKKU_SCALE declares scale -> web=1\u001b[K
remote: \u001b[1G=====> Processing deployment checks\u001b[K
remote: \u001b[1G-----> Attempting pre-flight checks (web.1)\u001b[K
remote: \u001b[1G       CHECKS expected result:\u001b[K
remote: \u001b[1G       http://localhost/ => ""\u001b[K
remote: \u001b[1G       Attempt 1/5. Waiting for 5 seconds ...\u001b[K
remote: \u001b[1G       All checks successful!\u001b[K
remote: \u001b[1G=====> jpeek web container output:\u001b[K
remote: \u001b[1G       Picked up JAVA_TOOL_OPTIONS: -Xmx1g -Xss512k -XX:CICompilerCount=2 -Dfile.encoding=UTF-8\u001b[K
remote: \u001b[1G       BUILD io.sentry.DefaultSentryClientFactory: No 'stacktrace.app.packages' was configured, this option is highly recommended as it affects stacktrace grouping and display on Sentry. See documentation: https://docs.sentry.io/clients/java/config/#in-application-stack-frames\u001b[K
remote: \u001b[1G       BUILD io.sentry.DefaultSentryClientFactory: No 'stacktrace.app.packages' was configured, this option is highly recommended as it affects stacktrace grouping and display on Sentry. See documentation: https://docs.sentry.io/clients/java/config/#in-application-stack-frames\u001b[K
remote: \u001b[1G=====> end jpeek web container output\u001b[K
remote: \u001b[1G-----> Running post-deploy\u001b[K
remote: \u001b[1G-----> Overriding default nginx.conf with detected nginx.conf.sigil\u001b[K
remote: \u001b[1G-----> Configuring i.jpeek.org...(using app-supplied template)\u001b[K
remote: \u001b[1G-----> Creating https nginx.conf\u001b[K
remote: \u001b[1GWriting to /home/dokku/jpeek/nginx.conf.d/timeout.conf\u001b[K
remote: \u001b[1G-----> Setting proxy timeout to 60s\u001b[K
remote: \u001b[1GReloading nginx configuration (via systemctl): nginx.service.\u001b[K
remote: \u001b[1G       Reloading nginx\u001b[K
remote: \u001b[1GReinitializing monit daemon\u001b[K
remote: \u001b[1G-----> Renaming containers\u001b[K
remote: \u001b[1G       Found previous container(s) (feae197c9691) named jpeek.web.1\u001b[K
remote: \u001b[1G       Renaming container (feae197c9691) jpeek.web.1 to jpeek.web.1.1587911390\u001b[K
remote: \u001b[1G       Renaming container (f6f0ecc6d17c) gracious_albattani to jpeek.web.1\u001b[K
remote: \u001b[1G-----> Shutting down old containers in 60 seconds\u001b[K
remote: \u001b[1G       feae197c9691bc14b8c67b4a6262a1781e32c4a310f74844768c50260ac77d11\u001b[K
remote: \u001b[1G=====> Application deployed:\u001b[K
remote: \u001b[1G       http://i.jpeek.org\u001b[K
remote: \u001b[1G       https://i.jpeek.org\u001b[K
remote: \u001b[1G\u001b[K
To dokku.jpeek.org:jpeek
 + 334f5fd...453c557 __rultor -> master (forced update)
+ git reset HEAD~1
+ curl -f --connect-timeout 15 --retry 5 --insecure --retry-delay 30 https://i.jpeek.org
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
100  3917    0  3917    0     0   3901      0 --:--:--  0:00:01 --:--:--  3901
+ mv /home/r/repo .
++ whoami
+ chown -R root repo
+ '[' -n '' ']'
++ whoami
+ sudo chown -R rultor repo
+ cd repo
+ for f in '"${sensitive[@]}"'
+ '[' -e settings.xml ']'
+ echo 'Sensitive file settings.xml is present, can'\''t release'
Sensitive file settings.xml is present, can't release
+ exit - 1
./run.sh: line 170: exit: -: numeric argument required
container 117141a8634c70bae4b1f299b97b1000256bb18335c634c939e213db69e5c369 is dead
Sun Apr 26 16:30:39 CEST 2020


Setup Pipeline

Overall, the focus of these commits was on improving the robustness of the GitHub Actions pipeline, refining the handling of build variants, fixing path and artifact management issues, and ensuring proper documentation for contributors.

Pipeline and Build Automation

Variant handling in the pipeline: Multiple commits implemented and refined the handling of build variants (e.g., setting variants for test execution, reading variant matrices from JSON, and adapting matrix variables). This includes refactoring the build script for parallel processing and dependency handling between jobs.
Artifact management: Several attempts were made to improve artifact handling in GitHub Actions, such as uploading and managing the dist directory and ensuring paths were properly set.
Docker publishing: Key steps were added to publish Docker images to GitHub Container Registry (GHCR), including setting up jobs for login, publishing, and handling environment variables for deployment.

Fixes and Refinements

Naming and path adjustments: Fixes were applied to improve the naming conventions (e.g., renaming "variant" to "version" in the strategy matrix), handling of directory paths, and ensuring that only relevant directories are uploaded and downloaded as artifacts.
Job execution and dependency management: Several commits attempted to fix issues with job dependencies, including ensuring proper sequencing of jobs, downloading artifacts, and preparing the test environment.

Testing

Test execution fixes: Commits added and fixed the execution of tests in the pipeline, ensuring node setup, proper variant handling, and execution order.
Pipeline testing: Tests were incorporated into the pipeline for both the image build and variants, improving coverage and ensuring stability.

Documentation

Improved documentation: The readme, contributing guidelines, and code of conduct were refined to improve clarity for contributors and users, along with the addition of testing and core features information.
Contributing guidelines: Updated instructions to ensure proper contributions to the repository, reflecting the latest pipeline changes and project structure.

Additional Enhancements

Linting: A linting step was added to the pipeline to ensure code quality.
Dependabot integration: Dependabot was configured to automate dependency management.
Semantic release: Semantic release automation was implemented to streamline versioning and releases.

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


BUG: hypergeom.sf sometimes fails with multidimensional inputs

Describe your issue.
In older scipy versions (<1.7.0) hypergeom and its methods were able to be vectorized over higher dimensional arrays (e.g. 2D ndarray's). However, some of the methods (e.g. .sf)  do not work consistently here in version >=1.7.0. More specifically, hypergeom.sf and hypergeom.logsf sometimes fail on multdimensional arrays, but hypergeom.cdf (which is equivalent to 1-hypergeom.sf) is OK.
Reproducing Code Example
import numpy as np
from scipy.stats import hypergeom

arg1 = np.array([
    [ 8,  7],
    [ 6,  6],
    [ 6,  7],
    [ 8,  8],
    [10, 11]]
    )

arg2 = np.array([
    [1020, 1020],
    [1020, 1020],
    [1020, 1020],
    [1020, 1020],
    [1020, 1020]]
    )

arg3 = np.array([
    [508, 507],
    [506, 506],
    [506, 507],
    [508, 508],
    [510, 511]]
    )

arg4 = np.array([
    [20, 20],
    [20, 20],
    [20, 20],
    [20, 20],
    [20, 20]]
    )

# Test explict loop - this works
result_exp = np.empty_like(arg1, dtype=np.float64)
for j in range(arg1.shape[1]):
    result_exp[:,j] = hypergeom.sf(arg1[:,j], arg2[:,j], arg3[:,j], arg4[:,j])

# Test vectorised algo - works in <1.7.0, does not work in >=1.7.0
result_vec = hypergeom.sf(arg1, arg2, arg3, arg4)

# but cdf still works
hypergeom.cdf(arg1, arg2, arg3, arg4)

# Compare results
print('Results match?', np.array_equal(result_exp, result_vec))

Error message
Traceback (most recent call last):
  File "test_scipy2.py", line 42, in <module>
    result_vec = hypergeom.sf(arg1, arg2, arg3, arg4)
  File "/home/ascillitoe/Software/alibi-detect/venv/lib/python3.8/site-packages/scipy/stats/_distn_infrastructure.py", line 3304, in sf
    place(output, cond, np.clip(self._sf(*goodargs), 0, 1))
  File "/home/ascillitoe/Software/alibi-detect/venv/lib/python3.8/site-packages/scipy/stats/_discrete_distns.py", line 542, in _sf
    k2 = np.arange(quant + 1, draw + 1)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

SciPy/NumPy/Python version information
1.7.2 1.19.5 sys.version_info(major=3, minor=8, micro=11, releaselevel='final', serial=0)

I can't reproduce this in main, and it doesn't look like a platform-dependent sort of thing, so closing. Please re-open if you can reproduce it in main.
@tirthasheshpatel would you check me? Just copy-paste and execute the example above.

Confirmed. It works for me too on main.


Finally does not execute synchronously

I am noticing that finally does not execute synchronously with then, as one would expect
example:
function finish () {
   console.log('finally');
}

for(var i=0; i < 100; i++) {
   doSomethingAsync.then(function () {
      console.log('then');
   }).finally(finish);
}

does not always output
then
finally
then
finally
... (98 more times)

occasionally, i will see
then
then
finally
finally

if i refactor to
function finish () {
   console.log('finally');
}

for(var i=0; i < 100; i++) {
   doSomethingAsync.then(function () {
      console.log('then');
      finish();
   });
}

i will always see:
then
finally
then
finally
... (98 more times)

Is there something like a setTImeout(finally, 0) somewhere in there? It is acting like there is.
I am seeing this behaviour in an Ember app (v2.14.1) running on Chrome  62.0.3202.94 (Official Build) (64-bit) on OSX Sierra

I am noticing that finally does not execute synchronously with then, as one would expect

I would not expect this.

I do actually not expect the output you are getting, and my tests confirm. I believe your example is incomplete.
 If you can provide a concrete example that violates the above ^, I'll gladly investigate.

I am confused by the examples you posted
Why are you doing then chained to then ?
It should be then chained to finally

This
doSomethingAsync.then(function () {
      console.log('then');
      finish();
   });

should be equal to
doSomethingAsync.then(function () {
      console.log('then');
   }).finally(finish);

This might be the way promises are expected to function, but that would be a real shocker to me, it should be a "tight run loop" as it would be with callbacks in JS.

uggh, just confirmed that native promises do this as well
for(var i=0; i < 100; i++) {
    let p = new Promise(function (resolve, reject) {
		resolve();
    });
    p.then(function (i) { console.log(i) }.bind(null, i)).then(function (i) { console.log(i) }.bind(null, i))
}

Wow, that is really not nice for users, oh well - thanks!

If you're curious https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Using_promises#Timing
Apparently its a feature

Why are you doing then chained to then ?

finally is just then under the hood:
https://github.com/tildeio/rsvp.js/blob/12b646c01623a57c4eff37b9b9864e6e892dd025/lib/rsvp/promise.js#L226-L232

Apparently its a feature - which is crazy to me - so "anti JS" to break the event loop.

👍  on digging deeper.

Apparently its a feature - which is crazy to me - so "anti JS" to break the event loop.

This feature is no accident.

there is nothing async in the first then so the second one should fire immediately.

Actually, this would be hazardous and promises where specifically designed to avoid this common problem. Ones code should not perform differently based on if the data is cached or not.
I believe @isaacs (of npm/node fame) wrote a great description of this problem: http://blog.izs.me/post/59142742143/designing-apis-for-asynchrony
http://blog.izs.me/post/59142742143/designing-apis-for-asynchrony

Not sure i agree that it would be hazardous, still feels bizarre to me given the nature of JS and the event loop, but i do agree that making everything async makes things simpler.
And thanks for the link, nice having some historical context!


Support for yarn & don't assume --save-dev

Laravel Mix Version: 5.0.7
Node Version: 10.19.0
NPM Version: 6.13.4
Yarn Version: 1.22.4
OS: Win10 1909

Description 1:
When running the webpack bundler using mix as the config wrapper and you have missing dependencies, mix automatically tries to install dependencies but assumes you are using npm as the package manager. We use yarn & need to cancel the build in order to manually install the missing dependencies.
> cross-env NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js "--watch"

Additional dependencies must be installed. This will only take a moment.

Running: npm install sass --save-dev --production=false

Description 2:
Mix assumes you want to install the missing dependencies using flags --save-dev --production=false. This assumption should not be made if you are automating steps on behalf of the user.
Since Laravel is not a nodejs application, none of the node packages are used in production, making the distinction between prod & dev arbitrary. We only install packages used for testing as dev dependencies, the rest as prod dependencies.
Steps To Reproduce:

Run yarn install to generate your yarn.lock file.
Make sure one of the dependencies is missing (in our example the sass package).
Run the bundler cross-env NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js "--watch"

Yeah I think we can safely get rid of the --production flag.
In reference to Yarn, we used to check which tool you used, but I'd rather not do that. I'm open to suggestions, though.


fix remainder loop, add partial shuffle

avx_xorshift128plus_shuffle32 seem to be limited to buffers with length that is multiple of 8. Or am I missing something? I've got a crash, and then added reminder loop - then things seem to work correctly.
Also, add an option for a partial shuffle (both in avx and non-avx versions).

I think that this is wrong:

if (lower_index_inclusive==0) lower_index_inclusive=1;

But I'll merge and fix.

Oh. I see you had fixed it. Good.


Loop unrolling takes up to 16 min. for some UE5 shaders

Hi all,
We are currently investigating a shader compile regression for some of our compute shaders in Unreal Engine 5 that takes up to 16 min. and it seems that 99% of that time is due to the SPIRV-Tools loop unrolling pass; disabling just this one pass cuts down the time to mere seconds.
To be fair, those shaders are indeed pretty heavy and the one I'm currently looking at has 271 loops, all of which are fixed 8 iterations and pretty small though, like this one (as translated back to GLSL):
    for (uint _4556 = 0u; _4556 < 8u; )
    {
        _436[_4556] = _424[_4556] * _4554;
        _4556++;
        continue;
    }

I was hoping @s-perron might have some insight to help out here as he worked on the CFG traversal once to use structured order (see commit 545d158). Using LLVM's default traversal via ForEachBlockInReversePostOrder roughly cuts down the time in half but this is still way longer than what the DXIL backend is doing. I also noticed that ComputeStructuredOrder builds up a linked list (std::list) with over 51k elements. Correct me if I'm wrong, but it looks like the loop unrolling pass does that for every single loop and I don't understand why there are so many elements necessary to unroll a tiny loop as shown above.
Hopefully we can identify some low hanging fruit and I'm happy to share some of these UE5 shaders, but would like to do that via email for license reasons.
Thank you
Laura

@s-perron that would be great. Can you send me your email address please?

I found the problem, and I believe I have a fix.  I'll open a PR in either today or tomorrow.
The unroll the loop, spirv-opt needs a list of the basic blocks in the loop in structured order.  The current code that computes the structured order for these blocks does not know it can stop at the merge block for the loop.  It will include every basic block in the function that can be reached from the loop header.  Since the function is larger, and gets larger after every loop is unrolled, this time it takes is noticeable.
The solution will be to add a new function that will also take an end block when computing the structured order.  Then the search can stop at the merge node.

With this change, the compilation still seems a little slower than it should be, but it is much faster than it was.  I will probably not look into it any further.

Thanks a lot Steven, we appreciate your effort.

Thank you very much Steven! What compilation time do we end up with that fix concretely?

You will want to measure it yourself.  We would have very different setups.  The numbers might not be directly comparable, but hopefully the gain will be similar.  I'm seeing it take about 1/3 of the time that it use to.  Hopefully it is around 5-6 mins for you.

I did run some tests and it cuts down the time to about 8 min. which is roughly 40% of what it was before, so already a huge improvement. It seems to me that SPIRV-Tools doesn't perform as many constant folding techniques as the DXIL backend does, like greaterLessThan intrinsics etc., but that's something for a different ticket/PR and I'm looking into this myself right now.
Thanks again for your help @s-perron


Provide easy access to specific consensus hashes when parsing

This PR adds a debug option to specify which blocks to generate a consensus hash for during parsing.
Previously to confirm a consensus hash for a particular block it was required to enable -omnidebug=consensus_hash_every_block during parsing to log the hash for every block which caused a significant slow down due to the extra work involved.
This leads to circumstances where to validate a single consensus hash it is neccessary to perform vastly more work than necessary.
This PR adds a -omnishowblockconsensushash startup option which can be used to generate consensus hashes for specific blocks.
For example, to validate the checkpoint for block 450,000 without using seed block filtering we can use:
./omnicored --startclean --omniseedblockfilter=false --omnishowblockconsensushash=450000

Which will then cause a consensus hash to be generated for the corresponding block and written to the log.  Multiple instances of the parameter can be used to specify multiple blocks to generate consensus hashes for.

Nice!
Can you add a description for omnishowblockconsensushash please here?
https://github.com/OmniLayer/omnicore/blob/develop/src/init.cpp#L506

Sure will do :)

Sorry bud forgot all about this one, only realized I'd forgotten when I was going over my notes to prepare for tomorrows meeting d'oh!
Descriptions added as requested bud :)

Thanks! :)


ccx: Initialize LPSPI2 bus

Initialize LPSPI2  and an interrupt signal on ccx board as per schematics.

Needs to be enabled in menuconfig and initialized in board logic


Pipe UE Stats to Sentry Performance Monitoring

Unreal has a robust Performance and Profiling system that is limited by the fact it keeps all of the information local. It would be incredibly useful to hook into these metrics and push them up to Sentry's Performance Monitoring system. By properly identifying Releases, game developers will be able to see the performance impact of their changes over time (and across platforms).
The most useful (in my biased opinion) would be:

CPU stats for ensuring players aren't hitting CPU bottlenecks (ex: CPU % utilized, per core)
GPU stats for GPU bottlenecks (ex: GPU memory usage, FPS)
Network Insights for keeping an eye on traffic size and ping in a multiplayer game

Thanks for raising this. Makes total sense.
We need to look into this in more details

Hey @bruno-garcia - Any updates here on if you're considering this for your roadmap?

@edanker-rs it definitely is on the roadmap, it's just a question of when.

perf API was added here: https://github.com/getsentry/sentry-unreal/pull/470
But are these stats "metrics"? If so, only place we can add them now are part of span metrics


Chrome fillText() Clipping Large Text

My specs:

Mac OS X El Capitan - 10.11.3
Chrome 48

When approaching larger text sizes, usually around 100px+, the text is getting clipped. I know there is some behind the scenes action going on to get more accurate bounding boxes for text alignment, and I think it might be happening due to that. This is not happening in Firefox 44.0.2.
Here is a demo link and screenshot below: http://codepen.io/jackrugile/pen/e72ce88783098194ea7a5c3de39a29e4

Oh my, I thought noone will use such big fonts :D
Your prediction is good - I am measuring real font size using pixel data - but I've set the test canvas height to 100 - so it doesn't detects font larger than that.
Now to get rid of antialiasing on Chrome (windows) - I render the font in an intermediate canvas then scale it down. I think I will disable that for fonts larger than 32px - nevertheless fixed height in measuring process needs a fix too.
Thanks for notice.

Okay - theoretically fixed.

Confirmed fixed. Thanks!
Heh, been working on something full screen, so I'm needing the bigger sizes for a title screen :)


Error: Deployment to Oxygen failed. Check the 'Build and Publish to Oxygen' step for more information.

Github action fails to deploy with the following .yml and logs.
Here's the section in the oxygen-deployment-######.yml file
      - name: Build and Publish to Oxygen
        id: deploy
        uses: shopify/oxygenctl-action@v4
        with:
          oxygen_deployment_token: ${{ secrets.OXYGEN_DEPLOYMENT_TOKEN_###### }}
          build_command: "HYDROGEN_ASSET_BASE_URL=$OXYGEN_ASSET_BASE_URL yarn build"

Create Github Deployment
Run shopify/github-deployment-action@v1
Error: Deployment to Oxygen failed. Check the 'Build and Publish to Oxygen' step for more information.

Build and Publish to Oxygen
🚀 Uploading deployment...
...
✅ Done in 4.40s.
...
📡 Checking deployment...
🔁 Trying again in 5s...
🔁 Trying again in 10s...
🔁 Trying again in 15s...
🔁 Trying again in 30s...
🔁 Trying again in 1m0s...
🤕 The deployment was uploaded but cannot be reached yet.

After chatting with support I was able to get around this, by adding oxygen_health_check: false. Is this the recommended route to fix this or is there another issue that is causing this?
      - name: Build and Publish to Oxygen
        id: deploy
        uses: shopify/oxygenctl-action@v4
        with:
          oxygen_deployment_token: ${{ secrets.OXYGEN_DEPLOYMENT_TOKEN_###### }}
          build_command: "HYDROGEN_ASSET_BASE_URL=$OXYGEN_ASSET_BASE_URL yarn build"
          oxygen_health_check: false

Thanks!

We had some issues at some point with the health check (can't remember exact timeline), but this should be resolved now


Selenide Chrome Firefox Edge Web Driver Manager

The problem
As the default open() method called chromeDriver, is there any method to customize this to support firefox and edge browser?
Details
Want to do cross browser testing using chromeDriver and FirefoxDriver supply from commandline value.
Tell us about your environment

Selenide Version:  6.6
Chrome\Firefox\IE Version:  Chrome 106
Browser Driver Version:
Selenium Version:  4
OS Version:  Win 10

Code To Reproduce Issue [ Good To Have ]
None, Looking for recommendation

AFAIK we use -Dselenide.browser=edge to set a browser using system prop.

How to read the properties from commandline and set the corresponding driver?

How to read the properties from commandline and set the corresponding driver?

I usually use Maven so in my case mvn -Dselenide.browser=firefox clean test. Also you can change the browser in your code ```
//(before open() method)
Configuration.browser = "firefox";
which is less flexible. You can refer to [quick guide](https://github.com/soydivision/Selenide-quick-FAQ), although it needs a few updates, generally you should be fine.

I tried with this Configuration.browser = System.getProperty("browser"); from -Dbrowser=firefox test but it throws exception like below:
SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:565)
at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:242)
at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:157)

I tried with this Configuration.browser = System.getProperty("browser"); from -Dbrowser=firefox test but it throws exception like below:
SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure. at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:565) at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:242) at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:157)
PS: I can start Chrome session using the method above but not firefox and edge browser.

Are you using any sort of builder e.g. maven, ant, gradle? What you've written above makes no sense. You don't need to to pass a browser value from command line, all you have to do is just set it using -Dselenide.browser=firefox. Or inside the code before the open() method you can use this line of code Configuration.browser = "edge";. Again. Selenide is not Selenium. Just replace -Dbrowser=firefox to -Dselenide.browser=firefox in your CLI command.

The answer is given in wiki: https://github.com/selenide/selenide/wiki/How-Selenide-creates-WebDriver
As mentioned above, you need to set system property selenide.browser to firefox. OR you can just add this line in the beginning of your tests: Configuration.browser = "firefox";.
I see you did it; now you are getting SessionNotCreatedException.
And this is not a Selenide problem. You need to read all your logs and find out the reason why Firefox driver failed to start. There are many possible reasons, e.g. Firefox browser is not installed on that machine.


Have public recipes

Use case I'm imagining:

I am a GME stonks trader, and I get my information from $PUBLIC_DATA_SOURCE
I know about a $PUBLIC_DATA_SOURCE recipe from here that gets my data whipped into some format I really like
I can plug a public recipe into a github action or some other favorite way of working, and make a pipeline I like

I'd imagine so!

This is a thing now too: https://github.com/dstockto/csv-chef-recipes


chore: move SetCwdToFileDir to new workdir package

so it can be imported into the file domain (otherwise there were import cycle issues when importing "common"). I'm entirely open to different package names, feel free to nitpick that!
I split this out into it's own PR because the file domain PR is a big mess and I know that there will be some merge conflicts with other in-flight PRs. I'm happy to wait for other PRs to merge and fix conflicts in this one.

I can handle resolving file locations differently, I was just trying to use what we had in place already.
Resolving filepaths is easiest if I can handle it by cd'ing into the directory that the paths are relative to, but it's not strictly necessary and if we're getting away from the directory-changing behavior altogether I'll handle it differently. (I don't like having to calculate absolute filepaths because once you get into different OSes and add in symlinks some things get really buggy really quickly, but it's all doable)


Timestamp column information alignment inconsistency

To fix:

[ ] Timestamp label alignment
[ ] Cell information alignment

Please combine with #2031


mypy plugin dataclass_transform decorated model bug fix.

mypy plugin dataclass_transform_spec reset bug fixed.
Fixes the issue.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 91.86%. Comparing base (aea6311) to head (4945e1d).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##              dev     #186   +/-   ##
=======================================
  Coverage   91.86%   91.86%
=======================================
  Files          27       27
  Lines        1537     1537
=======================================
  Hits         1412     1412
  Misses        125      125

Flag
Coverage Δ

unittests
91.86% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[e2e]: delegator-generic create main checkout slsa3

Repo: https://github.com/slsa-framework/example-package/tree/v48.0.194
Run: https://github.com/slsa-framework/example-package/actions/runs/7711570061
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.delegator-generic.create.main.checkout.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.delegator-generic.create.main.checkout.slsa3.yml
Trigger: create
Branch: v48.0.194
Date: Tue Jan 30 13:04:50 UTC 2024

Repo: https://github.com/slsa-framework/example-package/tree/v48.0.196
Run: https://github.com/slsa-framework/example-package/actions/runs/7741676723
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.delegator-generic.create.main.checkout.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.delegator-generic.create.main.checkout.slsa3.yml
Trigger: create
Branch: v48.0.196
Date: Thu Feb  1 13:08:53 UTC 2024
Tests are passing now. Closing this issue.


Documentation updates and adding the Docc plugin

Spreading the review load. This one is pretty simple as it starts to document stable internals


Error

Hello, can you please kindly guide me to fix this problem? Thank you

Hi @BardaktakiAdam, I haven't seen this error before.
There is a related issue due to the user interface changes in the IMDb website: https://github.com/cecobask/imdb-trakt-sync/issues/39
Once I get some time over the next few days I will resolve the issues.
Thanks for reporting!

Hi @BardaktakiAdam, I haven't seen this error before. There is a related issue due to the user interface changes in the IMDb website: #39 Once I get some time over the next few days I will resolve the issues. Thanks for reporting!

Thank you so much!

Hi, I'm still having this issue. Is there any solution? Thank you


Error handling

Reference Issues/PRs
#138
What does this implement/fix? Explain your changes.

Test system modified
load_data and load_config functions error handling updated

Any other comments?

Codecov Report

Merging #151 (299a31c) into dev (2fadfea) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##               dev      #151   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            4         4
  Lines          506       512    +6
  Branches        84        86    +2
=========================================
+ Hits           506       512    +6

Impacted Files
Coverage Δ

samila/functions.py
100.00% <100.00%> (ø)

samila/params.py
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Question :is it possible to have larger loaders?

Hi,
Is it possible to have larger loaders?
Thanks

Hi mastermind1982,
You will have to change / overwrite the css that comes bundled with angular-spinkit.
I came across the same "problem" / question and after playing around with a fork / pull-request decided to try to publish a custom breed of a angular-spinkit - plugin (ui-spinkit)


Formalize ChatAgent response as a dataclass with inner messages

To support inner monologues exposed to the team for

streaming output
sharing context

Resolves: #3970
A ChatAgent response is a dataclass:
@dataclass(kw_only=True)
class Response:
    """A response from calling :meth:`ChatAgent.on_messages`."""

    chat_message: ChatMessage
    """A chat message produced by the agent as the response."""

    inner_messages: List[InnerMessage] | None = None
    """Inner messages produced by the agent."""

Where inner message for inner monologue, chat message is for agent-to-agent:
InnerMessage = ToolCallMessage | ToolCallResultMessages
"""Messages for intra-agent monologues."""

ChatMessage = TextMessage | MultiModalMessage | StopMessage | HandoffMessage | ResetMessage
"""Messages for agent-to-agent communication."""

This looks good.
Some questions (which might be a separate PR)

Do you see more types of inner_messages beyond toolcall/toolcallresult? How do we communicate to developers what the expected use for InnerMessage is (e.g., restricted to tool call type etc) or
Does this Response pattern potentially get us closer being able to add Teams inside teams?

e.g.
team_a = Team([agent_a, agent_b])
team_b = Team([team_a, agent_c])
Again, these might all be future PRs.

Do you see more types of inner_messages beyond toolcall/toolcallresult?

Yes, I think this should include other message types. We can expand this when need comes up, eg, in magentic one, ledger orchestrator has inner messages that are text.

How do we communicate to developers what the expected use for InnerMessage is (e.g., restricted to tool call type etc) or

For general users this will be implementation details. For developers of agents, it is an optional field that will be streamed to an output channel and stored in the output task result.

Does this Response pattern potentially get us closer being able to add Teams inside teams?

Yes. Teams should implement on_messages and returns a Response so it can be nested like an agent.


MINOR: remove the group id from a restore consumer

@guozhangwang
A restore consumer does not belong to a consumer group.

Nice catch! LGTM.


Found no NVIDIA driver on your system

Hello,
After building the image, and running it
sudo docker build --tag testimage .
sudo docker run -t -i --privileged testimage bash
cd rcic/
python main.py --save testrun

We get
	Traceback (most recent call last):
	  File "main.py", line 504, in <module>
		main(args)
	  File "main.py", line 484, in main
		model = ModelAndLoss(args).cuda()
	  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 297, in cuda
		return self._apply(lambda t: t.cuda(device))
	  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 194, in _apply
		module._apply(fn)
	  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 194, in _apply
		module._apply(fn)
	  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 194, in _apply
		module._apply(fn)
	  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 216, in _apply
		param_applied = fn(param)
	  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 297, in <lambda>
		return self._apply(lambda t: t.cuda(device))
	  File "/opt/conda/lib/python3.6/site-packages/torch/cuda/__init__.py", line 178, in _lazy_init
		_check_driver()
	  File "/opt/conda/lib/python3.6/site-packages/torch/cuda/__init__.py", line 99, in _check_driver
		http://www.nvidia.com/Download/index.aspx""")
	AssertionError:
	Found no NVIDIA driver on your system. Please check that you
	have an NVIDIA GPU and installed a driver from
	http://www.nvidia.com/Download/index.aspx

Note that outside docker the GPU is working as intended
$ nvidia-smi
Sat Sep 26 17:26:50 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.64.00    Driver Version: 440.64.00    CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1080    On   | 00000000:01:00.0 Off |                  N/A |
| 33%   41C    P8    11W / 180W |      1MiB /  8119MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Am I not running your code as intended?
What were the steps you took to run the code in docker in your machine?

Add --gpus=all to your docker run command or change docker run to nvidia-docker run. By default docker doesn't pass gpus to the container. If you do this, you should be able to run nvidia-smi inside the container and get the same output as run outside the container.

I confirm installing nvidia-container-toolkit as per https://stackoverflow.com/a/58432877/1734357 and then adding --gpus=all to docker run resolves it


Slow page build time in development with Next.js 5.0 (compared to Next.js 4.2.3)

[X] I have searched the issues of this repository and believe that this is not a duplicate.

Expected Behavior
Using previous releases of Next.js build time for pages in development mode as been more or less instant (taking at most a second or two to build a page for the first time, with changes being hot reloaded instantly).
Current Behavior
When using Next.js 5.0 build times for pages in development mode are much slower, often taking several (e.g. 5-10) seconds both for initial page build and for a change to be hot reloaded.
I am seeing this same issue across multiple codebases, and someone has raised it as a performance issue against nextjs-starter project, which recently upgraded to 5.0: https://github.com/iaincollins/nextjs-starter/issues/67
Steps to Reproduce

Clone example project 'https://github.com/iaincollins/nextjs-starter' and start with npm run dev.
Observe performance building pages with Next.js 5.
Stop it and run npm i<EMAIL_ADDRESS>--save and start again with npm run dev.
Observe much better performance building pages.

Context
I don't see this issue on very small sites (4-5 pages, no components) but I see it across all production Next.js sites I run, so it's not just project specific for me.
I appreciate it might still be something specific I am doing in those projects, or perhaps just because they all have more components involved.
I'm not sure how best to investigate what's causing this (and was a bit uncomfortable raising an issue without a better understanding of how to diagnose it) but as the performance is fine with Next.js 4, it seemed worth raising as an issue as I assume I'm not the only one impacted and if there are things I'm doing that can be avoided to improve performance then it's good to know what they are for others too.
Your Environment

Tech
Version

next
5.0

node
7.10.1

OS
macOS 10.13.2

browser
Chrome, Safari

I am also running into this issue on a site with a custom server, 20+ pages, in local development. Some of my build times have taken over 8000ms, and sometimes they hang and nothing happens at all.
Switching back to NextJS 4.2.3 resolved the issue for me, thanks for posting this @iaincollins.

@iaincollins https://github.com/iaincollins/nextjs-starter/blob/master/next.config.js#L4-L19
Note that webpack is ran twice, once for the server, and once for the client. You can distinguish them using isServer. This means that all styles are compiled twice in this case.
Also, you're importing things like bootstrap into the sass entrypoint, sass-loader/webpack doesn't cache @imports in sass files, causing every change you make to compile bootstrap twice.
I'd like to further investigate this, but this is a starting point 🙏

Thanks Tim, that's super helpful as ever! :)
I'll start with that and follow up with what @jimmmylee was saying by comparing performance with a non-custom server to see if that makes a difference too.
I'd tried using the new approach for sass but didn't like the flash on initial load, but I see from another issue there is a fix for that in a call release already.
I'm sure you are busy and I don't want to make an issue about any specific project of mine, but rather just try and figure out general things that might be better handled differently now, so I will do some digging with the info I have and update with what I find.

Happy to help! Maybe we can make an upgrade guide from next 4 to next 5. There's probably more people running into this.
One idea I had is to move bootstrap into a different entrypoint, and importing it in _document.js using my SSR css imports work (in an open PR on next-plugins). Not sure if it'll work, but it'd solve the issue where bootstrap is compiled every single time.

I don't think the sass imports are root issue. I'm still getting long compile times after removing them :(

I'm seeing super slow build times in dev mode, it there a way we could cache stuff like styles and components (like my layout that doesn't change much)? That would help a lot.

I'm also running into this issue. It's become really slow with each hot reload. I have sass being imported into styled jsx. Is that the root cause?

@develomark same issue, but have sass imported as module into style-loader.
Can not rollback to branch 4, because separate server and client config makes possible import sass files as modules without styled components. Like it.
Tried enable ignore-loader on sass files instead another loaders (it does on server-side config, but i manually try enable it in front config), hot reloading time still bigger than 4.2.3 and same than before.
Wait for main contributors ideas.

try out the latest next release: canary 6
canary < 6: hot reload build time was between 4-12 seconds
canary 6: hot reload build time: 2-3 seconds.

@khuezy hot reload simply broken and not working in canary6 :)

@plag hmm, hot reload is working for me in 6.
Try canary.7, dev compile times are more reasonable. ~2-4 seconds on complete build and < 2 seconds on file change.

Still taking me 15+ seconds to compile on my project on canary.7. Not even close to 4.2.3 yet.

@plag @bryan it's impossible to debug it's slow or it's broken. Note that it can be numerous factors. So please provide a minimal reproduction, besides the one @iaincollins already provided 😄 Then I can have a look into it 👍  Thanks 🙏

@timneutkens It was more for khuezy's response, in case people start reading the end thinking it's fixed on the canary builds. But I'll keep my pains to a minimum.

@bryan
I'm not sure why we're seeing different dev build times.
Did you try to remove the node_modules and re installing?

@timneutkens prepared simple projects for test
https://github.com/plag/nextjs-slowtest4
https://github.com/plag/nextjs-slowtest5
In readme you can watch live examples and results. Here is average diff is 2x slower on v5.
In my real project (with same structure as demo apps but much more components) on v5 hot reload takes 1.7s vs ~500ms on v4.2.3.

Since next.js 5 I also started getting mobx issues like:
[mobx] Warning: there are multiple mobx instances active. This might lead to unexpected results. See https://github.com/mobxjs/mobx/issues/1082 for details.
I think I'll rollback for now..

Yeah rolling back, it's literally getting my Macbook pro to a standstill after a few hours of dev.

This is a huge problem for me - with next 5 my builds can take upwards of 15s and my laptop grinds to a halt. On next 4 it's more like a few hundred milliseconds.

I don't want to repeat myself. But:

it's impossible to debug it's slow or it's broken. Note that it can be numerous factors. So please provide a minimal reproduction, besides the one @iaincollins already provided 😄 Then I can have a look into it 👍 Thanks 🙏

I'll try and investigate this further - however: I have noticed my development environment is using a production React build. This doesn't occur with a fresh build of next.js (no custom Express server setup).
I'm wondering if anyone with this issue is noticing the same thing?

anyone here are using on next.config.js
const withSass = require('@zeit/next-sass');
module.exports = withSass({})

maybe can be this

Starting to feel like a robot 😅

I don't want to repeat myself. But:

it's impossible to debug it's slow or it's broken. Note that it can be numerous factors. So please provide a minimal reproduction, besides the one @iaincollins already provided 😄 Then I can have a look into it 👍 Thanks 🙏

Sorry I haven't had time to provide more info on this yet BTW, I will try and do more digging soon.

@iaincollins you and @plag actually are the only ones that did provide an example 😄

@gabrielmancini This seems to be the problem on my end.  I went from 730 MS to  8000-16474ms.
The only changes I made were setting up Semantic UI theme and importing external CSS to change the design of a simple login page.
There's nothing else beyond that, I have 2 separate branches to confirm this is the issue. I'm going to go down the list to try to replicate exactly at what point this issue came up, and come back here to post an update.
It's a consistent 10-20X increase in compile time even when only changing a single letter.
Here's my next.config.js file.
``
const withCss = require('@zeit/next-css')
module.exports = withCss({
webpack (config) {
config.module.rules.push({
test: /.(png|svg|eot|otf|ttf|woff|woff2)$/,
use: {
loader: 'url-loader',
options: {
limit: 100000,
publicPath: './',
outputPath: 'static/',
name: '[name].[ext]'
}
}
})
return config

}
})
``

Out of desperation I tried @hipabloangel's solution from a different issue.
I was able to fix my slowdown issues in 5.1 by adding this to next.js.conf:
module.exports = withSass({
  webpack(config, { dev }) {
    if (dev) {
      config.devtool = 'cheap-module-eval-source-map';
    }
    return config;
  }
});

I have no idea why this works, but I guess for some reason it's expensive to generate external source maps? From some experimenting, I was also able to speed things up by removing react-material-ui SSR CSS generation (but this was before I discovered the above source map fix).

@JonAbrams YES YES YES!! Thank you! Looks like source maps was reason of slowdown. 4.2.3 uses cheap-module-inline-source-map, version >=5 use source-map. Change it back makes SSR fast as before.

@timneutkens is working on changing over to the cheap-module-source-map in here https://github.com/zeit/next.js/pull/4156/files#diff-0f2f34c098f5954f99483c9bd61e439dR124 🎉

To confirm, I am no longer seeing this in<EMAIL_ADDRESS>Thank you @timneutkens and everyone else who worked on this. 🎉

I'm still seeing 80-200 second HMR on the client with the latest version and everything back up to https://github.com/zeit/next.js/commit/e093441bad588e98765a05df90f76f75283eda07
Before that (4.4.0-canary.3) it's really fast. Resolving react-hot-loader to 3.1.1 with Next 5.0.0-universal-alpha.14 is also fast, and Next 6.0.3 with
"resolutions": {
   "react-hot-loader": "4.2.0"
 },

Seems to work well again. Is anything keeping an upgrade to 4.2.0 back? @timneutkens

I was making the silly mistake of not ignoring the next build files and nodemon was restarting over and over and resulted in long loading time.  Now im happy to report hot-reloading only takes a few seconds
  "ignore": ["next_app"]
}```


fixed minor bug in get_ende_bleu.sh

This PR is to fix a minor bug in utils/get_ende_bleu.sh.

Thanks!


Update microsoft-defender-endpoint.md

The training module has been removed from https://learn.microsoft.com/en-us/users/collinschedler-0717/collections/3nrwbj15ggyjq5
Fixes  https://github.com/MicrosoftDocs/microsoft-365-docs/issues/11951

@JohanFreelancer9 Please copy review

@mjcaparas Please merge this PR. Thanks


NO-TICKET: clean up Key::Balance

This PR is a followup to to #960.  It addresses a few missing bits, typos, etc..

bors r+