Compiling flutter/engine on Windows

Do we support compiling the engine on Windows?
Looks like the build script has some issues with paths on Windows:
C:\src\flt\engine\src>python flutter\tools\gn --unoptimized
gn gen --check in out\host_debug_unopt
Traceback (most recent call last):
  File "C:/src/flt/engine/src/flutter/build/git_revision.py", line 42, in <module>
    sys.exit(main())
  File "C:/src/flt/engine/src/flutter/build/git_revision.py", line 34, in main
    'HEAD',
  File "C:\src\depot_tools\win_tools-2_7_6_bin\python\bin\lib\subprocess.py", line 566, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "C:\src\depot_tools\win_tools-2_7_6_bin\python\bin\lib\subprocess.py", line 709, in __init__
    errread, errwrite)
  File "C:\src\depot_tools\win_tools-2_7_6_bin\python\bin\lib\subprocess.py", line 957, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
ERROR at //flutter/common/version/version.gni:15:7: Script returned non-zero exit code.
      exec_script("$flutter_root/build/git_revision.py",
      ^----------
Current dir: C:/src/flt/engine/src/out/host_debug_unopt/
Command: C:/src/depot_tools/win_tools-2_7_6_bin/python/bin/python.exe -- C:/src/flt/engine/src/flutter/build/git_revision.py --repository C:/src/flt/engine/src/flutter
Returned 1.
See //flutter/common/version/BUILD.gn:5:1: whence it was imported.
import("version.gni")
^-------------------
See //flutter/shell/common/BUILD.gn:108:5: which caused the file to be included.
    "$flutter_root/common/version",
    ^-----------------------------

C:\src\flt\engine\src>C:/src/depot_tools/win_tools-2_7_6_bin/python/bin/python.exe -- C:/src/flt/engine/src/flutter/build/git_revision.py --repository C:/src/flt/engine/src/flutter
Traceback (most recent call last):
  File "C:/src/flt/engine/src/flutter/build/git_revision.py", line 42, in <module>
    sys.exit(main())
  File "C:/src/flt/engine/src/flutter/build/git_revision.py", line 34, in main
    'HEAD',
  File "C:\src\depot_tools\win_tools-2_7_6_bin\python\bin\lib\subprocess.py", line 566, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "C:\src\depot_tools\win_tools-2_7_6_bin\python\bin\lib\subprocess.py", line 709, in __init__
    errread, errwrite)
  File "C:\src\depot_tools\win_tools-2_7_6_bin\python\bin\lib\subprocess.py", line 957, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified

C:\src\flt\engine\src>dir C:/src/flt/engine/src/flutter/build/
Parameter format not correct - "src".

But this would work:
C:\src\flt\engine\src>dir C:\src\flt\engine\src\flutter\build\

Apparently we do not have build integration for Windows yet: https://github.com/flutter/flutter/issues/138.

Six month passed, No one want to fix this issue? I have the same issue here.

Just encountered this issue.

I've got the issue as well.

Got same issue after updating to 1.17.0 beta

Got same issue after updating to 1.17.0 stable

got same issue, can anyone help?

me too got the same issue after upgrading to flutter v1.17.1 stable channel.

me too, someone help, i think this is causing bugs in my app

I am having same issue after upgrading to v1.17.1 stable channel

Having the Same Issue After Upgrading Flutter to 1.17.1.
Anyone if they know the solution kindly help

Same here, having same issue after upgrading to v1.17.1 stable channel

An fix to this yet, my project relays on flutter 1.17.1, if i downgrade it, lots of things will change and i need google map on my app

@dcharkes The doc clearly says so: https://github.com/flutter/flutter/wiki/Compiling-the-engine#compiling-for-the-web-on-windows
But still, after more than a year, not a iota has changed here.

Now that Windows has a complete Linux subsytem and installing a Linux is literally just a few mouse clicks away, probably there is no longer a need to struggle with this. I moved on to a Linux. Here is the list of what I experienced:

Ubuntu 18.04, don't go any further. Although the doc mentions that some scripts are version-dependent, it fails to specify which, so you only know if you do the full process and learn at the ver end that you need to start all over again.

When you install depot_tools, make sure you run gclient without arguments, just like the description mentions for Windows. It forgets to mention for Linux but it is required.

So right now, the solution to building on Windows is to use Linux instead? 😅 Honestly that sounds like a great idea, but there are situations where native Windows is the only choice.

@thavlik Somebody told me he had success: https://github.com/flutter/flutter/issues/61440 I tried again and failed.

So right now, the solution to building on Windows is to use Linux instead?

There are CI bots that build the engine on Windows for every PR and every commit. A number of Flutter engine developers build regularly on Windows.
The issue here is not "it's impossible to build the engine on Windows", it's "there is some (currently unknown) configuration that causes issues for some people when building on Windows".

@stuartmorgan And in our quest to determine those configuration issues, can we start from the actual error message we all seem to get? :-) I'll write up my Windows process, just like the Linux under Windows above, starting for the umpteenth time.

This time for the original Windows setup. It's mostly the same as the one described by the Flutter team, obviously, but we might find the difference in the end.
Here is the list of what I experienced as I moved on:
Visual Studio

Download and install. It’s enough to install C++ development. Uncheck the Windows SDK.
Download the SDK separately, and uncheck everything but Debugging Tools for Windows.

If you use VS for other development, it’s not a problem if the SDK or other parts are also installed. They’re not required for compiling the engine, that’s all.
Depot_tools

(Download)[https://storage.googleapis.com/chrome-infra/depot_tools.zip] and extract
add the extracted folder to front of PATH
gclient -v (this is missing from the docs, important! -v is just for verbose so that you can see if it gets stuck). The running ends with the help screen showing commands and options.
where python and verify that python.bat precedes python.exe

Git

git bash
And in the new Git console:
git config --global user.name "YOUR GITHUB NAME"
git config --global user.email "YOUR GITHUB EMAIL"
git config --global core.autocrlf false
git config --global core.filemode false
git config --global color.ui true
exit
These will be stored in c:\Users\name\.gitconfig.

GitHub
Fork https://github.com/flutter/engine. Set up your key:

git bash
And in the new Git console:
ssh-keygen -t rsa -b 4096 -C "YOUR GITHUB EMAIL" (just reply Enter to the first question to use the default filename)
eval $(ssh-agent -s)
ssh-add ~/.ssh/id_rsa (yes, using Linux path)
Add the contents of c:\Users\name\.ssh\id_rsa.pub  to your GitHub account > Settings > SSH and GPG keys.
ssh -T<EMAIL_ADDRESS>accept the fingerprint if it matches the one in the docs: https://docs.github.com/en/github/authenticating-to-github/testing-your-ssh-connection
exit

Download the source

mkdir engine
cd engine
You can create the .gclient file in the docs, from Windows using your favorite text editor
gclient sync -j1 -v (it takes quite some time but at least the verbose switches will show you what it does; there will be a longish step after the last downloaded package (wuffs), so wait for the end patiently)
If git complains about access rights, either you start to troubleshoot the issue or you can simply revert to https access (instead of key-based) in .gclient:

"url": "https://github.com/<your_name_here>/engine.git",

For a one off download, it’s tempting to go the easy way but if you want to build regularly, it pays off to troubleshoot the issue.

@thavlik Thomas, this is my current write-up and the error I get. Does it resemble your problem?

If I assemble the command thatgit_revision.py tries to call:
git -C <repo-path> rev-parse HEAD

it runs all right and returns the long code.

@stuartmorgan This looks like a major scripting error on Windows. If I add Shell=True to the subprocess.check_output() in git_revision.py, that particular error goes away. And it stops at the next one. It looks like calling external executables is done in a way that's not cross-patform safe in any way.

@deakjahn I'm not sure why you pinged me in that comment.

This looks like a major scripting error on Windows. [...] It looks like calling external executables is done in a way that's not cross-patform safe in any way.

If you've identified the problem, then by all means please provide a description or a PR.

If I add Shell=True to the subprocess.check_output() in git_revision.py, that particular error goes away.

Running commands through the shell is bad practice unless there's a very, very specific need to do it, and makes cross-platform, or even just cross-machine, issues dramatically more likely. There is nothing in that command that should require being run through the shell. Whatever the underlying issue you are encountering is, adding shell=True is a workaround, not a viable solution.

It was just an idea. You mentioned a few days ago that there were situations where it worked, all the other contributors seemed to be of the idea that it doesn't. :-)
This is one single, small call from a very large, very interspersed, very complex toolchain. I don't know if you're inside Google and whether you routinely use this chain to build or not. I'm not, I would just need to use it to compile the engine to test other PRs I might possibly want to contribute. I think I can safely assume that Flutter people who are familiar with the toolchain has a much better chance to find out what's wrong with it, although the fact that this issue is more than a year old isn't very promising. :-)
I don't have the solution, unfortunately, that single test with Shell was just an attempt. As I can see, all problems stem from Python calling external executables. They don't return or don't return their stdout properly. I had some hope that mentioning the general direction might trigger some ideas from some of the contributors that could lead towards a solution, that's all.

that there were situations where it worked, all the other contributors seemed to be of the idea that it doesn't.

You can't extrapolate from "people who have found and commented in a bug about a specific build failure" to "all other contributors". The former is, by its nature, not at all a representative sample.
It sounds like you believe that it works in rare cases and for everyone else it fails; the opposite is true. Not only do plenty of people successfully build Windows regularly, but we have repeatedly—using several completely different CI systems based on different images—set up working Windows automated builds without any special steps, and without hitting the error described in this issue.

I think I can safely assume that Flutter people who are familiar with the toolchain have a much better chance to find out what's wrong with it

If there were people on the Flutter team hitting this issue, that would likely be true. But it is very hard to debug a problem without being able to reproduce it.

It sounds like you believe that it works in rare cases and for everyone else it fails;

I most certainly was before you chimed in because from the OP to the many comments saying me, too, me, too, there was no other opinion present. You were the first one to tell otherwise. :-)

to the many comments saying me, too, me, too

FWIW, the stream of me-too comments in May do not actually appear to be about this issue; many refer to updating to a specific version of Flutter, which makes no sense in the context of an issue about compiling the engine from source.

Then I was misled.
Anyway, I try to find out the cause, I can see that Python actually calls the external commands and those commands execute all right, only that the calling script doesn't get the exit status and the output correctly. And even if shell isn't good to use normally, it does help in this particular case, so I need to find out what makes the non-shell case fail here.

Actual compilation errors:
output: Unhandled exception:
Could not generate version
#0      getVersion.<anonymous closure> (file:///d:/Android/engine/src/third_party/dart/utils/compiler/create_snapshot_entry.dart:19:7)
#1      _RootZone.runUnary (dart:async/zone.dart:1450:54)
#2      _FutureListener.handleValue (dart:async/future_impl.dart:143:18)
#3      Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:696:45)
#4      Future._propagateToListeners (dart:async/future_impl.dart:725:32)
#5      Future._completeWithValue (dart:async/future_impl.dart:529:5)
#6      Future.wait.<anonymous closure> (dart:async/future.dart:410:22)
#7      _RootZone.runUnary (dart:async/zone.dart:1450:54)
#8      _FutureListener.handleValue (dart:async/future_impl.dart:143:18)
#9      Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:696:45)
#10     Future._propagateToListeners (dart:async/future_impl.dart:725:32)
#11     Future._completeWithValue (dart:async/future_impl.dart:529:5)
#12     Future._asyncCompleteWithValue.<anonymous closure> (dart:async/future_impl.dart:567:7)
#13     _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
#14     _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
#15     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:118:13)
#16     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:169:5)

@thavlik Thomas, if you're still interested in it. I finally managed to compile on Windows and modified my description to include the necessary steps.

Currently documented at https://github.com/flutter/flutter/wiki/Compiling-the-engine#compiling-for-windows. @cbracken I think you or @stuartmorgan have done this recently. When you have time, could you please update the wiki?

Currently documented at https://github.com/flutter/flutter/wiki/Compiling-the-engine#compiling-for-windows.

Possible related https://github.com/flutter/flutter/issues/61440

@kf6gpe See my comments above; the issue isn't that the wiki is out of date, it's that there is some people seem to have path issues that most people (including our CI) don't have, and so far we don't know why.
If at some point we figure out what the configuration issue for people hitting this is, then we can document it (or adjust scripts). But currently we don't know what it is, so I have no idea what I would update the wiki with.

This maybe caused by we didn't  install the git for windows. Install a standalone git in
https://www.git-scm.com/download/win and add this git into PATH environment
C:\Users\name>where git
C:\Program Files\Git\cmd\git.exe
C:\Users\name\depot_tools\git.bat

Closing this issue. Rationale:

There haven't been any additional comments/updates in ~8 months.
The comments appear to have described a collection of potentially unrelated issues, including documentation issues, rather than any one specific issue, and several of them are likely to have been fixed or are out of date at this point, given how many changes to our build have landed in the last year.
The engine has been building locally for engine developers and on our bots for some time.

For anyone still experiencing issues compiling the engine on Windows, I'd encourage you to file a new issue with:

Repro steps.
The specific error message you're seeing, or if you see an issue with the exact same error message as you're getting, add a thumbs-up, but please don't add other unrelated errors to the same issue, instead file a new one.


docker run fails: Error response from daemon: devmapper: Unknown device "<dev_name>

Issue Description:
"docker run" command  fails.
This is happening after we changed docker configuration - configured docker host with devicemapper with actual disk as the backend (without loop). Earlier docker host was running with the devicemapper storage driver operating in loop-lvm mode.
https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/
root@master-pwd-vol:/home/ubuntu# docker run -it ppc64le/ubuntu
Digest: sha256:5349f00594c719455f2c8e6f011b32758dcd326d8e225c737a55c15cf3d6948c
Status: Downloaded newer image for ppc64le/ubuntu:latest
docker: Error response from daemon: devmapper: Unknown device 200088a0cfef0520bc27e5094021d5442593345df6711028807fe896d8d11339.
See 'docker run --help'.
More details:
root@master-pwd-vol:~ uname -a
Linux master-pwd-vol 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 17:05:51 UTC 201
7 ppc64le ppc64le ppc64le GNU/Linux
root@master-pwd-vol:~
root@master-pwd-vol:~ docker --version
Docker version 1.12.0, build 5507242
root@master-pwd-vol:~  docker info
Containers: 8
Running: 0
Paused: 0
Stopped: 8
Images: 21
Server Version: 1.12.0
Storage Driver: devicemapper
Pool Name: docker-thinpool
Pool Blocksize: 524.3 kB
Base Device Size: 10.74 GB
Backing Filesystem: xfs
Data file:
Metadata file:
Data Space Used: 1.219 GB
Data Space Total: 20.4 GB
Data Space Available: 19.18 GB
Metadata Space Used: 544.8 kB
Metadata Space Total: 213.9 MB
Metadata Space Available: 213.4 MB
Thin Pool Minimum Free Space: 2.039 GB
Udev Sync Supported: true
Deferred Removal Enabled: true
Deferred Deletion Enabled: true
Deferred Deleted Device Count: 0
Library Version: 1.02.110 (2015-10-30)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: null overlay bridge host
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor seccomp
Kernel Version: 4.4.0-64-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: ppc64le
CPUs: 2
Total Memory: 3.977 GiB
Name: master-pwd-vol
ID: IBGL:BQGE:7BS4:HVU3:EG3A:4YVC:V72C:3MDQ:H7VE:T2AG:B6CL:YMHH
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 15
Goroutines: 23
System Time: 2017-03-02T13:05:40.911404Z
EventsListeners: 0
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
<IP_ADDRESS>/8
root@master-pwd-vol:~
root@master-pwd-vol:~ lsblk
NAME                    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0                      11:0    1  446K  0 rom
vda                     253:0    0   20G  0 disk
├─docker-thinpool_tmeta 252:0    0  204M  0 lvm
│ └─docker-thinpool     252:2    0   19G  0 lvm
└─docker-thinpool_tdata 252:1    0   19G  0 lvm
└─docker-thinpool     252:2    0   19G  0 lvm
vdb                     253:16   0   20G  0 disk
├─vdb1                  253:17   0   20G  0 part /
└─vdb2                  253:18   0    8M  0 part
root@master-pwd-vol:~

Did this already have images?
I'd recommend clearing out /var/lib/docker then try again.

Yes, it has images. We are trying the steps once again after removing the images.

@cpuguy83 , we removed the docker images , and re-tried the steps. It is successful now.
Closing this issue.
Thanks for your suggestion!


nixos/postgresql: set up sandboxing

Reduces the general exposure of the postgresql.service through systemd hardening options.
I fully expect that I've missed a number of use cases (e.g. through extensions), feedback is welcome!
Here's what systemd-analye security postgresql.service thinks of the remaining options:
machine:   NAME                                                        DESCRIPTION                                                                    EXPOSURE
✗ RootDirectory=/RootImage=                                   Service runs within the host's root directory                                       0.1
✗ RestrictAddressFamilies=~AF_UNIX                            Service may allocate local sockets                                                  0.1
✗ RestrictAddressFamilies=~AF_(INET|INET6)                    Service may allocate Internet sockets                                               0.3
✗ PrivateNetwork=                                             Service has access to the host's network                                            0.5
✗ PrivateUsers=                                               Service has access to other users                                                   0.2
✗ PrivateTmp=                                                 Service has access to other software's temporary files                              0.2
✗ DeviceAllow=                                                Service has a device ACL with some special devices: char-rtc:r                      0.1
✗ IPAddressDeny=                                              Service does not define an IP address allow list                                    0.2

→ Overall exposure level for postgresql.service: 1.4 OK :-)

Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable: (nixosTests.postgresql, postgresql.tests)

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

I've been using my option for a long time. I haven't found any problems.

Can you add ReadWritePaths?

I've been using my option for a long time. I haven't found any problems.

Same here with confinement, running since years. The confinement variant would be also more restrictive than the implementation here because all you get is the Nix closure of PostgreSQL. If the chroot-only mode is used you don't even need the Protect* options because those mounts aren't there in the first place.
The downside is that systemd-analyze doesn't detect this properly.
Nevertheless, using the options here is better than nothing, so :+1: from me.

I think the only way for such a change to not result in angry users is if the existing postgresql module gets some additional features. Essentially, every postgres extension module (like postgis) should specify a set of systemd features it is compatible with (or alternatively incompatible with) and the computed intersection of those options is how it will run then. So, right now your approach is to hard code some options, but that design won't scale.
The design should be such that someone can use a postgresql extension involving GPUs (incompatible with your current set of options) by enabling it via some compositional method and the required configuration should automatically work. (In fact, one could argue Postgresql is designed in the wrong way to not have per module hardening via a capabilities system. )
If this is merged someone will get error messages in the best case, which are likely hard to comprehend and certainly they won't get an error at configuration time.
It's somewhat useful that you presumably found a set of which 90% probably won't cause any issues die anyone, but making an actual improvement (for every user, ignoring the slower evaluation) would require my design or another design (which would require evidence of it also offering the compositional guarantee I outlined).
I think it would be better if a prototype were to be developed based on my compositional method and then the discussion could continue, since perhaps I missed something. (The actual implementation (which could easily be thousands of lines of code) and method of maintenance are independent of providing the basic guarantee.)

@mweinelt I'm actually wondering if we should do some passthru.sandboxAdjustments to each extension (although it's a little weird how a package exposes a partial module here). These could be mkMerged together. Bottom line is that each existing extension gets checked.
cc extension maintainers @1000101 @bbigras @chayleaf @danbst @DeeUnderscore @DerTim1 @diogotcorreia @esclear @euank @gaelreyrol @ggPeti @idontgetoutmuch @ivan @jopejoe1 @k0001 @MarcWeber @mmusnjak @netcrns @nialov @qoelet @shivaraj-bh @sikmir @spinus @steve-chavez @takeda @thenonameguy @typetetris @willcohen @WilliButz @wolfgangwalther @zimbatm @sikmir @imincik would be helpful if you could check which of your extensions will have compatibility problems.
I don't think we can do much for stuff that isn't packaged in nixpkgs though except for good release notes.

Yes, conditional hardening is the ideal outcome, thanks for pinging everyone.

I'd expect wal2json to be fine with this if the tests pass, and I can't think of any reason sandboxing would break it.
@ofborg test postgresqlPackages.wal2json.passthru.tests

I currently don't have NixOS running to test, but regarding pgmq which I just added, that extension only contains sql files, nothing extra is built, so if any extension work I would expect this one would too.

I don't anticipate any problems running the rum extension under this, and I did not show any in a quick test.

OK so from what I've seen so far, there's no extension packaged in nixpkgs that requires additional changes to the hardening, at least no maintainer gave feedback.
I'm still wondering if we need some kind of passthru.sandboxAdjustments, however, I'm not sure how: on the same level are also settings that shouldn't be overridden by an extension (e.g. ExecStart).
Also, which priority do we give things in sandboxAdjustments and the module? Like, otherwise we'd need extensive mkForce usage in the extensions (though that's probably the most reasonable thing).
However this makes it pretty hard to catch incompatibilities between extensions with different adjustments. If everything is written down in conditionals in the module, I'd expect it to be way easier to catch issues.
And if people package things out of nixpkgs, I think it's perfectly fine to expect them to override the relevant settings on their own. Hardening the service even more is a breaking change and shouldn't be backported then, however.

So tl;dr I'm leaning towards leaving the change code wise as-is and writing docs (release-notes and probably even a note in the manual).
WDYT @mweinelt @wolfgangwalther @h7x4 ?

The readWritePaths breaks users who use TABLESPACE
Affected users can fix it by setting systemd.services.postgresql.serviceConfig.ReadWritePaths = [ "/path/to/tablespace" ]


Wallet connect for react native apps / kotlin dApps

Hello team Nova,
Is there any component that can be used in react native (or Kotlin) acting as a wallet connect component for Nova Wallet?
My student is building Polkadot mobile dApp using React native (Or Kotlin, didn't decide yet) and needs wallet connect component.
If there is any provided by you, could you share the link to it?
Many thanks!
With kind regards,
Dudo

Found wallet-connect component that might be suitable for this. Closing for now. Will reopen later if needed.


KAFKA-13588: consolidate changelogFor methods to simplify the generation of internal topic names

More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.
Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

One meta comment regarding consolidating the functions: we have getPrefix in multiple classes (and thanks @sayantanu-dey your PRs remove some already), but there are still two duplicated functions in InternalTopologyBuilder and ProcessorContextUtils as well, could we just keep one and have the other reference it too?

Hi, @guozhangwang  I have removed the duplicate getPrefix function in InternalTopologyBuilder in my latest commit. now there exits only one function getPrefix in ProcessorContextUtils


 Now we are testing to see we are inside a
transaction before committing it to prevent an exception being thrown.
Reference to the comment on #50 this has changed a bit. Now the test for a transaction has been moved earlier and is returning. This was continuing on to release the save point 0 that never gets created.
Ref: #50

https://github.com/doctrine/migrations/pull/1131/files

After reviewing what other frameworks did it seems that the patch is valid.

@AdeAttwood Can you please merge the current master into your PR branch?

@AdeAttwood Thank! Most of the tests are now passing.

@wolfy-j @SerafimArts Just checking in on this. I see #66 has been merged. Is there anything you want to get this merged. I'm assuming we can just close this.

Yes, we have been debugging other issues and also solved this. Thank you for your PR!


Add typing support

It would be nice with typing support to get type warnings on code like the following:
# --- Instantiation
x: plist[int] = plist(['a'])  # Should give warning
# ---

# --- Append
x: plist[int] = plist()
x.append('a')  # Should give warning
# ---

# --- Extend
x: plist[int] = plist()
y: plist[str] = plist()
x.extend(y)  # Should give warning
# ---

The same goes for all the collections, of course.

This is a good idea (and thanks for the demo PR!), but I don't use typing in my own code, so this is a fairly low priority for me. Right now, I'm most focused on (1) documentation and (2) implementation these collections in a C extension. I'm not likely to get around to the C implementation for a few months at least, but if you are interested in working on the types here, I'd be happy to accept PRs and to help where I can.


[BUG] Upgrade tooltip on wrong option

Description
When trying to make an App private, the Upgrade tooltip is rendered on the wrong option (public)
Steps to reproduce

Go to https://cloud.illacloud.com/
Login with your account
Go to your apps
Click on the dropdown to make the app private/public.
The Upgrade label is rendered on the wrong option.

Version
cloud
ILLA Builder Version
3.2.1
Browser
Edge Browser Version 117.0.2045.31 (Official build) (arm64)
Operating System

[X] macOS
[ ] Windows
[ ] Linux

Additional Information
It should be rendered on the Private option because that is the paid feature. And please assign this to me, would love to work on it.

/assign @manancodes

The expectation is like this. It's not a bug.

Public is not a paid feature. So why put the label there?
To use the Private feature, we need an upgrade. Shouldn't the label be there?


Unable to use the Azure PowerShell commands

I'm trying to use the Get-AzureRmDataFactory and the similar commands but get the error "The term 'xyz' is not recognized as the name of a cmdlet, function, script file, or operable program."
I even tried Get-AzDataFactory as the AzureRM module is no more being used.
How do you log the Data factory Pipeline using PowerShell, is the cmdlet updated or is the work still in progress?? Could you please verify

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ab81ac50-5213-1282-d66c-e0c70794d2eb
Version Independent ID: 1068f896-b7a8-de23-ebe6-c03aad0398c9
Content: Overview of Azure PowerShell
Content Source: docs-conceptual/azurermps-6.13.0/overview.md
Service: azure-powershell
GitHub Login: @sptramer
Microsoft Alias: sttramer

@markcowl Can you please provide some information on the state of Az.DataFactory in the currently available release?

@ShashiKiranGowda Have you installed the latest Az module?  Please run Get-Module -ListAvailable so we cna see the module layout on your machine.

Is 1.26 the latest version ??
I have attached the version of my PowerShell and AzModule. Kindly suggest on the error attached.

@ShashiKiranGowda Please include the output of the following commands (as copy-paste, not a screenshot):
Get-InstalledModule -Name Az -AllVersions
Get-InstalledModule -Name AzureRM -AllVersions

This issue has gone for 4 months without a response when requesting more info; closing.


Could we get a total skill level added to the display like the HiScore display?

Pretty straightforward request. I'm hoping to have a total level segment added to the group ironman stats, similar to how the HiScore plugin displays this.
There was a commit added to the HiScore plugin to address the issue when a level is below 10 and can't be seen via the hiscores: https://github.com/runelite/runelite/issues/4691#issuecomment-445079703

Fixed in https://github.com/runelite/runelite/commit/233fbcf2d1c28cf458953df45e31ecf271d794a7 by showing stats the jagex hiscore api returns as unranked but has level/xp for.

Yeah I should be able to add this pretty easily, I'll try and submit the update to RL by the end of the week and I'll link the PR here so you can track it.

It was pretty quick: https://github.com/runelite/plugin-hub/pull/5492
Will take them anywhere from a day to two weeks to approve.

Merged into runelite via https://github.com/runelite/plugin-hub/pull/5492


[ADD] show youtube-dl preview next to youtubeDLG

The under list that shows about download, Can you add context menu to it?
Add features such as Open File, Open Folder, Show YOUTUBE-DL..

Moved to https://github.com/MrS0m30n3/youtube-dl-gui/issues/192


Update dotnet-developer-projects.md

Added the TomP2P.NET library, thank you!

Thanks @Biocodr. Can you rebase with master?

Thanks @Biocodr. Appreciated.


Doc: Add links to source code for showcases

When navigating the Example page it would be nice to add links to source code for each charts.
Today we need to open github / showcase / the good file. Clearly doable but it could be much nicer.
Very nice job by the way

All the examples pages have source code links now, good catch


feat: password visibility toggle button added

Related Issue
Closes: #716
Changes made 👷🏻♂

toggle button added
UserRegister.jsx and UserLogin.css updated

Screenshots 📸

Hey @IAmTamal !! Can you review my PR and merge it?

Hi there, please watch this, with VOLUME TURNED ON. The changes will be clear.

Hi! I have seen your video and noted down the changes you want.. I will start working on it and keep you updated.. thank you for the review..
This issue (#716) has a level 2 tag on it.. can you change it to level 3 please? Thanks again!

Hi there, please watch this, with VOLUME TURNED ON. The changes will be clear.

Hii @IAmTamal! I have made the changes that you asked for. Kindly review the PR.


2021-05-31 v. 3.6.6: added in-app update feature

1.Added:

play core library dependency in app/build.gradle
flexible in-app update feature in MainActivity

@vipozdn, thanks a lot. I'm added you to Contributors list in README.md.

@vipozdn, thanks a lot. I'm added you to Contributors list in README.md.

Thanks!


QI-Core 4.1.1 MedicationNotRequested Reason Code Mapping

After discussion with @brynrhodes and @mgramigna, there is agreement that there is an issue in how reasonCode is getting mapped to ELM for the MedicationNotRequested datatype in QI-Core 4.1.1. The QI-Core Model Info File 4.1.1 restricts reasonCode to cardinality of 1..1 for MedicationNotRequested based on the QICoreMedicationNotRequested  profile. However, the MedicationRequest resource's reasonCode normally has cardinality of 0..*.
Since ELM is supposed to contain FHIR equivalent as if FHIR is what was being used to author in, reasonCode output should be similar between MedicationRequest v MedicationNotRequested when similar structures are used. They aren't. Refer to the CQL and ELM in the attached zip.
@mgramigna's findings
My take on the issue is that I think Stan’s instincts are correct. I believe that when restricting something that is 0..* to 1..1, the property still must be an array, but it will only have one element (this can be verified using the IG publisher and looking at the examples in the QI Core IG). It looks like the translator/ModelInfo thinks that “reasonCode” on a “MedicationNotRequested” is just a CodeableConcept rather than a list of CodeableConcepts. Here are some CQL snippets and data types from the cql language server that I created to confirm this:

reasonCode is a list of Concepts as expected. Now for MedicationNotRequested:

In the MedicationNotRequsted case, the ModelInfo thinks that reasonCode is just one Concept, whereas I think it should be a list of just one Concept? (not sure how the communication between the cardinality constraint and the ModelInfo works in this instance).
@brynrhodes feedback
Looking at the output ELM, yes, that is correct, the translator is considering the reasonCode a singular element, rather than a list of one. Can you submit this as an issue to the CQL translator repository and I will get it addressed.
Stan, you are correct, the ELM engines should be able to run profile-informed CQL without needing to consider the profiles (other than the profile markers in the resources at the data access layer), so this is something that the translator should be accounting for.
ABCTest.zip

Likely relevant parts of ModelInfo:
MedicationRequest reasonCode has a ListTypeSpecifier
MedicationNotRequested reasonCode doesn't

Fixed in 2.7.0


clEnqueueNDRangeKernel times out on large buffers

I'm seeing a very strange behaviour when running relatively large buffers (87968 bytes). I have a test FIR filter:
https://github.com/dernasherbrezon/clDsp/blob/main/fir_filter.cl
It takes input buffer, multiplies to filter taps (constant) buffer and writes to output.
The simple test ( https://github.com/dernasherbrezon/clDsp/blob/main/test/test_fir_filter.c ) works fine.
Performance test is not. When I execute the loop 10 times ( https://github.com/dernasherbrezon/clDsp/blob/main/test/perf_fir_filter.c#L47 )  it might hang on reading the data. Any subsequent executions of any programs on GPU will hang. So only reboot helps.
When I execute performance loop only once, then everything is fine. It produces valid results.
Another observation: running using "sudo" never hangs.
Another observation: running using "sudo" take ~10times slower than under normal user ( "pi" ).
How can I troubleshoot the slowness? Can be it related to some memory constraints or some user-specific limits while working with /dev/mem?

Without sudo:
VC4CL_DEBUG=system ./perf_fir_filter
output length max: 199
output length max: 204
working_len_total: 10996
clGetDeviceIDs: 0
clCreateContext: 0
clCreateCommandQueue: 0
allocated working buf: 10996
gpuserv: vc_gpuserv_init: starting initialisation
[VC4CL](perf_fir_filter): [VC4CL] Using VCSM (CMA) for: memory allocation
[VC4CL](perf_fir_filter): [VC4CL] Using VCHI for: kernel execution
...
average time: 0.003389

With sudo:
sudo VC4CL_DEBUG=system ./perf_fir_filter
output length max: 199
output length max: 204
working_len_total: 10996
clGetDeviceIDs: 0
clCreateContext: 0
clCreateCommandQueue: 0
allocated working buf: 10996
[VC4CL](perf_fir_filter): [VC4CL] Using mailbox for: memory allocation, system queries
[VC4CL](perf_fir_filter): [VC4CL] Using V3D for: kernel execution, profiling, system queries
...
average time: 0.051971

Ok. So performance difference can be explained by different memory allocation types.

I turned on VCHI and now sudo is slow. So this is consistent with the "no sudo".
It looks like VCHI is causing this slowness. After some number of executions, it started calculating very slowly. But the output is still valid (so it is not a timeout):
done: 1 9 0.002897
output: 0.000051934, 0.421165079
done: 2 0 0.126826
output: 0.000051934, 0.421165079

Interesting results. I never got to the actual performance comparison of the different memory allocation methods, since non-root execution does not work reliably (see tickets I linked above).
IIRC, for VCSM, the kernel manages the memory and shares it by communicating asynchronously with the firmware. For Mailbox memory management, the kernel is completely unaware of what is going on. So maybe the involvement of the kernel causes the slow-down... Of course it could very well also be that I just mess up the caching flags...
If you are interested in playing around a bit more, you could rerun the program with the VC4CL_CACHE_FORCE environment variable set to 0 (uncached), 1 (host-cached), 2 (GPU-cached) and 3 (both cached), see here.
Generally, I would recommend to run any VC4CL client as root (I know, not so great), allocate memory via the Mailbox and execute kernels via the V3D registers (the defaults for root), since this the most-tested combination by far.

It gets weirder with more tests.

Execute the loop 2 times test

without sudo: average time: 0.004408,  output: 0.000000000, 0.000000048 0.000284149, 0.000077057
with sudo: average time: 0.102403, output: 0.000051934, 0.421165079 0.305830151, 0.193653673

VC4CL_CACHE_FORCE=0 + loop executed 2 times:

without sudo: average time: 0.004297, output: -0.002558924, 0.426762044 0.305830151, 0.193653673
with sudo (timeout): average time: 60.123802 0.000000000, 0.000000000 0.000000000, 0.000000000

VC4CL_CACHE_FORCE=1 + loop executed 2 times:

without sudo: average time: 0.004470, output: 0.000000000, 0.000000048 0.000284149, 0.000077057
with sudo: average time: 0.102424, output: 0.000051934, 0.421165079 0.305830151, 0.193653673

VC4CL_CACHE_FORCE=2 + loop executed 2 times:

without sudo: average time: 0.004419, output: 0.000000000, 0.000000048 0.000284149, 0.000077057
with sudo: average time: 30.062032 (!!!!!), output: 0.000000000, 0.000000000 0.000280094, 0.000072954

VC4CL_CACHE_FORCE=3 + loop executed 2 times:

without sudo: average time: 0.004467, output: 0.000000000, 0.000000048 0.000284149, 0.000077057
with sudo: average time: 0.102449, output: 0.000051934, 0.421165079 0.305830151, 0.193653673

So it looks like:

non-sudo execution (VCHI) doesn't wait for the result.

That's why it is so fast (25x faster).
That's why the output is corrupted. It doesn't wait for the second execution to complete, so the buffer with previous results is returned. Or some rubbish returned in case of VC4CL_CACHE_FORCE=0

GPU-cached memory produce rubbish when using with sudo.
Only with sudo and host-based cache produce valid results.

The performance gap between sudo/no-sudo is very important here, because executing very similar filter on CPU (+neon) gives "average time: 0.070378". So it lies somewhere between sudo and non-sudo.
There are number of things I can try to optimise performance, but first of all I would like to make the program stable :)
Some optimisations:

Cache filter taps into the local memory
Output results into local memory first and then copy them into the global
Use map/unmap for external buffer.

Performance-wise, having a quick look at your code, you could try some of these:

When the kernel is compiled, the buffer sizes (as well as the decimation parameter) are already known, so setting them via a macro could unroll the inner loop and optimize some arithmetic operations. This of course disallows caching the compiled kernel program to be run with different parameters...
If your data size is a multiple of 8/16 elements (or you can pad it to be), you may want to rewrite the kernel to use 8-/16-element vector types (i.e. load and store float16 values). This could greatly improve performance.
Even if data cannot be processed completely as vectorized types, loading and storing vectors (and thus having less memory access instructions) alone can give a good performance boost, since usually memory-access is a bottleneck on the VC4.

E.g. this code has only half the memory accesses and takes (without scheduling overhead) about 25% less cycles to execute:
__kernel void fir_filter_process_memory(__global const float2 *restrict input, __global const float2 *restrict taps, const unsigned int taps_len, __global float2 *output, const unsigned int decimation, const unsigned int output_len) {

    for (unsigned int i = 0; i < output_len; i++) {
        int output_offset = get_global_id(0) * output_len + i;
        int input_offset = output_offset * decimation;
        float real0 = 0.0f;
        float imag0 = 0.0f;
        for (unsigned int j = 0; j < taps_len; j++) {
            float2 in = input[input_offset + j];
            float2 tap = taps[j];
            real0 += (in.x * tap.x) - (in.y * tap.y);
            imag0 += (in.x * tap.y) + (in.y * tap.x);
        }
        output[output_offset] = (float2)(real0, imag0);
    }
}

This code could be twice as fast (again ignoring scheduling overhead), but requires the input and taps buffers to be padded to 8 floats and a constant tap-size of 3:
__kernel void fir_filter_process_vectorized(__global const float *restrict input, __global const float *restrict taps, __global float2 *output, const unsigned int decimation, const unsigned int output_len) {

    for (unsigned int i = 0; i < output_len; i++) {
        int output_offset = (get_global_id(0) * output_len + i);
        int input_offset = output_offset * 2 * decimation;
        float8 in = *((__global float8*)(input + input_offset));
        float8 tap = *((__global float8*)taps);
        float real0 = 0.0f;
        float imag0 = 0.0f;

        // 1 tap
        real0 += (in.s0 * tap.s0) - (in.s1 * tap.s1);
        imag0 += (in.s0 * tap.s1) + (in.s1 * tap.s0);
        // 2 tap
        real0 += (in.s2 * tap.s2) - (in.s3 * tap.s3);
        imag0 += (in.s2 * tap.s3) + (in.s3 * tap.s2);
        // 3 taps
        real0 += (in.s4 * tap.s4) - (in.s5 * tap.s5);
        imag0 += (in.s4 * tap.s5) + (in.s5 * tap.s4);

        output[output_offset] = (float2)(real0, imag0);
    }
}

Ok. I will go with sudo-enabled access and start optimising the code.
This is not related to this issue, but I switched to float8 and got 5x performance boost:

original - average time: 0.503197
float8 (https://github.com/dernasherbrezon/clDsp/blob/main/fir_filter_float8.cl) - average time: 0.176660
CPU (neon via libvolk) - average time: 0.261358

So it looks like it is possible to beat CPU on this task.

I have been extensively testing the timeout issue for the last several days:

It seems mailbox call 0x00030011 returns before the actual computation completes. Similar to the issue with VCHIQ. I've made a loop that runs the same kernel + sleep(1).

Normally computation takes 16k us and lots of execution cycles:
[VC4CL](VC4CL Queue Han): Elapsed time: 16264us
[VC4CL](VC4CL Queue Han): Clock speed: 0
[VC4CL](VC4CL Queue Han): Instruction count: 461
[VC4CL](VC4CL Queue Han): Explicit uniform count: 3
[VC4CL](VC4CL Queue Han): QPUs used: 12
[VC4CL](VC4CL Queue Han): Kernel repetition count: 17
[VC4CL](VC4CL Queue Han): Execution cycles: 38808180

But after some time it returns earlier:
[VC4CL](VC4CL Queue Han): Elapsed time: 548us
[VC4CL](VC4CL Queue Han): Clock speed: 0
[VC4CL](VC4CL Queue Han): Instruction count: 461
[VC4CL](VC4CL Queue Han): Explicit uniform count: 3
[VC4CL](VC4CL Queue Han): QPUs used: 12
[VC4CL](VC4CL Queue Han): Kernel repetition count: 17
[VC4CL](VC4CL Queue Han): Execution cycles: 801568

With much smaller execution cycles. However due to sleep(1) I can submit the same kernel again. It returns earlier, but the execution cycles (which are read from the GPU) stay relatively normal.
[VC4CL](VC4CL Queue Han): Elapsed time: 548us
[VC4CL](VC4CL Queue Han): Clock speed: 0
[VC4CL](VC4CL Queue Han): Instruction count: 461
[VC4CL](VC4CL Queue Han): Explicit uniform count: 3
[VC4CL](VC4CL Queue Han): QPUs used: 12
[VC4CL](VC4CL Queue Han): Kernel repetition count: 17
[VC4CL](VC4CL Queue Han): Execution cycles: 38803212

So it looks like GPU computes the kernel, but for some reason mailbox interface returns earlier.

Buffer size has no effect. I can reduce it and still get these early returns.
Simple kernel works well. I tried the following kernel with the same buffers, input parameters and never seen early returns:

__kernel void fir_filter_process(__global const float *restrict input, __global const float *restrict taps, __global float *output) {

    int output_offset = get_global_id(0) * 2;

    output[output_offset] = 1;
    output[output_offset + 1] = 2;

}

Couple more observations:

If I remove sleep(1), then after early return I cannot it again. Application will crash GPU (?) or saturate some internal buffer in ThreadX? Only power off/on helps.
With sleep(1) I can re-run application and get the same 16k us execution times. So something resets/re-initialize between 2 subsequent executions.

Tried running code similar to add.py:
    for x in range(100):
      start = time.time()
      drv.execute(
        n_threads=n_threads,
        program=code,
        uniforms=uniforms
      )
      elapsed_gpu = time.time() - start
      print('GPU: {:.4f} sec'.format(elapsed_gpu))

using py-videocore and got the same timeout:
GPU: 0.0044 sec
GPU: 0.0044 sec
Traceback (most recent call last):
  File "add.py", line 118, in <module>
    uniforms=uniforms
  File "/usr/local/lib/python3.7/dist-packages/videocore/driver.py", line 238, in execute
    r = self.mailbox.execute_qpu(n_threads, message.address, 0, timeout)
  File "/usr/local/lib/python3.7/dist-packages/videocore/mailbox.py", line 110, in f
    r = self._simple_call(name, tag, req_fmt, res_fmt, list(args))[5:]
  File "/usr/local/lib/python3.7/dist-packages/videocore/mailbox.py", line 98, in _simple_call
    ioctl(self.fd, IOCTL_MAILBOX, buf, True)
TimeoutError: [Errno 110] Connection timed out

Tried bullseye and got timeout after very first execution when executing via MAILBOX.
[VC4CL](VC4CL Queue Han): Mailbox buffer before: 00000020 00000000 00030002 00000008 00000004 00000005 00000000 00000000
[VC4CL](VC4CL Queue Han): Mailbox buffer after: 00000020 80000000 00030002 00000008 80000008 00000005 11e1a300 00000000
[VC4CL](VC4CL Queue Han): Mailbox request: succeeded
[VC4CL](VC4CL Queue Han): Mailbox buffer before: 00000028 00000000 00030011 00000010 00000010 0000000c bebdc290 00000000 00007530 00000000
[VC4CL](VC4CL Queue Han): ioctl_set_msg failed: -1
[VC4CL] Error in mbox_property: Connection timed out

Here is firmware version:
pi@raspberrypi:~ $ sudo vcgencmd version
Oct 29 2021 10:49:08
Copyright (c) 2012 Broadcom
version b8a114e5a9877e91ca8f26d1a5ce904b2ad3cf13 (clean) (release) (start)

Isn't this also related to https://github.com/raspberrypi/linux/issues/4321?
Can you check how long it actually took to time out (from the start of that particular Mailbox call to the timeout error)?

Isn't this also related to raspberrypi/linux#4321?

Unlikely. I'm executing exactly the same code all the time and it takes ~16264us to execute. On "buster" I've got timeout after several executions. While on "bullseye" it instantly fails.


Node.js  uvwasi team Meeting 2023-08-16

Time
UTC Wed 16-Aug-2023 18:00 (06:00 PM):

Timezone
Date/Time

US / Pacific
Wed 16-Aug-2023 11:00 (11:00 AM)

US / Mountain
Wed 16-Aug-2023 12:00 (12:00 PM)

US / Central
Wed 16-Aug-2023 13:00 (01:00 PM)

US / Eastern
Wed 16-Aug-2023 14:00 (02:00 PM)

EU / Western
Wed 16-Aug-2023 19:00 (07:00 PM)

EU / Central
Wed 16-Aug-2023 20:00 (08:00 PM)

EU / Eastern
Wed 16-Aug-2023 21:00 (09:00 PM)

Moscow
Wed 16-Aug-2023 21:00 (09:00 PM)

Chennai
Wed 16-Aug-2023 23:30 (11:30 PM)

Hangzhou
Thu 17-Aug-2023 02:00 (02:00 AM)

Tokyo
Thu 17-Aug-2023 03:00 (03:00 AM)

Sydney
Thu 17-Aug-2023 04:00 (04:00 AM)

Or in your local time:

https://www.timeanddate.com/worldclock/fixedtime.html?msg=Node.js+Foundation+uvwasi team+Meeting+2023-08-16&iso=20230816T1800
or https://www.wolframalpha.com/input/?i=06PM+UTC%2C+Aug+16%2C+2023+in+local+time

Links

Minutes Google Doc: https://docs.google.com/document/d/1qIZmI2in5FXWNYSfI3rhbA8B9gib7APbXY8h-f8etR8/edit

Agenda
Extracted from uvwasi-agenda labelled issues and pull requests from the nodejs org prior to the meeting.
nodejs/node

WASI preview 1: fd_readdir ignores cookie argument #47193

nodejs/uvwasi

What's new with uvwasi in 2023 #201
Implement 4 stubbed out socket methods #200
Changes to WASI spec in snapshot_preview2 #59

Invited

uvwasi team: @nodejs/wasi

Observers/Guests
Notes
The agenda comes from issues labelled with uvwasi-agenda across all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.
Joining the meeting

link for participants: https://zoom.us/j/91612484369
For those who just want to watch:
youtube admin page: https://www.youtube.com/my_live_events?filter=scheduled

Minutes from the meeting today
Node.js  uvwasi team Meeting 2023-08-16
Links

Recording:  https://www.youtube.com/watch?v=5Ydrm9piD4M
GitHub Issue: https://github.com/nodejs/uvwasi/issues/225

Present

Michael Dawson @mhdawson
Oscar Spencer @ospencer
Blaine Bublitz @phated
Guy Bedford

Agenda
Announcements
*Extracted from uvwasi-agenda labelled issues and pull requests from the nodejs org prior to the meeting.
nodejs/node

WASI preview 1: fd_readdir ignores cookie argument #47193

No update on this one

nodejs/uvwasi

What's new with uvwasi in 2023 #201

no updates this week

Implement 4 stubbed out socket methods #200

Landed

Changes to WASI spec in snapshot_preview2 #59

Oscar trying to have preview 2 finished next month
Guy

Hoping to have resources for JS next week
Priority will be having JS shims to be able to test out in Node.js, ready in about 2 months to start looking at what an implementation in uvwasi

Q&A, Other
Upcoming Meetings

Node.js Project Calendar: https://nodejs.org/calendar

Click +GoogleCalendar at the bottom right to add to your own Google calendar.


output format for PartNormalDataset

Hi,
As the PartNormalDataset dataloader for the part-seg task is not provided in the repository, I'm assuming that it is similar, if not the same, as the one here. But according to the training script (train_partseg.py),  it seems that both the input and output of the dataloader have some slight differences from the original one.
So could you please clarify the difference or provide the modified PartNormalDataset dataloader?
Thanks

sorry I don't understand your question, would you mind providing some examples? what is the original dataloader you are referring to?

One of the dataloaders for ShapeNetPart
https://github.com/hansen7/OcCo/blob/f5f2157ba4186ccaaa5d826b81a0af6222f1ee06/OcCo_Torch/train_partseg.py#L11
in train_partseg.py is not included in the repository. So the training script for the partseg task cannot be executed.
It seems that the dataloader is similar to the one here in PointNet++. But It turns out that both the input and output are slightly different.

Ohoh, just committed, thanks for noting this!


Feature/742 remove dependency on active record

Check for the presence of ActiveRecord before using it

@coorasse This should solve the issue of ActiveRecord not being present


Decide upon consistent coding style

We have a lot of inconsistencies, with different styles being added in different places.
Includes:

tab width?
braces on end of line or next?
Use braces for single line if/while statements
variable naming (private/protected/public)
method naming (private/protected/public)

Once we decide on a standard, we can all set up our visual studio settings to conform to the same standard, that way whenever submitting a PR, people don't end up changing unrelated code because they wanted to auto-format.

My proposal:

2 space width
braces on end
always use braces, even for single line if/while statements
_name for private, name for protected/public
name for private/protected, Name for protected/public

Agreed here.  @jo3w4rd That work for you?

Sure.

This bit is ambigious:
_name for private, name for protected/public
name for private, Name for protected/public
Can I vote for the first line everywhere?  Or is it too late to change that.

Yeah nothing is set in stone at all.  That part might have been a little ambiguous, the first line was talking about variables and the second line was talking about methods. Is your request to have _name for both private variables and methods?

VC_CSharp.zip
Here is a settings file that we can all use to setup the CSharp text editing settings.

Once the settings are loaded, you can auto-format a file by first pressing CTRL+K, then CTRL+D


Size mismatch in the morphing example

I am trying to run the morphing example from the docs. When I get to morphing in the STRAIGHT and run the following line:
mObject3 = directSTRAIGHTmorphing(neutralHai,angryHai,0.5,'linear');

I get the following error:

Subscripted assignment dimension mismatch.
Error in directSTRAIGHTmorphing (line 45)
ap(1:nr1,1:nc1) = (1-mRate)*mObject1.aperiodicityIndex;

Debugging shows that nr1 = 2049 while size(mObject1.aperiodicityIndex, 1) = 1025. I am confused as to what might cause this.

Thank you for pointing out this.
This problem is caused by the mismatch in default parameter setting in exstraightsource.m and exstraightspec.m .
Please edit the line 230 of  exstraightsource.m by changeing 40 to 80
prm.F0defaultWindowLength = 80; % default frame length for pitch extraction (ms)
This fixes the problem.
I will check the other side effects. If it is OK, then, I will update the code.

I checked side effects and found it is OK.


Create run.bat

run bat instead of typing commands in cmd

Kind of spoon feeding... but ok : )

Kind of spoon feeding... but ok : )


bump Go vertion to 1.22

Closes #314

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.41%. Comparing base (eaa8692) to head (fd43f48).
Report is 2 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #315       +/-   ##
===========================================
- Coverage   96.71%   84.41%   -12.31%
===========================================
  Files          60       97       +37
  Lines        2621     2630        +9
===========================================
- Hits         2535     2220      -315
- Misses         58      381      +323
- Partials       28       29        +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Inner union structs not generated

Input C/C++ Header
https://github.com/couchbase/libcouchbase/blob/master/include/libcouchbase/couchbase.h#L275
struct lcb_create_st {
    /** Indicates which field in the @ref lcb_CRST_u union should be used. Set this to `3` */
    int version;

    /**This union contains the set of current and historical options. The
     * The #v3 field should be used. */
    union lcb_CRST_u {
        struct lcb_create_st0 v0;
        struct lcb_create_st1 v1;
        struct lcb_create_st2 v2;
        struct lcb_create_st3 v3; /**< Use this field */
    } v;
    LCB_DEPR_CTORS_CRST
};

Bindgen Invokation
    let _ = bindgen::builder()
        .header("headers.h")
        .clang_arg("-I")
        .clang_arg(bindgen_path)
        .no_unstable_rust()
        .generate_comments(false)
        .generate()
        .unwrap()
        .write_to_file(Path::new(out_dir).join("bindings.rs"));

Actual Results
error[E0412]: cannot find type `lcb_create_st_lcb_CRST_u` in this scope
    --> /Users/daschl/code/rust/couchbase-rs/target/debug/build/couchbase-sys-4dafd1d829eed0ef/out/bindings.rs:3635:12
     |
3635 |     pub v: lcb_create_st_lcb_CRST_u,
     |            ^^^^^^^^^^^^^^^^^^^^^^^^ not found in this scope

error[E0204]: the trait `Copy` may not be implemented for this type
    --> /Users/daschl/code/rust/couchbase-rs/target/debug/build/couchbase-sys-4dafd1d829eed0ef/out/bindings.rs:3632:17
     |
3632 | #[derive(Debug, Copy)]
     |                 ^^^^
...
3635 |     pub v: lcb_create_st_lcb_CRST_u,
     |     ------------------------------- this field does not implement `Copy`

error: aborting due to previous error

#[repr(C)]
#[derive(Debug, Copy)]
pub struct lcb_create_st {
    pub version: ::std::os::raw::c_int,
    pub v: lcb_create_st_lcb_CRST_u,
}

Expected Results
I think the problem is that lcb_create_st expects lcb_create_st_lcb_CRST_u but its not created. I've been upgrading from 0.21 which used to work, but its broken since 0.22 I think (and is broken right now I tried 0.23.1).

Can you check on master? It's likely a dupe of https://github.com/servo/rust-bindgen/issues/643

yes that indeed seems to fix it - thanks!


Use organization_id instead of organization_url

This commit uses organization_id instead of organization_url
to create new stacks with. This is the UI part of aptible/api.aptible.com#231.
cc @fancyremarker @sandersonet

Looks good to me @blakepettersson -- Is it safe to remove the organizationUrl attr from shared too?

Not completely; there's a method findStacksFor in store.js which uses the organizationUrl to find all stacks by organization.

LGTM


doc, tools: Use make clean-all before change NEMU config

The NEMU was built as standalone binary w/o -fPIC in early section "Run workload using NEMU emulator". And in the following section "Run workload on Xiangshan core simulator", the NEMU was built as shared library, in which the -fPIC flag was requried.
But the default make clean target did not cleanup the build output file resource/softfloat/build/softfloat.a, which cause the linker report following error:
+ ccache g++ /home/xiangshan/xs-env/NEMU/build/riscv64-nemu-interpreter-so
/usr/bin/ld: resource/softfloat/build/softfloat.a(s_mulAddF64.o): warning: relocation against `softfloat_roundingMode' in read-only section `.text'
/usr/bin/ld: resource/softfloat/build/softfloat.a(f16_sqrt.o): relocation R_X86_64_PC32 against symbol `softfloat_approxRecipSqrt_1k0s' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status

So, change make clean to make clean-all to cleanup all outputs that may be in-compatible between builds.

Currently, make clean would also clean softfloat. This is added in NEMU#278.
Also, we should bump repos in xs-env.

got it, PR closed


Add new Rails/PluckId cop

https://rails.rubystyle.guide/#ids

Updated.

Is it as expected to prioritize this Rails/PluckId cop over Rails/PluckInWhere cop when using auto-correction?
% cat app/models/example.rb
class Foo < ApplicationRecord
  def bar
    Bar.where(baz_id: Baz.pluck(:id))
  end
end

% bundle exec rubocop app/models/example.rb -A --only Rails/PluckInWhere,Rails/PluckId
(snip)

Inspecting 1 file
C

Offenses:

app/models/example.rb:3:27: C: [Corrected] Rails/PluckId: Use ids instead of pluck(:id).
    Bar.where(baz_id: Baz.pluck(:id))
                          ^^^^^^^^^^
app/models/example.rb:3:27: C: [Corrected] Rails/PluckInWhere: Use select instead of pluck within where query method.
    Bar.where(baz_id: Baz.pluck(:id))
                          ^^^^^

1 file inspected, 2 offenses detected, 2 offenses corrected

% cat app/models/example.rb
class Foo < ApplicationRecord
  def bar
    Bar.where(baz_id: Baz.ids)
  end
end

Thanks! This is not expected. Updated the code to check if in where and added a test case for this.

Updated.

Thanks!


Be sad about Kotlin’s StringBuilder

Kotlin’s StringBuilder is O(N) on every operation because its underlying representation is a String. For example, here’s append():
    actual fun append(value: String?): StringBuilder {
        this.string += value ?: "null"
        return this
    }

Many stdlib functions build on StringBuilder, and therefore are superslow on large inputs. In this pull request I changed Zipline’s internal implementation to avoid this trap. (I gave up after an hour when timing calls to CharArray.concatToString() running QuickJS on my MacBook Pro.)

Array Size
Time (seconds)

100
0

1.000
0

10,000
0

100,000
1

1,000,000
24

10,000,000
3600+

We should confirm that all major JS engines implement an append optimization that makes this implementation forgivable. And then we should decide how to mitigate:

Upstream a similar optimization to QuickJS
Upstream a change to Kotlin/JS to switch to an implementation that’s efficient on all JS engines
Write our own routines for String concatenation, and avoid this trap

https://josephmate.github.io/java/javascript/stringbuilder/2020/07/27/javascript-does-not-need-stringbuilder.html

Looks like all the major JS engines use ropes, and there's a TODO to do that for QuickJS.
https://github.com/bellard/quickjs/blob/master/TODO#L49

Also this! https://github.com/bellard/quickjs/issues/67#issuecomment-897448486

I think our next step is sadly making our own custom build of kotlin-stdlib that uses different StringBuilder internals. Sigh!

Moving JSON encoding into Json.encodeToStringFast() has mitigated the need for this.


/bin/sh: ng: command not found

I am generating an app but failing with /bin/sh: ng: command not found.
Starter Welcome:
Welcome to the Ionic JHipster Starter!
The Ionic JHipster Starter ships with login and signup that will work with a JHipster backend (with JWT Authentication).
For more details, please see the project's README:
https://github.com/oktadeveloper/ionic-jhipster-starter/blob/master/README.md
[INFO] Next Steps:
   - Go to your newly created project: cd ./ionic4j
   - Run ionic serve within the app directory to see your app
   - Build features and components: https://ion.link/scaffolding-docs
   - Get Ionic DevApp for easy device testing: https://ion.link/devapp

/bin/sh: ng: command not found
Error jhipster-ionic
ng add<EMAIL_ADDRESS>--configUri=http://localhost:8080/api/auth-info --issuer=null --clientId=null

/bin/sh is there and working fine.
OS info
Darwin MacBookPro.local 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64

When using OIDC for Auth, you need to install Angular CLI as well.
npm install -g @angular/cli

On Sep 7, 2019, at 06:26, P. Mohan<EMAIL_ADDRESS>wrote:
I am generating an app but failing with /bin/sh: ng: command not found.
Starter Welcome:
Welcome to the Ionic JHipster Starter!
The Ionic JHipster Starter ships with login and signup that will work with a JHipster backend (with JWT Authentication).
For more details, please see the project's README:
https://github.com/oktadeveloper/ionic-jhipster-starter/blob/master/README.md
[INFO] Next Steps:

Go to your newly created project: cd ./ionic4j
Run ionic serve within the app directory to see your app
Build features and components: https://ion.link/scaffolding-docs
Get Ionic DevApp for easy device testing: https://ion.link/devapp
/bin/sh: ng: command not found
Error jhipster-ionic

ng add<EMAIL_ADDRESS>--configUri=http://localhost:8080/api/auth-info --issuer=null --clientId=null
/bin/sh is there and working fine.
OS info
Darwin MacBookPro.local 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Thanks a lot @mraible.


IDEM-2869: add the logins that are allowed for each server node

The ./tsh ls command did not display which user names are available for each node.
It is useful to know which usernames can be used with which remote server.
The tis certificate has the user traits, that in turn contains the user entitlements.
Parse the entitlements and then for the remote servers, get which logins can be used with each node.
Then when we display the server, display which logins the user can use.
Since the tis certificate is obtained when the ./tsh login command is used, if the user got new remote servers since last login those server logins will not appear. They will have to logout and log back in.


Lesson 12 Plant Anatomy and Plant Physiology

We need to create markdown for  Plant Anatomy and Plant Physiology
chapter-13.pdf

Hey, there..I have made some changes in the file Plant Anatomy and Plant Physiology with some alignment, indendations and images.


[70B-Part1] Prefetch weights separately

This PR allows for downloading the weights of the model first, before starting the DDP processes. The previous approach tends to time out when using 70B model.

@liPatrick I'm gonna add you to the PRs for visibility. You don't have to review.


[SPARK-24696][SQL] ColumnPruning rule fails to remove extra Project

What changes were proposed in this pull request?
The ColumnPruning rule tries adding an extra Project if an input node produces fields more than needed, but as a post-processing step, it needs to remove the lower Project in the form of "Project - Filter - Project" otherwise it would conflict with PushPredicatesThroughProject and would thus cause a infinite optimization loop. The current post-processing method is defined as:
  private def removeProjectBeforeFilter(plan: LogicalPlan): LogicalPlan = plan transform {
    case p1 @ Project(_, f @ Filter(_, p2 @ Project(_, child)))
      if p2.outputSet.subsetOf(child.outputSet) =>
      p1.copy(child = f.copy(child = child))
  }

This method works well when there is only one Filter but would not if there's two or more Filters. In this case, there is a deterministic filter and a non-deterministic filter so they stay as separate filter nodes and cannot be combined together.
An simplified illustration of the optimization process that forms the infinite loop is shown below (F1 stands for the 1st filter, F2 for the 2nd filter, P for project, S for scan of relation, PredicatePushDown as abbrev. of PushPredicatesThroughProject):
                             F1 - F2 - P - S
PredicatePushDown      =>    F1 - P - F2 - S
ColumnPruning          =>    F1 - P - F2 - P - S
                       =>    F1 - P - F2 - S        (Project removed)
PredicatePushDown      =>    P - F1 - F2 - S
ColumnPruning          =>    P - F1 - P - F2 - S
                       =>    P - F1 - P - F2 - P - S
                       =>    P - F1 - F2 - P - S    (only one Project removed)
RemoveRedundantProject =>    F1 - F2 - P - S        (goes back to the loop start)

So the problem is the ColumnPruning rule adds a Project under a Filter (and fails to remove it in the end), and that new Project triggers PushPredicateThroughProject. Once the filters have been push through the Project, a new Project will be added by the ColumnPruning rule and this goes on and on.
The fix should be when adding Projects, the rule applies top-down, but later when removing extra Projects, the process should go bottom-up to ensure all extra Projects can be matched.
How was this patch tested?
Added a optimization rule test in ColumnPruningSuite; and a end-to-end test in SQLQuerySuite.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/591/
Test PASSed.

Test build #92485 has started for PR 21674 at commit 11fde8b.

cc @gatorsmile

Test build #92486 has started for PR 21674 at commit f45a8b8.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/592/
Test PASSed.

Merged build finished. Test PASSed.

Test build #92486 has finished for PR 21674 at commit f45a8b8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92486/
Test PASSed.

Merged build finished. Test PASSed.

Thanks! Merged to master/2.3


Adding support for streaming over XHR

Implemented basic support for streaming when using XHR.
That was implemented by requesting small chunks of the video each time, using HTTP Range headers.
Decoder was changed in order to avoid BitReader from trying to read past the end of the current buffer. When this happens, jsmpeg transitions to buffering state and, as soon as the next chunk loads, video continues playing

Obsolete


not able to plug in simple swift project

Hi,
I like your controls, this is really helpful,
Some reason I am not able to plug in my project.
I have navigation view project, in home view I need to include only left slide view.
Thanks in advance,
Imtayaz

hi,
Do you want to use only the leftViewController ?
If so, try to do as follows.
ex)
let slideMenuController = SlideMenuController(mainViewController:nvc, leftMenuViewController: leftViewController)

Hi Yuji,
Thanks for quick reply, some reason I am not able to use this controls, I will retry tonight and let you know if I have same issue.
Thanks,
Imtee

All right.

Because like no problem, I will close.


Typo in definition for "Development status" (SWO_9000061)

The human readable definition (obo:IAO_0000115) for SWO_9000061 currently reads:
Development status is an information content entity which indicates the maturity of a sofrware entity within the context of the software life cycle.
The word "sofrware" is a typo and should be corrected to "software"

Thanks very much for letting us know! I'll modify the OWL this week and get back to you.

Fixed.


Add media kit page and assets

Resolves #25
Brief description
Add media kit page so that 3rd parties can have easy access to Koinos branding assets.
Checklist

[x] I have built this pull request locally
[ ] I have ran the unit tests locally
[x] I have manually tested this pull request
[x] I have reviewed my pull request
[ ] I have added any relevant tests

Demonstration

Should this be accessible via link? There's no way to get to the page or know it exists. Let's discuss at standup.


Serialization doesn't work with forking

flo version: 0.4.1
Minimal reproducible example:
package com.spotify.data.clickhouse;

import com.spotify.flo.Task;
import com.spotify.flo.context.FloRunner;
import org.apache.beam.sdk.options.PipelineOptions;
import org.apache.beam.sdk.options.PipelineOptionsFactory;

public class App {
  public static void main(String[] args) {
    PipelineOptions opts = PipelineOptionsFactory.fromArgs(args).create();

    FloRunner
        .runTask(task(opts))
        .waitAndExit();
  }

  public static Task<String> task(PipelineOptions opts) {
    return Task.named("task")
        .ofType(String.class)
        .process(opts::toString);
  }
}

Workaround
Create resources/flo.conf and put flo.forking = false there.

Flo uses kryo for serialization and org.apache.beam.sdk.options.PipelineOptions is not kryo-serializable.
This can be worked around by instantiating the PipelineOptions inside the task, or passing in a closure instead:
import com.spotify.flo.Task;
import com.spotify.flo.TaskBuilder.F0;
import org.apache.beam.sdk.options.PipelineOptions;
import org.apache.beam.sdk.options.PipelineOptionsFactory;

public class App {

  public static void main(String[] args) {
    F0<PipelineOptions> opts = () -> PipelineOptionsFactory.fromArgs(args).create();

    FloRunner
        .runTask(task(opts))
        .waitAndExit();
  }

  public static Task<String> task(F0<PipelineOptions> opts) {
    return Task.named("task")
        .ofType(String.class)
        .process(() -> opts.get().toString());
  }
}

The difficulty of serializing PipelineOptions seems somewhat intentional:

{@link PipelineOptions} is intentionally not marked {@link java.io.Serializable}

https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/options/PipelineOptions.java#L182
However, it is json serializable. Would perhaps be good if flo could attempt json serialization for such types.

They really don't want PipelineOptions to be java serialized:

PipelineOptions objects are not serializable

https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/options/ProxyInvocationHandler.java#L175

Not entirely fixed, but error messages are improved in https://github.com/spotify/flo/pull/178

Should get much better error messages in https://github.com/spotify/flo/pull/179


Undocumented trim option

Describe the bug
Both the Python sources and JavaScript sources support an option called trim. The overrides section of the docs omits this option.
Expected behavior
The trim option documented along with size and friends.
Additional context
Related to https://github.com/openjournals/joss-reviews/issues/766.

This option was originally intended for a string of fixed length with a delimited string inside. In a later versions, this functionality is offered by using the size and delimiter parameters together. The only use left for trim is to control the padding character when writing such a field. I think I will remove this option.


Feature Request: Load Testing Setup with Grafana

I recently saw there is a new Azure Typescript function. Congratulations!
I was going to write my own, but I will use this insteas.
Could we include load testing?
For example, check this out: https://alex-klaus.com/load-test-k6-typescript-azure/

Thanks @AndrewRedican !
I'm not sure what you mean by including this. The package is simply a tool to run Azure Functions in an NX workspace. If you want to implement with them load testing, you can do so just like with any other Azure Functions setup.
Am I missing something?

I'm closing this for now due to inactivity of the thread.

I'm closing this for now due to inactivity of the thread.


Re-order the essentials bank "show" to be in the same order as the essentials bank "edit"

Summary
The "show" for Organization is currently showing things in a different order than the "edit" for Organization.   Make the show the same as the edit
Details
Recreation
Sign in as<EMAIL_ADDRESS>  Left hand menu -- My Organization.  To see the edit, scroll down and click on the "edit" button
Make the elements appear on the "show" page in the same order as they are in the "edit" page.  There may be some items that are not appearing on the "show" page.  Please add them, in the appropriate place.
Criteria for completion

[ ] All elements that are on the edit page, appear on the show page in the same order
[ ] Screenshots to demonstrate that should be attached to the PR

Hey. I would like to work on this

@napster235 Go ahead!


Is there no onFocus and onblur function?

I want to trigger a function as soon as it is focused on the input.

Hi @jonhy751, can you follow the issue template next time you open an issue? it help us to get back to you faster
for now, the best way to achieve what you asked is:
  const editorState = useBridgeState(editor);

  useEffect(() => {
    if (editorState.isFocused) {
      // Do water you want when the editor is focused
    }
  }, [editorState.isFocused]);

In the future we are planning on implementing onFocus and onBlur into useEditorBridge

closing this for now


Regression, Custom models

This framework needs support for regression and for custom models to send in.  It is currently only for classification.

I accept pull requests!


Jest broken on Node 11.11.0

This is my intent (choose one)

[x] I want to report a bug
[ ] I want to request a feature or change
[ ] I want to provide or change a feature

The problem
Unfortunately, Jest ist broken on the lates Node version, 11.11.0, see

https://github.com/facebook/jest/pull/8050
https://github.com/facebook/jest/issues/8070

I only noticed this because the canarist checks in untool that do not use hops’ yarn.lock are failing.
Proposed solution
Upgrade to<EMAIL_ADDRESS>- since we are hesitant to upgrade to jest@24 in general (#780, #788, #799), we should probably consider alternatives such as updating the engines fields of our packages.

Could we use a range for the dependencies in question here, i.e.  support borh jest@23 and jest@24?

I would also propose considering a more conservative strategy of declaring our engines fields by explicitly mentioning the major versions we want to support.
 "engines": {
    "node": "^8.10 || 9 || 10 || 11",
    "yarn": ">=1.10.1" // Maybe also with yarn?
  },

It might be particularly useful as long as we're keeping pretty strict policies about realising major versions on our side.
Beside that > means that hops is/will be compatible with the next versions of Node or Yarn, and we don't know that, yet :).

I asked whether it would be possible to release a fix for Jest 23, but unfortunately it is not (https://github.com/facebook/jest/pull/8050#issuecomment-471639713) so we will have to upgrade Jest on our end to v24.
The problem is, that this would be a breaking change, because we expect our users to have jest installed as a dependency in their projects, but we ship the jest-preset-hops which is currently tied to jest 23.
In our preset we have some dependencies to jest packages https://github.com/xing/hops/blob/master/packages/jest-preset/package.json#L30-L37 and since these are dependencies instead of peer dependencies, we can't specify a range on them to support both versions, therefore we have to update the preset to only support v24 and the breakage would then be on the user's side, because they will have to update their local jest dependencies to v24 too.
Unfortunately I don't see another way around this at the moment, so we'll have to discuss whether we want to delay this until the end of the year when we ship Hops 12, or whether we want to release Hops 12 now or if we want break the semver rules and publish it in Hops 11 anyways..
In the long term I would like to see jest completely encapsulated in Hops, so that we provide a command (eg. hops test) which will execute jest, so that our users don't need to install jest or anything else.
That would allow us to update and ship any version of jest that we want to.
@dmbch @toomuchdesign @herschel666 @robin-drexler @jhiode what are your opinions?

How about a new packages like jest-24-preset-hops?  This way users can decide which Jest version they want to use.

If we were to postpone the Jest upgrade until Hops 12, we would have to limit node versions to <11.11, at least for jest-preset-hops. I personally would consider that an acceptable trade-off, especially considering that it is just a devDependency.
With regards to wrapping Jest altogether..: :scream:

Oh, and we could consider kickstarting a next branch sometime soon. I would prefer that to adding new packages...

In the long term I would like to see jest completely encapsulated in Hops, so that we provide a command (eg. hops test) which will execute jest, so that our users don't need to install jest or anything else.

We're also not wrapping babel in a hops-transpiler package or webpack in a hops-bundler package. What makes jest special in this regard?
Couldn't we, in the future (i.e. hops 12), have jest dependencies be peerDependencies? This would allow to open up the range should new jest versions become available (and still be compatible with what ever we are doing with it :D)

In the long term I would like to see jest completely encapsulated in Hops, so that we provide a command (eg. hops test) which will execute jest, so that our users don't need to install jest or anything else.

We're also not wrapping babel in a hops-transpiler package or webpack in a hops-bundler package. What makes jest special in this regard?

Yes we do actually - that's what hops build is doing ;)
And other projects, such as CRA are doing the same thing: https://facebook.github.io/create-react-app/docs/getting-started#npm-test-or-yarn-test https://facebook.github.io/create-react-app/docs/running-tests https://github.com/facebook/create-react-app/blob/master/packages/react-scripts/scripts/test.js

If we can't update to jest v24 only because of the babel-jest dependency declared in jest-preset-hops, releasing a new temporary jest-24-preset-hops package looks a fair trade-off to me in terms of non-breaking releases and providing a working setup on Node 11+.


gitlab-runner: 15.0.0 -> 15.1.0

Description of changes
https://gitlab.com/gitlab-org/gitlab-runner/-/tags/v15.1.0

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

/cc @max-wittig


Interpretation

Please can the contents of the readme files for the challenges be interpreted to English?

Please can the contents of the readme files for the challenges be interpreted to English?

In the next time, I have plans to translate the readmes. Thanks!


ENH: support creation of analog SOS outputs

Reference issue
Closes gh-5668.
What does this implement/fix?
Add analog argument, default False, to zpk2sos, and add new pairing
option 'minimal' to construct analog and minimal discrete SOS arrays.
tf2sos uses zpk2sos; add analog argument here as well, and pass it on
to zpk2sos.
iirfilter already had an analog argument; pass it on to zpk2sos.
Added tests for new arguments and behaviour.
Additional information
This introduces a form of sos output that won't work with sosfilt, sosfiltfilt, etc. -- but the same is true of analog=True output type ba being used with lfilter.
There are at least two applications for this: the one is in designing electronic filters (e.g., cascaded Sallen-Key filters).  The other is as a first step in a zpk2ss transformation; in this case the new minimal pairing would be useful for both discrete and analog filters.
I haven't updated sos2zpk to handle the new minimal pairing type; I don't understand why sos2zpk normalizes the sections, or at least, I think it normalizes them incorrectly.  See gh-7345.

I'll fix the lint errors too, though if I run python tools/lint_diff.py here I don't see those errors.  FWIW, `flake8 --version' gives
$ flake8 --version
4.0.1 (mccabe: 0.6.1, pycodestyle: 2.8.0, pyflakes: 2.4.0) CPython 3.8.8 on Linux

Thanks for the review @tupui !  I think I've addressed all comments except the assert, and I've fixed the lint errors.

I'll fix the lint errors too, though if I run python tools/lint_diff.py here I don't see those errors. FWIW, `flake8 --version' gives
$ flake8 --version
4.0.1 (mccabe: 0.6.1, pycodestyle: 2.8.0, pyflakes: 2.4.0) CPython 3.8.8 on Linux

Thanks for the update. I will let code owners have a look.
For the linter, we have flake8==3.9.2 on Python 3.9. Maybe that can explain the differences you observed (I did not have a look).

@roryyorke can you resolve the conflict here?

I fixed the conflict using the GitHub UI, feel free to force push over it if you want to take care of it differently @roryyorke

Thanks @roryyorke !

Thanks @larsoner.  I guess Github doesn't send mail notification when it detects a conflict, and I wasn't aware of it.
There are a few duplicate issues of gh-5668, I'll try to add comments to them this weekend.


Add slot component with fallback capability

eng-7087
https://github.com/user-attachments/assets/b209e72a-df59-44cc-bec3-f49038dce6b2

eng-7087
Screen.Recording.2024-12-16.at.4.12.55.PM.mov

Are we able to just pass null for the empty fallback? Or better yet, omit the fallback prop altogether?

Yeah - updated the recording, thanks

Are we able to just pass null for the empty fallback? Or better yet, omit the fallback prop altogether?

So right now, this means the developer has to pass a fallback or nothing shows? I would personally expect to see the default box if I don't pass a fallback at all. I think that might mess with the conditional checkbox logic a bit though?

updated the changeset message + fixed prettier formatting


Fix "find"-command in xspec.bat

XSpec failed with message:
find: ÔÇÿdÔÇÖ: No such file or directory
ERROR: XSPEC_HOME is not a directory

It seems the "find"-command is supposed to look for a letter "d" in file properties.
correct command for this task seems to be "findstr".
Line 281 changed this way things work

find on Windows is a standard command to look for strings. (The executable is C:\Windows\System32\find.exe.)
You probably have another find command in your PATH environment variable before C:\Windows\System32. Try where find to find out what.

in cmd.exe I only get
C:\Users\pagina>where find
C:\Windows\System32\find.exe

but you're right – I launched xspec.pat form a bash-terminal in Tower and there I get:
Heino Schmull@pt-823 MINGW64 ~/parsx4 (dev/version-4.0_xsd_alpha3)
$ where find
C:\Program Files (x86)\fournova\Tower\vendor\Git\usr\bin\find.exe
C:\Windows\System32\find.exe

the first one here is actually looking for files. So wouldn't it be better to use findstr in Windows to avoid confusion?

find is a standard command for more than 30 years. If you place another find before the standard one, sooner or later you'll jump into a lot of problems.
findstr is not a superset of find. They does different jobs like this.
In general findstr is a bit more tricky and bug-prone.

But it's also true that some people want to work in a *nix-like mixed environment despite its inherent compatibility problem.
So I think a more preferable way to work around the problem is not to replace find with findstr but to replace find with %SYSTEMROOT%\system32\find. What do you think?

I launched xspec.bat from a bash-terminal in Tower

@hschmull: just out of interest: if you are using a bash terminal, have you tried to use the xspec.sh script instead?

@cirulls the problem is that we run the xspec.bat as soon as we encounter OS=Windows. As we're running the XSpec tests for this project from a gradle wrapper script we can only rely on the OS information and switch to the xspec.sh as soon as we are not on OS=Windows. However, when launching the terminal from GitTower this is the default git bash which seems to be a special case as it supports bash functionality on windows but also inherits the windows path and the windows exe cli tools all together. A bit puzzling...
I bet the xspec.bat works very well when running from a normal Windows CMD.
If @hschmull confirms that the proposed fix by @AirQuick works, this should be a good solution for now.

If @hschmull confirms that the proposed fix by @AirQuick works, this should be a good solution for now.
%SYSTEMROOT%\system32\find "d" works fine in bash terminal and cmd als well.

@AirQuick @cirulls @tofi86 thanks and sorry for the delay

I'm closing this issue/pr as the answer has been accepted.

Okay. Are you going to open a new PR for this or fix it right away on master?

Sorry guys, I think I read this too quickly and misunderstood the outcome. @hschmull : feel free to update the pull request with the suggested solution. @AirQuick: I'll leave the review and merge to you, just ping me if/when it's merged so that I can update the draft release notes.

@cirulls
Once Chocolatey gets back to normal, could you restart the AppVeyor job for this PR and merge it if green?

I restarted the AppVeyor job and tests are green.
@hschmull, @tofi86 and @AirQuick: I'm merging this into master.

@cirulls @AirQuick thank you 👍


Search: Implement keyboard navigation

Currently, it is not possible to navigate through the search results using the up and down arrow keys.

This issue is currently blocked by https://github.com/AdvisorySG/dawn-advisory-theme/issues/51.

As https://github.com/AdvisorySG/dawn-advisory-theme/issues/51 has been resolved, this issue is no longer blocked.


Store jwt in cookies?

Is it possible to store the jwt in cookies (with httponly and secure both true), to prevent Cross Site Scripting (XSS) attacks?

Yep! It's important to understand the tradeoffs, but that is one approach I've worked on in the Stormpath libraries that I maintain. There's some discussion on how to achieve this in the article: https://stormpath.com/blog/token-authentication-asp-net-core#validating-tokens-in-asp-net-core

Hi,
is it possible to use both Cookies AND Headers for Authentication?

@ayhamclickandgrow Sure, it's possible. Whether the token gets to your application via an HTTP or via cookies (which are just a special case of HTTP headers), you can handle the incoming request appropriately. However, see my previous message about the tradeoffs. It's important to understand how to keep tokens secure.


Does not parse links for Google feedburner feeds

I'm trying to parse a few feedburner feeds, for example:
http://feeds.feedburner.com/blogspot/lQlzL
The link that is returned is of the: type="application/atom+xml".  I'm not able to target the link to go to the type="text/html".   The RSS feed also has feedburner:origLinkhttp://googleadsdeveloper.blogspot.com/2016/06/register-now-for-fall-2016-adwords-api.html/feedburner:origLink which would work as well for the link.  This may be too specific of an issue, but still thought I'd try.

Sorry, not sure I understand the issue here. I added a test case for the feedburner link above (by downloading the XML to a local file) and it seems to parse fine.  Why do you need type="text/html"?

When I used your library on a feed like: http://feeds.feedburner.com/blogspot/lQlzL, there are multiple link types.  When I used your library, the link that went with the titles led to pages like:
https://www.blogger.com/feeds/7815614485808579332/posts/default/8394866751819460570
vs.
http://googleadsdeveloper.blogspot.com/2016/06/adwords-and-dfp-java-client-library.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+blogspot%2FlQlzL+(Google+Ads+Developer+Blog+-+AdWords+API)&utm_content=FeedBurner
However, I ended up using something else to get around this issue.

Marking as closed, I think with customFields this should be fixed.


New config object fail: error loading source file: open config/settings: The system cannot find the file specified.

Describe the bug
Version: 2.0.6
.\go-admin.exe migrate -c=config/settings.dev.yml
New config object fail: error loading source file: open config/settings: The system cannot find the file specified.

To Reproduce
Steps to reproduce the behavior:

go build
copy config/settings.yml to config/settings.dev.yml
.\go-admin.exe migrate -c=config/settings.dev.yml

Expected behavior
init success
Desktop (please complete the following information):

OS: Windows 10

.\go-admin.exe migrate -c config/settings.dev.yml
this work, need remove "=".
I apply a PR https://github.com/go-admin-team/go-admin-doc/pull/8


goog.crypt.base64.decodeStringToByteArray adds null bytes if length isn't a multiple of 4.

This issue was imported from Closure Library's previous home at http://closure-library.googlecode.com
The link to the original issue is:
https://code.google.com/p/closure-library/issues/detail?id=626

Fixed in 1a6a0acd85042859d63802976ad0ace352e8055f


Better process icon support

When an icon is not mapped, it will be empty. Is it possible to display a default icon or obtain the icon from the icon of the executable file?

Just like this.
https://stackoverflow.com/questions/77692520/how-to-extract-an-icon-from-an-exe-file

The link you provided is unfortunately not applicable for this, to get the icon the client is matching against the process/title of the window, glazewm does not provide us with any icon info per se.
The way it is currently setup it defaults to ti-brand-${process-name}. This is since a lot of popular apps have a matching brand icon and process name (eg. discord, spotify, firefox etc.). However I have been thinking about this some as well. And I think a nice default icon would be good.


elfeed-show.el: fix yank to clipboard in OSX

While  sexp ''(kill-new link)" is present in function elfeed-search-yank, it is missing from elfeed-show-yank. Pressing y in entry window has not been adding the article URL to the clipboard in OS X. Adding this sexp fixes it.

Nice catch, thanks!


Report publications associated with projects on the frontend

Please label your issue with a label to specify its priority: breaking, high, medium, or low.
This is something we should do as soon as we can get to it in my opinion. I thought it was tracked somewhere, but I can't find the issue.
Context
There's at least one dataset that has a publication associated with it in the project metadata.
Problem or idea
We should expose this information on the frontend. Specifically, I'd expect to see it on the project details page.
Edit on 12/11/22 We should support multiple publications per project.
Solution or next step
Tagging @dvenprasad for input

Add 2 fields on view experiment page
DOI: 10.1016/j.devcel.2022.04.003
Citation: Patel A. G., X. Chen, X. Huang, M. R. Clay, N. Komorova, et al., 2022 The myogenesis program drives clonal selection and drug resistance in rhabdomyosarcoma. Dev Cell 57: 1226-1240.e8. https://doi.org/10.1016/j.devcel.2022.04.003

@dvenprasad should the DOI show up as a hyperlink in the citation as well? Or only from the citation field in the list?

Yes, please hyperlink the DOI field too


Export-NavContainerObjects : Unexpected error message in OnPrem

Hi,
I've exported reports from my Docker container based on OnPrem image using the following syntax:
Export-NavContainerObjects -containerName BCOnPremES01 -objectsFolder C:\ProgramData\NavContainerHelper -filter Type=report -exportTo "txt folder (new syntax)"
Export seems to work fine (the corresponding *.txt are placed in the folder), but I just wanted to let you know that PowerShell outputs an error message that seems Cloud related:

Also, I'd to ask a generic rookie question:
Some cmdlets allow for a string of parameters, but I've been unable to figure out the right syntax when I want to specify more than one parameter...
In the example above, if I'd like to export a certain object (filter by Type and ID), what would be the correct PS command line syntax?
(as compared to:
finsql.exe Command=ExportToNewSyntax, File=”C:\Users\Tester\Desktop\Extensions\CAL\export.txt”, Database=”Demo Database NAV (11-0)”, ServerName=.\NAVDEMO, Filter=Type=report;ID=50100)
I've tried with
Export-NavContainerObjects -containerName BCOnPremES01 -objectsFolder C:\ProgramData\NavContainerHelper -filter Type=page;ID=21 -exportTo "txt folder (new syntax)"
but it ignores the ID filter (exports all pages).
Thanks,
Eyrún

You're absolutely right on both issues - thanks a million!!