Abort on restart after maximum resets

Describe the bug
After exceeding the maximum number of unplanned resets allowed per CFE_PLATFORM_ES_MAX_PROCESSOR_RESETS, the system attempts to perform a POR instead of a PROCESSOR reset. Unfortunately this orderly reset fails due to an apparent deadlock and the system eventually times out and calls Abort.
Note that this does not occur when using CFE_ES_ResetCFE, only with CFE_PSP_Restart(CFE_PSP_RST_TYPE_PROCESSOR).
To Reproduce
Steps to reproduce the behavior:
Modify any app to call CFE_PSP_Restart(CFE_PSP_RST_TYPE_PROCESSOR) on command

I used sample app's noop command
Issue the command to trigger the restart and then re-spawn the executable
Repeat the restart until the system falls back to a POR reset

Expected behavior
Expect a clean POR restart without the 10 second timeout and abort
Code snips
System observed on:

Hardware
Intel i7-10870H
64 GB RAM
OS: [e.g. Linux 4.4]
Linux -------- 5.18.10-76051810-generic #202207071639~1657252310~21.10~7d5e891 SMP PREEMPT_DYNAMIC Fri J x86_64 x86_64 x86_64 GNU/Linux
Versions [e.g. cFE 6.6, OSAL 4.2, PSP 1.3 for mcp750, any related apps]
Latest cFS distribution as of July 28, 2022, modification to sample_app to call CFE_PSP_Restart.

Additional context
Stack Trace from running threads at the time of the abort
(gdb) r
Starting program: /home/ldmiller/cFS/build/exe/cpu1/core-cpu1
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
OS_BSP_Initialize():Maximum user msg queue depth = 100
CFE_PSP: Default Reset SubType = 1
CFE_PSP: Default CPU ID = 1
CFE_PSP: Default Spacecraft ID = 66
CFE_PSP: Default CPU Name: cpu1
OS_Posix_GetSchedulerParams():187:Policy 1: available, min-max: 1-99
OS_Posix_GetSchedulerParams():187:Policy 2: available, min-max: 1-99
OS_Posix_TaskAPI_Impl_Init():373:Selected policy 2 for RT tasks, root task = 99
OS_Posix_TaskAPI_Impl_Init():390:Could not setschedparam in main thread: Operation not permitted (1)
[New Thread 0x7ffff7fc1640 (LWP 451164)]
[New Thread 0x7ffff7fbc640 (LWP 451165)]
CFE_PSP: Instantiated software timebase 'cFS-Master' running at 10000 usec
CFE_PSP: Using POSIX monotonic clock as CFE timebase
CFE_PSP: Using MMAP simulated EEPROM implementation
CFE_PSP: Physical RAM access not implemented
CFE_PSP: I/O Port access not implemented
CFE_PSP: EEPROM Range (2) created: Start Address = 7FFFF7B80000, Size = 00080000 Status = 0
CFE_PSP: Normal exit from previous cFE instance
CFE_PSP: Starting the cFE with a PROCESSOR reset.
1980-003-11:57:59.61581 CFE_ES_SetupResetVariables: POWER ON RESET due to max proc resets (Watchdog).
CFE_PSP: Exiting cFE with POWERON Reset status.
CFE_PSP: Critical Data Store Shared memory segment removed
Reset Area Shared memory segment removed
User Reserved Area Shared memory segment removed

Thread 1 "core-cpu1" received signal SIGABRT, Aborted.
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737353758528) at pthread_kill.c:44
44      pthread_kill.c: No such file or directory.
(gdb) info threads
  Id   Target Id                                      Frame
* 1    Thread 0x7ffff7fa3740 (LWP 451163) "core-cpu1" __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737353758528) at pthread_kill.c:44
  2    Thread 0x7ffff7fc1640 (LWP 451164) "core-cpu1" __futex_abstimed_wait_common64 (private=<optimized out>, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x5555556ec640 <OS_impl_console_table>) at futex-internal.c:57
  3    Thread 0x7ffff7fbc640 (LWP 451165) "core-cpu1" 0x00007ffff7c4323a in __GI___sigtimedwait (set=set@entry=0x5555556ed658 <OS_impl_timebase_table+280>, info=info@entry=0x7ffff7fbbc80, timeout=timeout@entry=0x0)
    at ../sysdeps/unix/sysv/linux/sigtimedwait.c:61
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737353758528) at pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737353758528) at pthread_kill.c:80
#2  __GI___pthread_kill (threadid=140737353758528, signo=signo@entry=6) at pthread_kill.c:91
#3  0x00007ffff7c42476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7c287b7 in __GI_abort () at abort.c:79
#5  0x00005555555a2f93 in CFE_PSP_Restart (reset_type=2) at /home/ldmiller/cFS/psp/fsw/pc-linux/src/cfe_psp_support.c:114
#6  0x00005555555730d9 in CFE_ES_SetupResetVariables (StartType=1, StartSubtype=1, BootSource=1) at /home/ldmiller/cFS/cfe/modules/es/fsw/src/cfe_es_start.c:407
#7  0x0000555555572bdc in CFE_ES_Main (StartType=1, StartSubtype=1, ModeId=1, StartFilePath=0x5555555a6271 "/cf/cfe_es_startup.scr") at /home/ldmiller/cFS/cfe/modules/es/fsw/src/cfe_es_start.c:121
#8  0x00005555555a2a84 in OS_Application_Startup () at /home/ldmiller/cFS/psp/fsw/pc-linux/src/cfe_psp_start.c:432
#9  0x00005555555a517e in main (argc=1, argv=0x7fffffffdeb8) at /home/ldmiller/cFS/osal/src/bsp/generic-linux/src/bsp_start.c:238
(gdb) thread 2
[Switching to thread 2 (Thread 0x7ffff7fc1640 (LWP 451164))]
#0  __futex_abstimed_wait_common64 (private=<optimized out>, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x5555556ec640 <OS_impl_console_table>) at futex-internal.c:57
57      futex-internal.c: No such file or directory.
(gdb) bt
#0  __futex_abstimed_wait_common64 (private=<optimized out>, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x5555556ec640 <OS_impl_console_table>) at futex-internal.c:57
#1  __futex_abstimed_wait_common (cancel=true, private=<optimized out>, abstime=0x0, clockid=0, expected=0, futex_word=0x5555556ec640 <OS_impl_console_table>) at futex-internal.c:87
#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x5555556ec640 <OS_impl_console_table>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=<optimized out>)
    at futex-internal.c:139
#3  0x00007ffff7c9ca3f in do_futex_wait (sem=sem@entry=0x5555556ec640 <OS_impl_console_table>, abstime=0x0, clockid=0) at sem_waitcommon.c:112
#4  0x00007ffff7c9cad8 in __new_sem_wait_slow64 (sem=0x5555556ec640 <OS_impl_console_table>, abstime=0x0, clockid=0) at sem_waitcommon.c:184
#5  0x000055555559abbc in OS_ConsoleTask_Entry (arg=0xc0001) at /home/ldmiller/cFS/osal/src/os/posix/src/os-impl-console.c:97
#6  0x00007ffff7c94947 in start_thread (arg=<optimized out>) at pthread_create.c:435
#7  0x00007ffff7d24a44 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100
(gdb) thread 3
[Switching to thread 3 (Thread 0x7ffff7fbc640 (LWP 451165))]
#0  0x00007ffff7c4323a in __GI___sigtimedwait (set=set@entry=0x5555556ed658 <OS_impl_timebase_table+280>, info=info@entry=0x7ffff7fbbc80, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:61
61      ../sysdeps/unix/sysv/linux/sigtimedwait.c: No such file or directory.
(gdb) bt
#0  0x00007ffff7c4323a in __GI___sigtimedwait (set=set@entry=0x5555556ed658 <OS_impl_timebase_table+280>, info=info@entry=0x7ffff7fbbc80, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:61
#1  0x00007ffff7c428ac in __GI___sigwait (set=0x5555556ed658 <OS_impl_timebase_table+280>, sig=0x7ffff7fbbd44) at ../sysdeps/unix/sysv/linux/sigwait.c:28
#2  0x000055555559d7f5 in OS_TimeBase_SigWaitImpl (obj_id=524289) at /home/ldmiller/cFS/osal/src/os/posix/src/os-impl-timebase.c:159
#3  0x0000555555598e8f in OS_TimeBase_CallbackThread (timebase_id=524289) at /home/ldmiller/cFS/osal/src/os/shared/src/osapi-timebase.c:422
#4  0x000055555559db03 in OS_TimeBasePthreadEntry (arg=0x80001) at /home/ldmiller/cFS/osal/src/os/posix/src/os-impl-timebase.c:319
#5  0x00007ffff7c94947 in start_thread (arg=<optimized out>) at pthread_create.c:435
#6  0x00007ffff7d24a44 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100
(gdb)

Reporter Info
Lorn Miller
Red Canyon Engineering & Software

Response from cfs-community mailing list:

This looks like a bug to me - I'd recommend documenting on github as a CFE issue.
CFE_PSP_Restart isn't supposed to be called from the context of the init thread (i.e. that which is running CFE_ES_Main) as it relies on that thread having reached the idle loop.  It is only supposed to be called from the spawned tasks.
Joe Hickey

I was able to confirm this one fairly simply (no code change required!):

start CFE normally (PO mode, default)
CTRL+C (note message about reset count 1/2, shutting down for PROCESSOR restart)
start CFE with --reset PR option (this agrees with last shutdown)
CTRL+C again (note message about reset count 2/2, shutting down for POWERON restart)
start CFE again with --reset PR option (this does NOT agree with last shutdown)
Observe startup failure with abort described here

The first order problem here is that CFE was started in processor reset mode, which did NOT concur with the boot record which requested a poweron reset.  This is simply because it was launched with --reset PR and the PSP actually obeyed that request even though the reset area (aka boot record) said it was due for a poweron reset.  As a result, the CFE tried to restart itself again via CFE_PSP_Restart().
This seems like a questionable design decision -- somewhat like the age-old idiom of trying the exact same thing over again but expecting a different result.  The PSP is did not adhere to the restart type, why would a different result be anticipated by calling CFE_PSP_Restart() again?  Seems like a recipe for a boot loop and a non-recoverable system.
My recommendation is that this attempt to restart again should be removed.  The fact is that CFE is running, just with the wrong reset type.  That is arguably better (and more recoverable by an operator) than a system that has gotten into a boot loop and fails to start at all.  Instead, it should just be noted through event reporting that the system started with the wrong reset type.

If we end up in a processor reset loop, perhaps due to some third party "fault protection" type application, I would like to see something make that POR attempt. Perhaps this decision should be deferred to that same third party application, but it was my understanding that CFE_PLATFORM_ES_MAX_PROCESSOR_RESETS was intended to manage this particular functionality.

Works for me.


Cannot uninstall real-vnc 6.0.2

General troubleshooting steps

[X] I have checked the instructions for reporting bugs (or making requests) before opening the issue.

[X] None of the templates was appropriate for my issue, or I’m not sure.

[X] I ran brew update-reset && brew update and retried my command.
[X] I ran brew doctor, fixed as many issues as possible and retried my command.
[X] I understand that if I ignore these instructions, my issue may be closed without review.

Description of issue
Seems like uninstall_preflight is no longer run before uninstall, likely due to this change:
https://github.com/Homebrew/brew/blob/master/Library/Homebrew/cask/lib/hbc/installer.rb#L323
For real-vnc, this means that the uninstall scripts no longer run before uninstall and they then don't exist to be run causing the failure. The intention with this package was to let the uninstall scripts run and then do post-cleanup in case anything was missed.
Is this a bug, or should the cask be refactored?
Output of your command with --verbose --debug
$ brew cask uninstall real-vnc.rb --verbose --debug
==> Uninstalling Cask real-vnc.rb
==> Hbc::Installer#uninstall
==> Un-installing artifacts
==> Determining which artifacts are present in Cask real-vnc
==> 4 artifact/s defined
#<Hbc::Artifact::Uninstall:0x007fa5fc008788>
#<Hbc::Artifact::PreflightBlock:0x007fa5fc008878>
#<Hbc::Artifact::Pkg:0x007fa5fc008800>
#<Hbc::Artifact::Zap:0x007fa5fc008710>
==> Un-installing artifact of class Hbc::Artifact::Uninstall
==> Running uninstall process for real-vnc; your password may be necessary
==> Removing launchctl service com.realvnc.vncserver
==> Executing: ["/bin/launchctl", "list", "com.realvnc.vncserver"]
==> Executing: ["/usr/bin/sudo", "-E", "--", "/bin/launchctl", "list", "com.realvnc.vncserver"]
==> Removing launchctl service com.realvnc.vncserver.peruser
==> Executing: ["/bin/launchctl", "list", "com.realvnc.vncserver.peruser"]
==> Executing: ["/usr/bin/sudo", "-E", "--", "/bin/launchctl", "list", "com.realvnc.vncserver.peruser"]
==> Uninstalling packages:
==> Executing: ["/usr/sbin/pkgutil", "--pkgs=com.realvnc.vncserver.pkg"]
==> Executing: ["/usr/sbin/pkgutil", "--pkgs=com.realvnc.vncviewer.pkg"]
==> Un-installing artifact of class Hbc::Artifact::PreflightBlock
==> Executing: ["/usr/bin/sudo", "-E", "--", "/Applications/RealVNC/Uninstall VNC Server.app/Contents/Resources/uninstaller.sh"]
Error: Command failed to execute!

==> Failed command:
/usr/bin/sudo -E -- /Applications/RealVNC/Uninstall VNC Server.app/Contents/Resources/uninstaller.sh

==> Standard Output of failed command:

==> Standard Error of failed command:
sudo: /Applications/RealVNC/Uninstall VNC Server.app/Contents/Resources/uninstaller.sh: command not found

==> Exit status of failed command:
#<Process::Status: pid 5876 exit 1>
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/system_command.rb:65:in `assert_success'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/system_command.rb:36:in `run!'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/system_command.rb:14:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/system_command.rb:18:in `run!'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/dsl/base.rb:14:in `system_command'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cask_loader.rb:20:in `block (2 levels) in load'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/artifact/abstract_flight_block.rb:35:in `instance_eval'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/artifact/abstract_flight_block.rb:35:in `block in abstract_phase'
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/set.rb:232:in `each_key'
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/set.rb:232:in `each'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/artifact/abstract_flight_block.rb:34:in `abstract_phase'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/artifact/abstract_flight_block.rb:28:in `uninstall_phase'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/installer.rb:330:in `block in uninstall_artifacts'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/installer.rb:327:in `each'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/installer.rb:327:in `uninstall_artifacts'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/installer.rb:312:in `uninstall'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/uninstall.rb:26:in `block in run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/uninstall.rb:9:in `each'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli/uninstall.rb:9:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:115:in `run_command'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:158:in `process'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:8:in `cask'
/usr/local/Homebrew/Library/Homebrew/brew.rb:91:in `<main>'
Error: Kernel.exit
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:163:in `exit'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:163:in `rescue in process'
/usr/local/Homebrew/Library/Homebrew/cask/lib/hbc/cli.rb:149:in `process'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:8:in `cask'
/usr/local/Homebrew/Library/Homebrew/brew.rb:91:in `<main>'

Output of brew cask doctor
$ brew cask doctor
==> Homebrew-Cask Version
Homebrew-Cask 1.1.13
caskroom/homebrew-cask (git revision 1e100; last commit 2017-04-20)
==> Homebrew-Cask Install Location
<NONE>
==> Homebrew-Cask Staging Location
/usr/local/Caskroom
==> Homebrew-Cask Cached Downloads
~/Library/Caches/Homebrew/Cask (3 files, 99.2MB)
==> Homebrew-Cask Taps:
/usr/local/Homebrew/Library/Taps/caskroom/homebrew-cask (3634 casks)
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core (0 casks)
==> Contents of $LOAD_PATH
/usr/local/Homebrew/Library/Homebrew/cask/lib
/usr/local/Homebrew/Library/Homebrew
/Library/Ruby/Site/2.0.0
/Library/Ruby/Site/2.0.0/x86_64-darwin16
/Library/Ruby/Site/2.0.0/universal-darwin16
/Library/Ruby/Site
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/vendor_ruby/2.0.0
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/vendor_ruby/2.0.0/x86_64-darwin16
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/vendor_ruby/2.0.0/universal-darwin16
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/vendor_ruby
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/x86_64-darwin16
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/universal-darwin16
==> Environment Variables
LC_ALL="en_US.UTF-8"
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/Homebrew/Library/Homebrew/shims/scm"
SHELL="/bin/bash"

Oops. Just noticed #32300.
Closing.


Dart2JS : WebComponents performance issues (very serious)

From @LSMetag on June 30, 2015 11:47
Hi,
From a long time, running my web project (based on Polymer Dart) as Javascript caused the Polymer components to load extremely slowly (1 second with Dart, 5 to 10 seconds in JS).
Using firebug, I got this message :
"mutating the [[Prototype]] of an object will cause your code to run very slowly; instead create the object with the correct initial [[Prototype]] value using Object.create webcomponents.js:6007:6"
Did you think about this ? I indeed plan to further optimize that, but If Dart2JS itself is a cause, It can be a real problem.
Here is the project repository : https://github.com/LSMetag/CVWebkit/, that can be a good testing.
The polymer components are in WebApp/Web/ZoneRuban.html and WebApp/Lib/zoneRuban.dart.
The "zoneruban" macro-component tag is invoked in WebApp/Web/CVWebkit.html.
config.yaml is the source file that is read to create layout and use components.
It would be a great help too if you could advise me about performance improving, if I am too at fault.
Thanks in advance !
Copied from original issue: dart-lang/sdk#23752

From @floitschG on June 30, 2015 13:31
/cc @sigmundch
According to my tests, setting up an inheritance chain using "proto" is the fastest, and V8 doesn't seem to mind.
Since IE doesn't support changing the prototype we already have a fallback in place. We could use that one, when detecting FF.
In this case the proto change seems to come from webcomponents.js which isn't a dart2js produced file (afaik). So maybe this is a polymer issue and not dart2js related.

From @LSMetag on June 30, 2015 13:53
It's related to polymer, it's certain, because the performance issue is only on polymer components. And indeed, I tested on Firefox. I just tested in Chrome and I don't have this issue.
I thought It used VM but in fact no. The problem is linked to polymer, after being converted to JS, and using a browser other than Chrome.
But why FF doesn't like "proto", while it's the opposite for V8 ?

@jakemac53
Jake has been working on a newer port of polymer.js, which should address many performance issues. I am not sure if part of that includes fixes to webcomponents.js to avoid this performance problem, but I know that the new architecture tries to avoid using some web-component features (for example, shadow-dom) if they seem to be too much of a performance bottleneck.
Jake can tell you a bit more, but my understanding is that load times are much better now.

From @jakemac53 on June 30, 2015 16:52
You can try out https://github.com/dart-lang/sample-todomvc-polymer/tree/polymer-0.17 to see the new version. In non-chrome browsers especially it is much faster because of the lack of shadow dom and a few other things.
I don't know specifically if firefox is still giving that warning with the lite version of web components.

Can you set this version as "dev" for pub ? I don't know how to integrate it in my project to test and give a feedback.
Or you can test it with my project, to be sure, before closing this.


Weird at-command completion behaviour

v3.1.108 (pre-release)
https://github.com/smallcloudai/refact-chat-js/assets/43416739/8cd29bb5-3f4d-4b5e-a015-b3505dc7518a

duplicates in at-command completion output are already reported in https://github.com/smallcloudai/refact-lsp/issues/151

Yes, I was able to recreate the issue when duplicates where in the combobox.
Here's a quick fix https://github.com/smallcloudai/refact-chat-js/pull/10/commits/c0f7e4d2feee99f6afc6450d69e2a210ff88770b

There's still that weird behaviour
https://github.com/smallcloudai/refact-chat-js/assets/43416739/1b8b8f7f-8eec-49d0-9aa5-b81f9820b826


Update version for the next release (v0.9.0)

This version makes this package compatible with Meilisearch v0.29.0 :tada:
Check out the changelog of Meilisearch v0.29.0 for more information on the changes.
🚀 Enhancements

New Meilisearch option matchingStrategy #832

⚠ Breaking Changes
This breaking change may not affect you, but in any case, you should check your search queries if you want to keep the same behavior from v0.28.

The NOT filter keyword does not have an implicitly EXIST operator anymore. Check out for more information: https://github.com/meilisearch/meilisearch/issues/2486

Thanks again for contributing with Meilisearch :heart:
If you are participating in Hacktoberfest, and you would like to receive a small gift from Meilisearch too, don't forget to fill this form.


Fortran support

There are some port requests on the tracker that require a Fortran compiler to compile the respective libraries (#23, #233). I want to create this issue to start a discussion on how Fortran support could be brought to vcpkg.
Since Visual Studio does not come with a Fortran compiler, on Windows one usually uses one of the following

Intel Fortran (ifort)
gfortran through MinGW/MSYS

Intel Fortran is proprietary and thus may not be available for everyone but it has a nice Visual Studio integration which makes it pretty straight forward to consume libraries compiled with ifort from within Visual Studio.
gfortran on the other hand is free but does not have an integration into Visual Studio. In general one uses MinGW/MSYS to compile fortran libraries for windows when using gfortran. This comes with some implications:

one can not use the Visual Studio generator in CMake but instead has to use some makefile system that is supported by MSYS (GNU Makefiles in general). As far as I know there is a special branch of ninja with fortran support maintained by kitware (https://github.com/Kitware/ninja) but I have no experience in using it and do not know how far the implementation is.
The compiled libraries will be linked against the gcc and gfortran runtime libraries provided by MinGW.
When compiling projects that include both Fortran and C or C++ code gcc/g++ is used to compile the C/C++ code instead of Visual C++.

So the first question would be which version should implemented in vcpkg:

Intel Fortran (in my opinion more easy to implement)
gfortran (may be preferred since it is free)
both (choice is always the best solution. Triplets could be used to determine which toolset to use)

I do not have a lot of experience in using gfortran on windows but I do have a port for netlib blas+lapack (http://www.netlib.org/lapack/) which works with some minor additional modifications to vcpkg itself when using Intel Fortran.

As discussed in https://github.com/Microsoft/vcpkg/issues/233, at least initially I am for using Intel Fortran. However, I fear that requiring all the vcpkg users that want to install Fortran-dependent ports to buy the Intel Fortran compiler is a too much limiting. A possible workaround is to upload the binaries produced using Intel Fortran to a website and then let the port download them in binary form, as it is already done for ports such as tbb.
Perhaps Microsoft could provide some support for hosting such binaries produced by the community? @ras0219-msft @alexkaratarakis

More in the long term, it would be interesting to see how the flang project  for integrating a Fortan compiled in  LLVM evolves. It was announced one year ago by PGI/Nvidia and it still not open source, but it is in progress. [1]
Considering LLVM compatibility with Visual Studio ABI, perhaps in a few years it could be a viable solution for vcpkg. : )
[1] : https://developer.arm.com/-/media/developer/developers/hpc/files/ARM-HPC-UG-SC16-PGI.pdf

On the use of gfortran: checking Lapack documentation on how to build it on Windows I discovered of the GNUtoMS CMake property, described in this article regarding the improvement to Fortran support in CMake.

Requiring Intel Fortran for a complete out-of-the-box experience is definitely not desirable considering its licensing. However, it does look like it's a very popular (the most popular?) solution on Windows, so we should support it. This is exactly what triplets should do -- specify the toolchain and build parameters to be used for all built libraries.
We can totally handle tools that don't have VS IDE integration; we could easily download Ninja or use MSYS internally as long as we can produce crt-compatible libraries out the other side.
As for hosting prebuilt binaries -- assuming it is fine with the intel fortran license (I'm not too familiar on this point), we could very easily host them inside the NuGet gallery, treating the NuGet package as a simple zip archive of the prebuilt binaries. We even already have the capability to download the NuGet.exe tool (for the MSBuild per-project integration).
As has been mentioned many times elsewhere, building from source would be preferable for users who can (i.e. that have intel fortran), but it's better to have binaries than nothing at all -- as long as the fortran library doesn't have dependencies. We don't want the fact that library X required a binary drop to obstruct the maintainer of dependency Y from updating.

The only thing I had to change to make it work with the Intel Compiler was to remove the extra arguments that are passed to "cmake --build" in vcpkg_install_cmake and vcpkg_build_cmake. This is because when the Visual Studio project files include Fortran code, "cmake --build" will invoke devenv.exe instead of msbuild.exe which will not understand the extra arguments.
Maybe an additional parameter that is passed to vcpkg_install_cmake and vcpkg_build_cmake to disable all extra arguments can be introduced (just like the one that has recently been introduced to disable the /m switch only).
The Intel compiler always got detected correctly by cmake automatically. I don't know whether this is the case under all circumstances (e.g. if you have multiple versions installed) but one could always set "CMAKE_Fortran_COMPILER" in the triplet. The problem would here would be to auto-detect and choose the correct paths to ifort.exe.

As for hosting prebuilt binaries -- assuming it is fine with the intel fortran license (I'm not too familiar on this point), we could very easily host them inside the NuGet gallery, treating the NuGet package as a simple zip archive of the prebuilt binaries. We even already have the capability to download the NuGet.exe tool (for the MSBuild per-project integration).

Interesting read on this:
https://software.intel.com/en-us/forums/intel-visual-fortran-compiler-for-windows/topic/626497

I think that this approach https://github.com/Microsoft/vcpkg/issues/23#issuecomment-277437432 to packaging a fortran library such as LAPACK is relevant also here.

The preliminary version of the flang compiler (based on PGI frontend) was open sourced in https://github.com/flang-compiler/flang . I don't think it is ready for use in vcpkg (I doubt there are binary availables) but it is interesting to look at it.

Very interesting! As far as I can see there is no official announcement available yet?!
I will go on and compile everything and try how far I can get with it. If it is just the missing binaries we may will be able to host the binaries ourselves somewhere.

As far as I can see there is no official announcement available yet?!
Apparently no: http://www.phoronix.com/scan.php?page=news_item&px=LLVM-NVIDIA-Fortran-Flang .

Would be curious to hear about your experiences with Flang. We (at conda-forge) have been eyeing progress on it for some time (though it was only recently open sourced). In particular, we were looking at it for building things like OpenBLAS and SciPy, but inevitably including Fortran support for other things.

Relevant flang issue on Windows support: https://github.com/flang-compiler/flang/issues/143 .

Good news regarding support for ucrtbase.dll support in mingw-w64, if I understood correctly, at least partial support for ucrtbase.dll has been merged in the master branch :

https://sourceforge.net/p/mingw-w64/mailman/mingw-w64-public/thread/1505134111-13637-2-git-send-email-martin%40martin.st/#msg36033731
https://sourceforge.net/p/mingw-w64/mingw-w64/ci/bc6a87488995675ae80c312e3585cb1ace739b43/log/?path=

I don't know if this directly affects the libraries build with gfortran, and when this will be available in a released version of mingw-w64 .

This sound like great news!
The PR at #1098 already includes MinGW+w64 support, but it requires the not yet released version of CMake 3.10 to support GnuToMS for the Ninja generator and of course has the known problems of linking against msvcrt.
I am really looking forward to a release version of MinGW that includes the final ucrt suppprt.

Ironically, using vcpkg to build flang would be the most time-efficient method to get a fortran compiler with MSVC ABI compatibility. But,  MSVC does not have OpenMP 4.0, which flang requires. So essentially, the steps would be:

Add clang support to vcpkg (clang has ABI compatibility with MSVC and supports OpenMP 4.0).
Fill in the required libraries (specifically, clang-flang, which is a modified clang)
Add openmp-llvm to the ports.
Use vcpkg to install the libraries and then build flang with MSVC.

Now whether this will actually happen is a different story MSFT might not want to add clang support to this project because then people may obviously not use their compiler. But based on their responses, there is little chance of MSVC adding OpenMP 4 support.
In other words, this is technically solvable, but when business interests / people come into the picture it might not be (and I have nothing against MSFT; I think it's a good company and I think this is an excellent project that was long overdue).

@xoviat have you tried to build flang for windows with the described method manually? And then use that flang binary to build any fortran code on windows? As far as I understood, flang does not yet support windows at all (not only being build by MSVC, but in general. Correct me if I am wrong!).

Flang supports windows but someone just needs to build it, which is not trivial for the same reason that vcpkg exists.

If that's true, why aren't there any updates in issue ( https://github.com/flang-compiler/flang/issues/143 )? Where has this work been occurring?

No one has built it yet but it's "just" CMake.

Actually, after reconsidering the problem, it's possible to override the generator with a PreLoad.cmake. If my theory is correct, then we don't actually need MSFT to modify the tool sources to build openmp-llvm. I think we have a viable path forward.
So to summarize, we need ports for:
-openmp-llvm

clang-flang

If someone else wants to port one of these, then I may port the other.

i don't think even ports are necessary. We could just precompile flang and treat it like one of the external tools, and then use it like that. Though I'm not sure if flang is actually buildable on Windows, even with msys2 and friends.

Though I'm not sure if flang is actually buildable on Windows, even with msys2 and friends.

Let me be clear: flang is buildable on windows, but it has a complex dependency chain, which is the exact problem that vcpkg was designed to solve. The only problem is  that one dependency is openmp-llvm, which MSVC can't build. However, it appears that the default generator can be overridden, making it buildable with once the above libraries are ported.

We could just precompile flang

Yes, but precompiling is the problem. The libraries aren't there yet. But they could be with vcpkg.

LLVM's openmp is definitely buildable on Windows. We build it in conda-forge currently.
ref: https://github.com/conda-forge/openmp-feedstock

With MSVC?

Update: looks like you are using clang as well. So essentially what I've already said.
@jakirkham Can conda install the headers and libs? If so, then the problem could be solved!

Assuming yes, we just need a port for clang-flang. That is completely doable.

PR: #2010.

@albertziegenhagel Thanks so much for doing that. I was given incorrect information apparently. What I'm going to do then is work on the POSIX API issue and then we can try again when the C code is compatible.

@xoviat, yes conda-forge package has the headers and libraries.

Update: looks like you are using clang as well. So essentially what I've already said.

I think it is just using vanilla MSVC. We only require llvmdev (static build of LLVM) be present, but we don't actually install clang for the build. We do require clang at run time though.

Interesting. My research on that topic was apparently not correct.

Actually, I have another wild idea that may be more practical than any of the current options. It is a bit complex if done manually but can be made to work seamlessly with a bit of vcpkg coordination.
Windows now has the "Windows Subsystem for Linux," which is essentially a way to run ubuntu applications inside windows. In addition, flang can target LLVM IR. So essentially what we need to do is install flang within the "Subsystem for Linux" and then we can have an executable file that simply does the following:

Takes the input file and copies it to a temporary location within the subsystem for Linux
Runs flang inside the subsystem for Linux and generates LLVM IR
Copies the LLVM IR back to the windows space and compiles it with LLVM

Sounds a bit tricky and somewhat complicated. But how difficult can it be to write a script that simply copies a few files and runs a few commands? And then maybe optimize that script by rewriting it in a lower-level language (like C++, which I know the vcpkg folks love).
@ras0219-msft What do you think of that idea? Sounds completely doable to me.

If you could make that work, that would be really cool! I'd be happy to merge support in that form :)

Update: we've built flang on win32. I expect there to be bugs but we need real-world testing to find them. I'm going to look at the existing fortran PR and look to update it.
Flang on win32 requires:

a development version of CMake (changes may not even be in master, but they will soon)
the following cmake configuration flags:

cmake -G "NMake Makefiles" -DCMAKE_Fortran_COMPILER=flang -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=Release

Also, thanks so much to @isuruf  and @albertziegenhagel who made this possible.

@xoviat, with https://github.com/isuruf/flang/commit/970f43bb7387a4f29b1b7a87cace277b64da55ec, I don't think we need development version. Latest is fine.

Okay.

Nice work! Is there a PR with the conda recipe somewhere?

I'm working on the conda recipe here, https://github.com/isuruf/staged-recipes/tree/flang2
This needs a custom clangdev package as well.

Will conda have a prebuilt binary package we can download? There's really no reason to build the Fortran compiler itself from source for us; only the Fortran libraries themselves.

you can do,
conda config --add channels conda-forge
conda config --add channels isuruf/label/flang
conda install flangdev

to download the compiler and the runtime libraries

@ras0219-msft Do you have any questions about how to download the compiler or need help creating a redistributable? What are the next steps and what do you need from us?

Is there a way to download the executables from your conda package as an archive file (without having to install conda)? If that would be possible I could very easily integrate flang as Fortran compiler in my PR #1098. Else I could upload my own builds of flang somewhere.
To make this work, we would need to finish the cmake integration at [1].
[1] https://gitlab.kitware.com/cmake/cmake/merge_requests/1430

Is there a way to download the executables from your conda package as an archive file (without having to install conda)?

The conda installer is just "resolve, download and spread." Of course, it's a .tar.bz2 format, but you can resolve dependencies using paper and pencil and then download the archives directly from the website. The spread part is just unpacking to different directories.

Here's the resolver solution for flang:
   clangdev:  4.0.0-flang_git_2b0d6a5_0 isuruf/label/flang [vc14]
    flangdev:  4.0.0.git.fd7f1c0-vc14_0  isuruf/label/flang [vc14]
    libiconv:  1.14-vc14_4               conda-forge        [vc14]
    libxml2:   2.9.5-vc14_0              conda-forge        [vc14]
    llvmdev:   4.0.0-default_0           conda-forge        [vc14]
    openmp:    4.0.0-vc14_0              conda-forge        [vc14]
    zlib:      1.2.11-vc14_0             conda-forge        [vc14]

Perfect, thank you!
.tar.bz2  should not be a problem. I will integrate this tomorrow and then try to build all the fortran ports I have with it.

I've implemented initial support for Flang in #1098 based on the binaries provided in the conda packages of @isuruf.
Currently one needs to make sure a CMake version that includes the patches from [1] is in the PATH before calling vcpkg.
I was able to build reference LAPACK (including BLAS) with that configuration. I did not check whether the resulting binaries do actually work correctly. I will try my other Fortran ports soon.
[1] https://gitlab.kitware.com/cmake/cmake/merge_requests/1430

@albertziegenhagel, we published the packages in conda-forge and cleaned up dependencies a bit. The solver resolution is now
clangdev:       5.0.0-flang_1 conda-forge [flang vc14]
flang:          5.0.0-vc14_1  conda-forge [vc14      ]
libflang:       5.0.0-vc14_1  conda-forge [vc14      ]
openmp:         5.0.0-vc14_0  conda-forge [vc14      ]

Sorry for the delay, but I've now updated the PR to download the new version from conda-forge.
Btw: I could build LAPACK and ScaLAPACK with flang, but when I try to compile MUMPS it fails with a message about a missing header, although on the first look it seems that the correct include path is set. If you want to try it yourself:

checkout the branch fortran-ports from https://github.com/albertziegenhagel/vcpkg/tree/fortran-ports
Add set(VCPKG_FORTRAN_COMPILER Flang) to the triplet file
Make sure a current CMake build is your PATH
Try vcpkg install mumps

Hi everyone, how is this PR evolving?
Is there any chance to see https://github.com/Microsoft/vcpkg/pull/1098 going through 😄?

Some interesting recent discussion related to this issue:

https://github.com/isuruf/flang/issues/129#issuecomment-465851596 : Recent discussion in the Python community on a Fortran UCRT-compatible, involving the same people behind the conda's Flang port (cc @rgommers that may be interested in this discussion)
http://lists.llvm.org/pipermail/llvm-dev/2019-February/130497.html : RFC for comment for moving the development of the new experimental f18 Fortran LLVM front-end to LLVM upstream. The front-end is not a currently able to generate code (see http://lists.llvm.org/pipermail/llvm-dev/2019-March/130694.html for an update on a current progress) but in the long-term it could be a good solution.
In the f18 llvm-dev thread, they posted a link to http://fort-compiler.org/, another Fortran front end for LLVM that I had never heard of.

http://lists.llvm.org/pipermail/llvm-dev/2019-February/130497.html : RFC for comment for moving the development of the new experimental f18 Fortran LLVM front-end to LLVM upstream. The front-end is not a currently able to generate code (see http://lists.llvm.org/pipermail/llvm-dev/2019-March/130694.html for an update on a current progress) but in the long-term it could be a good solution.

http://lists.llvm.org/pipermail/llvm-dev/2019-April/131703.html
The f18  Fortran LLVM front-end will be integrated in LLVM upstream.

@ghost: is the flang win32 version actually 32 bit or you mean x64? conda-forge has dropped support for Windows 32 bit so only 64 bit packages can be found. Have you produced a 32 bit version of flang? If so, it is published through a conda channel or available as a standalone tar?
I have tried the channel isuruf/label/flang but no luck.

@gmanlan, it's x64. flang doesn't support 32 bit version.

I note that there are ports using fortran at this time so the original issue filing appears to be resolved, but as you note above there are still some 'holes' in that support. You might want to consider opening issues for the specific CRT problems above to make it easier for folks familiar with this problem space to find. I'll leave this open for now though.

Since Intel OneAPI is freely available now and includes the intel fortran compiler ifort we should try to switch from gfortran to ifort if it is possible to install Intel OneAPI on the CI machines.  This should in theory cleanly resolve the CRT problems.

Since Intel OneAPI is freely available now and includes the intel fortran compiler ifort we should try to switch from gfortran to ifort if it is possible to install Intel OneAPI on the CI machines.  This should in theory cleanly resolve the CRT problems.

For what regards CRT problems, I recently noticed that msys2 now provides ucrt-based builds on top of the already existing MINGW64 builds: https://packages.msys2.org/package/mingw-w64-ucrt-x86_64-gcc-fortran?repo=ucrt64 . If people are still interested in this, a possible strategy may be to switch the vcpkg-gfortran port (added in https://github.com/microsoft/vcpkg/pull/12805) to use the ucrt gfortran version.


Experimental Features tab simply don't work

Describe the bug
By clicking in the 'Experimental Features' tab, nothing shows up as it should be

Desktop Setup

Operating System: Windows 11
Spotify: <<IP_ADDRESS>3.gc2965cdf>
Spicetify: 2.9.1
Dribbblish: 4.1.1

See original bug report: https://github.com/khanhas/spicetify-cli/issues/1485

See original bug report: khanhas/spicetify-cli#1485

Thanks!!!


loadWithXhr response handling

Add a workaround for xhr.responseType = 'json' for IE, which doesn't support that type. Additionally, add checks before attempting to access xhr.responseXML or xhr.responseText, which throws if responseType isn't an appropriate type.
loadWithXhrSpec currently only has tests for data URI loading. Is there a good way to test sending/loading of data without passing an actual URL? Mocking/spying doesn't seem possible as is since the request isn't publicly accessible.

This branch is ancient compared to master (even though it would merge cleanly).  Can you please merge in AGI master so that it's up to date.

loadWithXhrSpec currently only has tests for data URI loading. Is there a good way to test sending/loading of data without passing an actual URL? Mocking/spying doesn't seem possible as is since the request isn't publicly accessible.

It's okay to actually have the test request data, just use a relative path.  If you just need a non-JSON file, you can request anything from Specs/Data.  If there's a specific file you need, just put it into Specs/Data.

Thanks @klim705!
Specifically, we ran into the problem where the server ended up returning HTML when JSON was expected and loadWithXhr would never reject the promise.
@kring or @shunter, can you do a quick review.  You guys probably know the edge cases here a little better than I do.

Merged AGI's master, ran the autoformatter, and switched the catch block's deferred to reject with the exception. Also added unit tests for URL loading following a format similar to in Core/loadTextSpec to mock loading.

Does this warrant an update to CHANGES.md?

Does this warrant an update to CHANGES.md?

This is a trivial edge case that we just happened to never encounter, so probably not.  I doubt any of our users call loadWithXhr directly and higher level functions (like loadJson) did not have this problem.

Ready for another look.

Ready.

Thanks @klim705


Fix types

What's Changing and Why
What else might be affected
Tasks

[ ] Add tests
[ ] Update Documentation
[ ] Update jimp.d.ts
[ ] Add SemVer Label

:rocket: PR was released in v0.6.5 :rocket:

The types change was incorrect - there is no default export for jimp.
Try this:
const jimp = require('jimp');
console.log(jimp)

If it would be a default export, it should be visible in the log. Instead, you see this:
[Function: Jimp] {
  AUTO: -1,
  HORIZONTAL_ALIGN_LEFT: 1,
  HORIZONTAL_ALIGN_CENTER: 2,
  HORIZONTAL_ALIGN_RIGHT: 4,
  VERTICAL_ALIGN_TOP: 8,
  VERTICAL_ALIGN_MIDDLE: 16,
  VERTICAL_ALIGN_BOTTOM: 32,
[...]


Need to redeploy HPA after any cluster agent restart

Describe what happened:
I modified a configuration in the cluster agent yaml, but when I restarted it was not connected to the Horizontal Pod Autoscaler anymore. I had to go and redploy my hpa config
Describe what you expected:
I expect the cluster agent to be able to restart and still connect to the hpa
Steps to reproduce the issue:

restart the cluster agent
hpa is no longer connected

Additional environment details (Operating System, Cloud provider, etc):

Thanks for opening this @nicholas-devlin !
We identified an issue in the lifecycle that can occur if the Cluster Agent is running with the leader election enabled (or a Node Agent used to be the leader upon transitioning to the deployment of the Cluster Agent) but the External Metrics Provider is enabled for the first time thereafter.
If an HPA on External Metrics is already present, then the Cluster Agent (with it's informer) will List the HPAs (to extract the metric(s) that should be queried to Datadog) and keep a local cache for a certain period of time (30 seconds).
Now, only the leader can write in the ConfigMap that we use to store the External Metrics data, and for a Cluster Agent that is not the leader, the cache is flushed after the 30s.
Lastly, the Leader Election Lease is set at 90s by default.
Thus, if a Cluster Agent is not yet the leader, it will flush the cache prior to taking the leadership and therefore won't store any data in the ConfigMap.
If you do delete/recreate your HPA though, the Cluster Agent, that is now the leader will get the notification, parse through the HPA, query Datadog and store the metric in the ConfigMap and expose it (if it is valid obviously).
A temporary fix should be to set external_metrics_provider.batch_window to 120s in the datadog.yaml, this also exists as an environment variable DD_EXTERNAL_METRICS_PROVIDER_BATCH_WINDOW.
We are nevertheless working on this at them moment to improve the lifecycle and make it more resilient.
Thank you for opening this and apologies for the headache.
Best,
.C

should be solved in 1.3.0 according to @CharlyF


Can you add support INSERT/UPDATE hierarchical data?

You've solved problem #53, thanks. Can you add support of inserting and updating hierarchical data? Or suggest how i can use your library to insert / update hierarchical data.
You can use this repository as an example.
https://github.com/Makarov628/ODataToEntity_WebAPI

OdataToEnitys support batch requests.
Batch tests.
Test data.


Add option for workspace query concurrency to override default=10

Even with --concurrent-instances=1 (default), "skeema diff" performs its create - test - drop TABLE operations 10 at a time. That number (10) is too low in some environments, and too high in others. In some cases, a stream of CREATE and DROP operations over hundreds of temp tables can choke out normal operation of mysqld due to the Linux mutex contention.

Thank you for the feature request! Just to clarify a few details here, in case an open source contributor wants to pick this up before I get a chance:

This would be handled by a new option (maybe workspace-concurrency) which would then get passed through as a new field in workspace.Options, populated by workspace.OptionsForDir().
Workspace table creation concurrency is handled in workspace.ExecLogicalSchema(). This one is easy to change since the opts struct value is already available here.
Workspace table drop concurrency is more annoying to change. This is handled by each Workspace implementation in its Cleanup() function, by calling tengo.DropTablesInSchema(). It would need a new third param indicating concurrency, or perhaps its second param (currently a bool) should be changed to a new options struct. Either way, this breaks backwards compatibility... but since tengo is pre-1.0, that's acceptable (albeit non-ideal).
Potentially this new option could/should also affect schema introspection concurrency in several other places in tengo's instance.go. Undecided on this.
Overall this issue is unrelated to concurrent-instances. (That option controls concurrency among multiple mysqld instances in a sharded environment; it has no involvement with concurrency within a single mysqld instance.)

Happy to say this is now implemented in the master branch, and will be included in the release of Skeema v1.4, currently planned for later this month.
First, some background on what I found doing testing on Skeema's previous concurrency defaults:

For most users, dict_sys mutex contention shouldn't be a major problem; running Skeema with its default of workspace=temp-schema generally won't cause a noticeable impact to production queries.
Users with a very large number of tables may encounter problems though, especially if the normal query workload also involves high client connection counts. Increasing the size of MySQL's table cache can help, but this has memory implications.
Another major cause of dict_sys contention is workloads involving inserts to InnoDB tables that lack a primary key. Skeema's previous concurrency defaults could be problematic for workloads with many such inserts.
I suspect extensive use of foreign keys may make the contention worse (since at minimum it increases the amount of work that each CREATE or DROP does), but I didn't explore this.
MySQL 8's new data dictionary greatly improves the situation (less contention there).

Two new options have been implemented in Skeema to address this:

temp-schema-threads controls concurrency for workspace operations. The default is 5, just to lean on the safer side (relative to Skeema's previous hardcoded value of 10).
temp-schema-binlog controls whether or not workspace operations are written to the binary log. The default is to auto-detect whether or not Skeema's user has sufficient privs to set sql_log_bin=0 for workspace sessions, and skip binlogging when possible. Users may reconfigure this to "off" to never write the workspace queries to binlog (fatal error if insufficient privs), or "on" to always write to binlog.


Validate passphrase

It'd be useful to have a dedicated 'validate passphrase' option in the Passphrase Action menu, to double check a seed passphrase was set as intended after it's initially set.

Alternatively/additionally, it could be useful to have a 'show passphrase' option in the Seed Phrase views (after selecting a seed in the temp storage)

We're taking a super-close look at how we want to handle passphrases in our UI redesign. The basic principles for this new world are that you enter your passphrase when the seed is loaded or created, verify that it was entered right (w/a chance to edit if not), and then it's locked in. Can't display it again. Can't edit it. Can't add it later if you passed on the initial chance.
From that point on the seed would be displayed with an icon to indicate that it has a passphrase on it.
I'm also pushing to surface the master fingerprint more. That's the real verification that you'd want to make sure you entered your passphrase correctly (typo in passphrase = different/wrong fingerprint).

In my opinion, the only implementation of something like this that would make sense is to have a flow starting on the SeedOptionsView where you can reenter your passphrase and seedsigner will tell you whether it is correct or not. Though this would be a little bit at odds with the fact that the interface does not really indicate whether a seed was generated with a passprase or not (and it seems that the consensus is leaning towards keeping it that way https://github.com/SeedSigner/seedsigner/pull/286#issuecomment-1363363860).
I am wondering if when a user goes through the backup flow for a seed that was generated with a passphrase if there should be a prompt at the end for the user to verify their passphrase (similar to the step for verifying the seed words). Currently when you backup a seed that has a passphrase the only thing backed up are the seed words and there is no indication whether the seed was generated using a passphrase.

Related to this, it would also be nice to have an "edit seed phrase" as an option, but even after a correct phrase has been entered/approved.
Use case: you have multiple wallets with different seed phrases from the original creation of the root fingerprint. You can import the original seed then add and/or change the phrases to access unique seeds/wallets.
Thanks for what you do!

Use case: you have multiple wallets with different seed phrases from the original creation of the root fingerprint. You can import the original seed then add and/or change the phrases to access unique seeds/wallets.

In that case (well, in any case), I'd create a SeedQR so that the one seed can be quickly reloaded. At that point you can just keep scanning it back in and set a different BIP-39 passphrase each time.


adding more currencies

Currently it seems the only crypto supported is btc.  Thoughts about adding more?  Would you accept a PR?

Oh I guess this is a limitation of free.currencyconverterapi.com?

Sure, I'll accept a PR. Currently free.currencyconverterapi.com is used to get rates. They support only these currencies: https://free.currencyconverterapi.com/api/v5/currencies


Unstable dependencies

Hi Alex, first of all thanks for creating this great tool.
I've successfully generated Appstore Connect APIs using your tool at https://github.com/onmyway133/AppStoreConnect
And I'm ready to include it into https://github.com/onmyway133/Swiftlane. However, Xcode complains about urlqueryencoder being unstable dependencies.

Dependencies could not be resolved because root depends on 'AppStoreConnect' 0.0.2..<1.0.0.
'AppStoreConnect' >= 0.0.2 cannot be used because no versions of 'AppStoreConnect' match the requirement 0.0.3..<1.0.0 and package 'appstoreconnect' is required using a stable-version but 'appstoreconnect' depends on an unstable-version package 'urlqueryencoder'.

I guess in the generated Package.swift, we need to specify from instead of branch?
.package(url: "https://github.com/kean/URLQueryEncoder", from: "0.2.0"),

I think so. Updated in the main branch.

Thank you 👍


Is there anyway to avoid the Auth0 sign-in fields?

Hello all, I am using react-native-lock to connect with google through Auth0, but i can't find the way to avoid the "auth0" login view.
I want one of two things:
Avoid the auth0 sign in view to pass directly to Google sign in
or at least remove the Auth0 sign in fields.

@IgnacioAbadie don't use this library and use https://github.com/auth0/react-native-auth0#webauth, specifying connection parameter google-oauth2.


feat(accessibility): use correct aria properties

Description
Based on aXe recommendations, fix accessibility:

Use proper roles for the various components (ex: role="button" not needed on buttons).
Reorder navigation and pagination to better match the tabbing order.
Make navigation accessible by tabbing.
Change the HTML elements to better match W3's recommended organization

Motivation and Context
Accessibility of the carousel.
How Has This Been Tested?
Tested using vue-play in Firefox and Chrome on Linux.
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[X] My code follows the code style of this project.
[ ] My change requires a change to the documentation.

[ ] I have updated the documentation accordingly.

[ ] I have included a vue-play example (if this is a new feature)

^ resolved some small conflicts caused by the base branch switch

Coverage decreased (-0.9%) to 66.355% when pulling 7593846ea83de70530e672d5401de1baa3a5a73d on AdminXVII:feat-fix-aria into 7d5cb5c05006797ab2440ca60c3de57e3576a5c3 on SSENSE:v0.17.0.

Ah dang looks like this will need a rebase with v0.17.0. Do you mind rebasing and pushing back up when you get a sec?

Done

Perfect @AdminXVII, will merge into the working branch! Thanks for the upgrade 🚀


text to image arraybuffer to imageData

@coyotte508 @radames any suggestions for converting the text to image arrayBuffer to an imageData object. This does not seem to have the correct dimensions

      let UAC = new Uint8ClampedArray( objectDetectionRes, 768, 768);
      let imageData = new ImageData(UAC, 768, 768);

Hi @hpssjellis feel free to use discussions. @TimMikeladze what do you think about returning a blob instead from @huggingface/inference, it will contain the mime type of the image


Start defining the new user guides layout

First update to User Guides to start defining the layout and test basic functionality is working

Appears to be out of synch with local repo after 1st push?  Retrying


Revert "Removed time checks from PID library."

Commits in this PR

Revert "Removed time checks from PID library."
This reverts commit be574eaddb12501e9bfff7f74b8f7210dbe554af.
The time checks were removed from PID because we were only calling PID
from the precisely-timed control loop.  We tried to update the PID every
(say) 10ms, and we were confident we would call every 10ms, plus or
minus a few microseconds.  But if we happened to call with a delta of
9.99ms, the PID code would skip our sample!  That's bad.
However, for flow correction, we need our PID library to be able to
handle the case when the time interval is not constant.  In this case,
we do want time checks.
So in this change we add time checks back.  The next commit makes time
checking optional, so the PID library can work for both cases.

Make PID jitter mode configurable.
This way you can tell PID whether you're calling it on a strict schedule
(like we do from the high-priority loop) or "just whenever", like we
want to do for flow zeroing.

PR chain

👉 #462 Revert "Removed time checks from PID library." 👈 YOU ARE HERE
#463 First pass at TV zeroing.
#465 Granularity of Time and Duration 1ms -> 1us.

cc @sglow

With apologies for making you all do a review on this, I put together https://github.com/RespiraWorks/VentilatorSoftware/pull/465 to increase our time/duration granularity to 1us.  With that change, I don't think we actually need the JitterMode parameter at all; we can run both PIDs with the true clock.
So (again with apologies for asking you to review this), I'm going to drop the second commit in the PR and just do the straightforward revert.  Everything should Just Work.


Can download task add new property to avoid directly throw exception?

Is your feature request related to a problem? Please describe.
we don't want download task direclty throw exception when it can not dowload src file.  We want to an anlternative closure if the download task failed  and set a ignore this property, then no exception and dowload task can directly returen the existed download file lastt time.
A clear and concise description of what the problem is. Example: I'm always frustrated when [...]
we don't want download task direclty throw exception when it can not dowload src file.
Describe the solution you'd like
It will be better that task can add a new property to enable or disable throw exception action.
A clear and concise description of what you want to happen.
It will be better that task can add a new property to enable or disable throw exception action, that means task will not give an error console result.
Describe alternatives you've considered
It will be better that task can add a new property to enable or disable throw exception action.
A clear and concise description of any alternative solutions or features you've considered.
It will be better that task can add a new property to enable or disable throw exception action.

@michel-kraemer  Can you help to take a look?

@simonjhy How about this?
task downloadFile {
    doLast {
        try {
            download.run {
                src '<enter-your-url-here>'
                dest layout.buildDirectory
            }
        } catch (e) {
            println("Ignore error. Do something else instead.")
        }
    }
}

@michel-kraemer i
I tried and we can not use this way.  Below is we used in our code. We add the download task in our gradle plugin, we don't drectly call download.run.   https://github.com/liferay/liferay-portal/blob/7c720a2bc9c2a31d7f3ecd5144844851331557c4/modules/sdk/gradle-plugins-workspace/src/main/java/com/liferay/gradle/plugins/workspace/configurator/RootProjectConfigurator.java#L1009

I think what you have to do in this case is to write a custom task like this:
class MyDownload extends DefaultTask {
    private def action

    MyDownload() {
        action = new de.undercouch.gradle.tasks.download.DownloadAction(project, this)

        action.src('https://example.com/404')
        action.dest(project.layout.buildDirectory)
    }

    @TaskAction
    void download() {
        action.execute(false).exceptionally(t -> {
            println("Ignore error. Do something else instead.")
        }).thenRun(() -> {
            // copied from de.undercouch.gradle.tasks.download.Download.download()
            if (action.isUpToDate()) {
                getState().setDidWork(false);
            }
        });
    }
}

The important thing here is to pass false to action.execute. This will make the action not throw an exception immediately but return a CompletableFuture that completes exceptionally if the download failed and normally if it succeeded (see the method's javadoc).
I hope this helps. Let me know if it works or if you need more assistance.

@michel-kraemer Thank you for your help. I tested your function. When I set false to the value of execute, that will not throw any exception. I hope that I can add some logic to determine what exception I can throw in exceptionally method. For example, when netwok is borke and the download file is existed, then I don't throw exception, but I want to throw excetion when the destinantion file is not existed and the network broke.

This should be possible. Have a look at the javadoc of CompletableFuture. The whenComplete method might be the one you're looking for.

Seeing the emojis you sent, it appears this solution has worked for you. I'm closing this issue. If you still have questions, feel free to comment here again.


Dockerfile for portable use

I found myself using this with a Dockerfile to avoid modifying my filesystem on OSx (Catalina and Big Sur require mounted filesystems on the /nix path of the filesystem).  I thought perhaps it could be useful here as well.
Definitely feel free to edit or make suggestions or close if this isn't useful. I could understand wanting documentation around this or a hosted Docker image to pull.
The way I use this is as follows:

Build the container (could be pulled if hosted on a registry)

docker build --pull --rm -f "Dockerfile" -t Shopify/comma:latest

Update ".${SHELL}rc" to wrap the docker run command in an alias

alias ,='docker run -it shopify/comma'

Run executable as desired

, cowsay "Hello from Docker!"

My apologies, large oversight with mounting volumes I just ran into.


Erronious Characters Sometimes Copied When Viewing a Password

Only occurs sometimes, not sure why.

Windows only?


Add example to get a team ID from its name using ListTeams

This PR adds an example on how to use ListTeams to get the ID of a team based on its name.

Ah, OK, @aerostitch... thanks for the info.  Typically, we develop PRs locally or in our own forked repos and don't actually create a PR on the main repo until we are happy with all the changes. That way, when a PR comes in, anyone can start reviewing it. No big deal... I just want to help you out with the typical "flow". I'll take another look.

Thanks for the review @gmlewis

Typically, we develop PRs locally or in our own forked repos and don't actually create a PR on the main repo until we are happy with all the changes.

Yes, that's what I usually try to stick to but I got second thoughts after reading the PR again. Sorry for that.
Good catch for the usage of %q, I must say I never used it before! :sweat_smile:
I know for the accessors, you told me last time and I forgot again. Sorry for that. :disappointed:
Anyway, I updated the PR based on your feedback. Thanks! :)
Joseph

Thanks. New lines added! :)

Thanks for the review @shurcooL !
PR updated accordingly! :)

Thanks! :)


QEMU and twister support on windows not possible?

Describe the bug
I can build project with qemu (qemu_cortex_m3)

Build Target
west build -b qemu_cortex_m3 $ZEPHYR_BASE/samples/hello_world -p

Run Target with QEMU
west build -t run

Please also mention any information which could help others to understand
the problem you're facing:

zepyhr 3.2.0

To Reproduce
Steps to reproduce the behavior:
If i now want to write a unit test with ztest and call twister to build and run the tests
python $ZEPHYR_BASE/scripts/twister -vvv -T MyTestSuit -p qemu_cortex_m3
Build OK
RUN skipped
Expected behavior
Test should not only build, also run !
Impact
The tests are never running on windows hosts. Unit testing is essential for quality software development. ZTest is not usefull in that scenario. Tests should have an reason, why they are skipped to run.
Logs and console output
If applicable, add console logs or other types of debug information
e.g Wireshark capture or Logic analyzer capture (upload in zip archive).
copy-and-paste text and put a code fence (```) before and after, to help
explain the issue. (if unable to obtain text log, add a screenshot)
Environment (please complete the following information):

OS: Windows
Toolchain Zephyr SDK
Version 3.2.0

Additional context
Build and RUN from QEMU Targets seams not to be a problem. But running unit tests, which build seems also to work but not the RUN seams to me a broken feature which is not clearly commented neither wiki or in the docu itself: https://docs.zephyrproject.org/latest/develop/test/twister.html# If i read this, it seams promising to me, that unit tests also work in windows with qemu.
scripts/pylib/twister/twisterlib/testinstance line 192:
# Global testsuite parameters
def check_runnable(self, enable_slow=False, filter='buildable', fixtures=[], hardware_map=None):

    # running on simulators is currently not supported on Windows
    if os.name == 'nt' and self.platform.simulation != 'na':
        return False

Tests are prevented to run with twister to run windows! WHY not mention that serious impact!
If forcing over that, it seams to have an issue with named piped for qemu communication
´´´DEBUG   - adding xt-sim to default platforms
DEBUG   - Reading platform configuration files under C:\ncs\v2.3.0\zephyr\scripts\pylib\twister\boards...
DEBUG   - adding unit_testing to default platforms
DEBUG   - Reading platform configuration files under C:\ncs\v2.3.0\nrf\boards...
DEBUG   - platform filter: ['qemu_cortex_m3']
DEBUG   -   vendor filter: []
DEBUG   -     arch_filter: None
DEBUG   -      tag_filter: None
DEBUG   -     exclude_tag: None
INFO    - Building initial testsuite list...
INFO    - Writing JSON report C:\Projekte\CES\git\twister-out\testplan.json
INFO    - Adding tasks to the queue...
DEBUG   - adding qemu_cortex_m3\UnitTests\test.hf.ztest
INFO    - Added initial list of jobs to queue
DEBUG   - Launch process 0
DEBUG   - Launch process 1
DEBUG   - Launch process 2
DEBUG   - Launch process 3
DEBUG   - Launch process 4
DEBUG   - Launch process 5
DEBUG   - Launch process 6
DEBUG   - Launch process 7
Exception in thread qemu_cortex_m3\UnitTests\test.hf.ztest:
Traceback (most recent call last):
File "threading.py", line 932, in _bootstrap_inner
File "threading.py", line 870, in run
File "C:/ncs/v2.3.0/zephyr\scripts/pylib/twister\twisterlib\handlers.py", line 836, in _thread
out_fp, in_fp, log_out_fp = QEMUHandler._thread_open_files(
File "C:/ncs/v2.3.0/zephyr\scripts/pylib/twister\twisterlib\handlers.py", line 784, in _thread_open_files
os.mkfifo(fifo_in)
AttributeError: module 'os' has no attribute 'mkfifo'
´´´

This is not supported and currently not planned, so it is not a bug.

Any idea if it will be supported in the near future? how much work we talking here?
I agree with OP that it should be mentioned in the doc that it won't run on windows although building & running with qemu is possible.

@Dermiste you can use the workaround (west build -t run) to build the test and run.....  What you dont have is the test case management to collect a test collection from different tests.... but in my case... we use this for libs or specific target and can manage them in one project.
Problem is to export the test reports.... but for local tests it works. For CI pipelines we will setup an linux enviroment.

Thanks that's what I figured too, we will also have a CI pipeline allowing a linux environment, in the meanwhile I'll just manage by just building and checking the output myself, I wish this would have just been mentioned in the doc.

This is not supported and currently not planned, so it is not a bug.

is there some specific incompatibility with Windows? considering qemu works with west, what's the blocker? I'm hoping to work around this but if there's code written to specifically skipping this platform I guess that won't work.


Strip 'fbid:' from message senderID in util.js

Fix for Issue 438
This uses string functions substr and indexOf to strip fbid: from senderID as referred in my issue.
This should be compatible whether the senderID has the fbid: prefix or not. If the senderID string does not have the prefix, it will do substr(0) which will not strip anything.

Have chat groups too fbid: prefix?

I thought using indexOf(':') would be better because I saw in a different issue (can't remember which) where the prefix was just "id:". This method would be able to strip the prefix independent of what was before the colon.
Both chat groups and normal conversations have the fbid: prefix.

Probably this is world-wide affect, here in Slovakia it's too (we got really late updates).
You already answered, but yes, groups are affected too.
It's possible the delta messages have this prefix too, I will lookup it

It's looks like some functions will be migrating to fbid prefix too

Could you make sure this very similar issue is also fixed? https://github.com/Schmavery/facebook-chat-api/issues/282

What about .replace(/(fb)?id:/, "")

@Schmavery
I'm not sure if this all "stripping" is good idea at all.
I think the best idea will be to lookup the Messenger/Facebook sources how they parse "real" threadID, because for sample, Message ID have mid. prefix, and without that mid. prefix is not possible to delete messages etc. + again Message IDs have many formats. I know it's not threadID related, but in future could be problem with it. 🤔

@GAMELASTER You could be right, I think there's probably some design decision here.
I'm sure that it's not necessary to have the prefix on the ID, because we can always just add it back on as necessary when making the call.  However, it could allow us to give better error messages if a user passes a message id to a function that is expecting a thread id, for example.
Currently, everything expects ids not to have a prefix, so this is the right fix short term.  If we decide we want to prefix ids, I'd want to do it all at once.  There are a couple edge cases here too, because user ids and thread ids are sometimes treated the same and sometimes not.
Historically, we did it this way because no one cared very much about message ids (you couldn't do many interesting things with them, so usually you didn't need to use it).  Now that we have reactions, which is based on message id, I'd imagine we'll get a couple more messageID-related issues and will have to clean up that area.
@mangopearapples Do you think you could make a utils function that's like "formatID" or something (if we don't already have one) and use that?  If we can gradually move over to having everything use the same function, it will be easier to remove or change behaviour in the future.

@Schmavery
The delimiter in issue #282 is a . and not a : which could make the regex a bit too complicated. Since indexOf() does not use regex, we could use search instead? sender_id.substr(sender_id.search("\\.|:"))
Working on util function.

Is this too complicated?
/(fb)?id[:.]/

Yeah I guess not. Thanks.
Participant ID's also started using the fbid: prefix since I posted this PR so I have included those in this commit as well.

@mangopearapples is this ready to merge?  Were you able to add the fix for #282 as well?

@Schmavery Yes should be ready unless I've missed out something. I've applied formatID to every applicable field (senderID, threadID, participantIDs) I could find.

Awesome thanks a lot for that! Added some comments, ready to merge once they're addressed :)

Oh also could you wrap those in utils.formatID
https://github.com/Schmavery/facebook-chat-api/blob/9b85a79c3dafca33d6f4d36f6c00fc373903698a/src/getFriendsList.js#L30
and
https://github.com/Schmavery/facebook-chat-api/blob/9b85a79c3dafca33d6f4d36f6c00fc373903698a/src/getUserID.js#L8

All done.
@Schmavery Should be ready to merge now.

Thanks a lot for this!


BREAKING!: change from demo to board

Change all occurrences of simulation/demo in internal/public APIs and comments to board.
Remove the props property that is no longer relevant in the new format.

ah, file name is simulation-fixtures.tsx rename that too?


singleplayer does not work

when i play singleplayer, i just get a black floor and the blocks on the top. that's it. i use google chrome.

Hey when I opened the game everything was just black! Is there something wrong with the lighting? If you could help that would be great, but so far good job on the game, just need to fix some minor issues!

@LublubXT
If you just open the html file in a browser, textures will be missing and look black.
If you use any simple web server (e.g. python3 -m http.server) singleplayer.html will work correctly.
Multiplayer.html needs the custom server.js script: https://github.com/Overv/WebCraft/issues/46#issuecomment-605383125

@LublubXT
If you just open the html file in a browser, textures will be missing and look black.
If you use any simple web server (e.g. python3 -m http.server) singleplayer.html will work correctly.
Multiplayer.html needs the custom server.js script: #46 (comment)

Thanks!

Also check this out:

https://github.com/Cobra-Tec-Inc/Number_Line_1.0

It the app we made. It does have some bugs but should be fixed by next week.


problem with Fragilariopsis

When reannotating the Tara Oceans V9 metabarcodes using PR2 v4.14, we noted the lack of Fragilariopsis in the new annotation. Using the most abundant Fragilariopsis OTU in the original dataset, we found that the best hits in PR2 v4.14 correspond to 100% identity to a group of sequences including a Dinophyceae (obviously an error in PR2 v4.14) and a bunch of diatom reference sequences poorly assigned as 'Raphid-pennate_X_sp', in addition to a Fragilariopsis sequence:
query:

1533bc0882cb58d28b5c56f49269c1c2
gtcgcacctaccgattgaatggtccggtgaagcctcgggattgtggttagtttcctttattggaagttagtcgcgagaacttgtctaaaccttatcatttagaggaaggtgaagtcgtaacaaggtttcc

Subjects (100 % id):
KC771185.1.1787_U;tax=k:Eukaryota,d:Stramenopiles,p:Ochrophyta,c:Bacillariophyta,o:Bacillariophyta_X,f:Raphid-pennate,g:Raphid-pennate_X,s:Raphid-pennate_X_sp.
EF100371.1.1232_U;tax=k:Eukaryota,d:Stramenopiles,p:Ochrophyta,c:Bacillariophyta,o:Bacillariophyta_X,f:Raphid-pennate,g:Raphid-pennate_X,s:Raphid-pennate_X_sp.
KC771174.1.1777_U;tax=k:Eukaryota,d:Stramenopiles,p:Ochrophyta,c:Bacillariophyta,o:Bacillariophyta_X,f:Raphid-pennate,g:Raphid-pennate_X,s:Raphid-pennate_X_sp.
KC771149.1.1790_U;tax=k:Eukaryota,d:Stramenopiles,p:Ochrophyta,c:Bacillariophyta,o:Bacillariophyta_X,f:Raphid-pennate,g:Raphid-pennate_X,s:Raphid-pennate_X_sp.
KC771193.1.1786_U;tax=k:Eukaryota,d:Stramenopiles,p:Ochrophyta,c:Bacillariophyta,o:Bacillariophyta_X,f:Raphid-pennate,g:Raphid-pennate_X,s:Raphid-pennate_X_sp.
KC771190.1.1789_U;tax=k:Eukaryota,d:Stramenopiles,p:Ochrophyta,c:Bacillariophyta,o:Bacillariophyta_X,f:Raphid-pennate,g:Raphid-pennate_X,s:Raphid-pennate_X_sp.
KC771168.1.1787_U;tax=k:Eukaryota,d:Stramenopiles,p:Ochrophyta,c:Bacillariophyta,o:Bacillariophyta_X,f:Raphid-pennate,g:Raphid-pennate_X,s:Raphid-pennate_X_sp.
EF140623.1.1779_U;tax=k:Eukaryota,d:Stramenopiles,p:Ochrophyta,c:Bacillariophyta,o:Bacillariophyta_X,f:Raphid-pennate,g:Fragilariopsis,s:Fragilariopsis_curta
KJ757846.1.1801_U;tax=k:Eukaryota,d:Alveolata,p:Dinoflagellata,c:Dinophyceae,o:Dinophyceae_X,f:Dinophyceae_XX,g:Dinophyceae_XXX,s:Dinophyceae_XXX_sp.
KJ757881.1.1791_U;tax=k:Eukaryota,d:Stramenopiles,p:Ochrophyta,c:Bacillariophyta,o:Bacillariophyta_X,f:Raphid-pennate,g:Raphid-pennate_X,s:Raphid-pennate_X_sp.
KJ758129.1.1800_U;tax=k:Eukaryota,d:Stramenopiles,p:Ochrophyta,c:Bacillariophyta,o:Bacillariophyta_X,f:Raphid-pennate,g:Raphid-pennate_X,s:Raphid-pennate_X_sp.
KJ758140.1.1795_U;tax=k:Eukaryota,d:Stramenopiles,p:Ochrophyta,c:Bacillariophyta,o:Bacillariophyta_X,f:Raphid-pennate,g:Raphid-pennate_X,s:Raphid-pennate_X_sp.
KJ758229.1.1792_U;tax=k:Eukaryota,d:Stramenopiles,p:Ochrophyta,c:Bacillariophyta,o:Bacillariophyta_X,f:Raphid-pennate,g:Fragilariopsis,s:Fragilariopsis_sp.
I wonder whether there are more mistakes in the last PR2 version. Maybe a way of checking this is by running some re-assignment test, or random checks to spot other mistakes, or maybe by detecting cases where the taxonomy of the best hit sequences (with identical %id) are strongly contradictory  - a dinoflagellate followed by diatoms as in this case.

Dear Juan
This is an interesting case which shows that V9 in some case is not really a good marker.  I aligned the sequences that matched at 100% your Tara barcode.   This set of sequence contains only one Fragilariopsis strain (sequence in bold in the list).  Other are Fragiliariopsis environmental sequences (so they have been annotated as Fragilariopsis based on their similarity to the strain (see in bold in list of sequences below).  Outside the V9 in the V4 region, there are at least 3 sequences annotated as  Raphid-pennate_X_sp. which have a very different signature from Fragilariopsis .
The Dino sequence is interesting because it is 100% similar to your barcode in the V9 region but very different outside V9.  In fact, doing a BLAST shows that it is most similar (still lower than 96% to some dino sequences).  I suspect it could be a chimera and I will quarantine for the next version of PR2.
When you use dada2 assignTaxonomy (same as RDP classifier) or DECIPHER using your sequence against PR2, it finds correctly Fragilariopsis...
Final word concerning "mistakes" in PR2. It is a bit more complex.   PR2 rely on expert annotation for specific groups as it is the best way to have a solid database.  In this process some sequences are flagged as suspicious and removed or reassigned.  In the case of the dinoflagellate even doing a BLAST against the database will not work because it will hit other dinoflagellate.  One solution would be to run a chimera detection program, but the output needs to be thoroughly processed to make sure that bona fide sequences are not excluded.  So the PR2 annotation should always be taken with a grain of salt... and we are welcoming experts to help us improving it.  Diatoms were one of the group on which people worked during the last EukRef workshop but unfortunately things were not finalized....

The dino sequence will be labelled as chimera in version 5.0.0 soon to be released.


fix: android styling

fix: missing adaptive icon (uses Expo default). derived one from the main icon, following google guidelines re: size, spacing etc.
fix: status bar not matching the header background
fix: header dropdown offset
fix: status bar not matching splash screen
fix: header back button spacing (styling borrowed from react-navigation)
(unrelated): fix: simple demo wrong imports


Copying back tempWeight to weight when in Evaluate mode so make sure …

…weights are consistent for consequtive in memory forwards.

@sagarwaghmare69 Thanks!


params is not being passed to downstream build

Hi there,
I'm using both the latest versions of drone and drone-trigger and the following upstream test pipeline:
pipeline:
  deploy:
    image: quay.io/ukhomeofficedigital/drone-trigger
    drone_server: server
    drone_token: token
    repo: owner/repo
    deploy_to: testing
    params: FOO=BAR
    verbose: true

and the following downstream pipeline:
pipeline:
  deploy:
    image: alpine
    commands:
      - echo "$FOO"

.
The downstream pipeline is called from the upstream pipeline so drone-trigger generally works but the result of the deploy step is always:
1 + echo "$FOO" 0s
2

exit code 0

.
Even calling the pipeline directly via drone deploy --param FOO=BAR owner/repo 1 testing yields the same results. Do I need to enable/set some flags anywhere?
Thanks in advance.

Hi,
sorry it seems it was a bug in drone itself and fixed with the new v0.7.


Feature: pip support

Adds support for pip dependencies. For this, we take the same approach as conda-lock.
In addition, I have modified the logic of Environment.is_prepared to do a more robust comparison of packages installed vs. specified in the lock file.

Codecov Report

Merging #39 (49df4a2) into main (0f7806d) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #39      +/-   ##
==========================================
- Coverage   99.79%   99.79%   -0.01%
==========================================
  Files           8        7       -1
  Lines         489      483       -6
==========================================
- Hits          488      482       -6
  Misses          1        1

Impacted Files
Coverage Δ

conda_project/project.py
99.55% <100.00%> (+<0.01%)
:arrow_up:

conda_project/cli/main.py
100.00% <0.00%> (ø)

conda_project/project_file.py
100.00% <0.00%> (ø)

conda_project/__init__.py

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Thanks. I guess I should pull the windows fix to a separate PR

Yeah I was going to do that after I saw these results 🙂.
Could use feedback on whether it's okay to check return code or if there's
a better way to check if a package got installed via pip.
-Matt
Sent via Mobile
On Sun, Nov 6, 2022, 7:49 AM Albert DeFusco @.***>
wrote:

Thanks. I guess I should pull the windows fix to a separate PR
—
Reply to this email directly, view it on GitHub
https://github.com/conda-incubator/conda-project/pull/39#issuecomment-1304805828,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUGYKO42V4N3IZC7FM6SUDWG6SPNANCNFSM6AAAAAARYIFNXI
.
You are receiving this because you authored the thread.Message ID:
@.***>

I would look into using conda list, which includes pip packages to verify that they were installed. This could be a small change to is_prepared to include a check for any pip packages, which would also be good to do even if no pip packages appear in the env sources.
I think the pip install return code will be ok as well.

Cool. I'll take a look. I've been working on this, too and was using sha256 for pip and conda
https://github.com/AlbertDeFusco/conda-project/commit/4c44dfe6bea3662dd0164cfa7ae499f26e59fc6a

wow, good catch on the prefixdata cache. That's why my tests were failing.

That was a tricky one!
I think the general skeleton is there in this PR, if sha256 would work better feel free to swap that out.

You're approach looks much more streamlined than I did. I bet we can just as easily use sha256.
What I found with my tests was some unfortunate slowdowns in the is_prepared property. I'll run yours a few times and see how it does.
Either way, I think we'll need to see about further optimizations here (and maybe even ensuring the is_prepared is called as infrequently as possibler

I can take a look at yours and pull out the relevant bits. I saw you had more extensive handling of the different types of pip packages (I only found one).
I agree a property may be inefficient here. Probably something to sort out in another PR. I actually had the realization that this whole thing is a finite state machine so actually we may be able to simplify all the methods a lot with some refactoring.
For now, I propose I modify this PR to capture all package types associated with pip packages and compare based on sha256 hash instead of package name and version. I'll work on that this evening probably.

That would be great to explore. Right now I've been going after an exhaustive comparison between source spec, lock, and live env. Perhaps there are faster ways to determine that "something" doesn't mathch rather then working through all packages. I imagine it can only get worse for large envs

Alrighty, made a few cleanups and added sha256 to the comparison. I have left in package name, version, and manager since they should be lightweight and I figured it opens the possibility to have better messaging for why an environment may not match the lockfile. Let me know if you think I should remove.

Okay, I think I've got it where it needs to be. I think the tests should pass now. Let's see ..

Can you also edit the setup.py with the pin


Method setToken does not exist

Can you tell me where you put this setToken method?
$this->guard()->setToken($token);
i got some error becouse i cant found this method

i'm sorry, i've found it in JWTGuard


DocumentElement.TextContent incorrectly processed style tag

Test code:
var html = @"

hello
";

var parser = new HtmlParser();
var dom = parser.Parse(html);
var text = dom.DocumentElement.TextContent;
Console.WriteLine(text);

Output:

                hello

This all seems to be alright. The output in modern browsers is:

    hello

This seems to be the same. What exactly is your issue? What do you expect? Are you familiar with the HTML specification or have you tried to reproduce your issue in at least one of the evergreen browsers?

Expect:
hello

This is wrong. Please make sure you understand the HTML specification / test with browsers in case of doubt.
Style uses a different mode (rawtext, more about these modes in the HTML5 mastery series) for its content, but its still using a Text node to store its content. I understand your confusion, but this is all as specified by the W3C.
However, nothing prevents you in AngleSharp to serialize a document according to your wishes (and thus skipping text from special nodes, e.g., script or style tags).


Add test for setup.py::dependency-link field

The issue
Issue #2613 was fixed via #2643, but the fix did not include a regression test.
The fix
This pull request proposes an integration test that failed prior to #2643 but succeeds now.
By taking a look at #2643 it wasn't clear to me how to write a unit test instead of this integration test. Because this integration test relies on web connectivity to a github repository, I created a dummy repo to be used just by this test. I suspect it is preferable to fork that repo under the pypa org so as not to have it under my personal account though.
The checklist

[x] Associated issue #2613
[ ] A news fragment in the news/ directory to describe this fix with the extension .bugfix, .feature, .behavior, .doc. .vendor. or .trivial (this will appear in the release changelog). Use semantic line breaks and name the file after the issue number or the PR #.

There doesn't seem to be a relevant fragment for just adding a regression test

Any chance I can get invited to the Build Kite org? What would really simplify me iterating and figuring out how to fix this is seeing how the tests are failing each time. In our org we use CircleCI and there are some per-project settings that I configured for accessing private repositories. This is not the case here, but I imagine we may need SSH keys to be set up nevertheless

Ah I think i see the issue:
Could not fetch URL http://<IP_ADDRESS>:38445/simple/test-private-dependency/: 404 Client Error: NOT FOUND for url: http://<IP_ADDRESS>:38445/simple/test-private-dependency/ - skipping
Cleaning up...
Removed build tracker '/tmp/pip-req-tracker-vXvve7'
Exception information:
Traceback (most recent call last):
  File "/tmp/pipenv-DqaIHC-project/.venv/local/lib/python2.7/site-packages/pip/_internal/basecommand.py", line 141, in main
    status = self.run(options, args)
  File "/tmp/pipenv-DqaIHC-project/.venv/local/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 299, in run
    resolver.resolve(requirement_set)
  File "/tmp/pipenv-DqaIHC-project/.venv/local/lib/python2.7/site-packages/pip/_internal/resolve.py", line 102, in resolve
    self._resolve_one(requirement_set, req)
  File "/tmp/pipenv-DqaIHC-project/.venv/local/lib/python2.7/site-packages/pip/_internal/resolve.py", line 256, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/tmp/pipenv-DqaIHC-project/.venv/local/lib/python2.7/site-packages/pip/_internal/resolve.py", line 209, in _get_abstract_dist_for
    self.require_hashes
  File "/tmp/pipenv-DqaIHC-project/.venv/local/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 218, in prepare_linked_requirement
    req.populate_link(finder, upgrade_allowed, require_hashes)
  File "/tmp/pipenv-DqaIHC-project/.venv/local/lib/python2.7/site-packages/pip/_internal/req/req_install.py", line 308, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/tmp/pipenv-DqaIHC-project/.venv/local/lib/python2.7/site-packages/pip/_internal/index.py", line 543, in find_requirement
    'No matching distribution found for %s' % req
DistributionNotFound: No matching distribution found for test-private-dependency (from testdeplinks==0.1)
1 location(s) to search for versions of pip:
* http://<IP_ADDRESS>:38445/simple/pip/
Getting page http://<IP_ADDRESS>:38445/simple/pip/
Starting new HTTP connection (1): <IP_ADDRESS>:38445
http://<IP_ADDRESS>:38445 "GET /simple/pip/ HTTP/1.1" 404 233
Could not fetch URL http://<IP_ADDRESS>:38445/simple/pip/: 404 Client Error: NOT FOUND for url: http://<IP_ADDRESS>:38445/simple/pip/ - skipping

I think you need to mark this with needs_internet and remove the pypi argument everywhere it occurs so we stop trying to retrieve info about it from the mocked index.  If tests continue to fail you can first check for the return_code from the installation and if it is non-zero you can print c.err for better debugging and I can pass that along

Hm I think i was wrong, but I made one small change to your branch (hope you don't mind) which was to add the environment variable PIP_PROCESS_DEPENDENCY_LINKS using the temp_environ context manager we have setup.  Without any instruction otherwise, pip doesn't actually use the information in dependency_links

Awesome, thanks for the help @techalchemy! I'm trying out some alternatives to include a test for a git+ssh dependency link, and I should have something more polished and ready to merge soon.

Ready to merge on my side

@techalchemy ready to merge on my side (forgot to at-mention before)

thanks, sorry for the delay!


Add TPM support to the dockerfile

@madninja yes, it would be. For some reason I thought those were mutually exclusive. If not, we should definitely do it because this build setup is going to get messy fast.

Closing in favor of #376


Missing print function in contact box

We are missing the print 	functionality behind the print button in the contact-box. This feature was already included in the old webguidelines.
Is it not implemented or are we not able to see it?
Thanks for your help.

We didn't implement this feature. I'll ask the Chancellerie.

the print function is still missing for the "map" part of the contact box

when I choose the option "links as footnotes", the display is adapting but the links are missing at the end of the page, is it a bug or it is just because they are fake links?

it is not working properly on IE9

The function "links as footnotes" isn't displaying anything on IE9 + shouldn't the urls only be displayed when the function "links as footnotes" is active?

The function is working on my side. Note that we don't display any footnote if the URL doesn't start with http://. We did this to avoid a footer bloated with # links.
We'll add a clear comment about this in the docs.

Will be solved with SWG-80

This issue has been addressed in version 3.0.0.


Can't instantiate Claude API Client

Here is my code trying to instantiate this Claude API
from claude_api import Client
import os

claude_api = Client('sk-ant-my-key-here')

However, I'm getting the following error:
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[205], line 1
----> 1 claude_api = Client('sk-ant-my-key-here')

File ~/miniconda3/envs/truth/lib/python3.11/site-packages/claude_api.py:13, in Client.__init__(self, cookie)
     11 def __init__(self, cookie):
     12   self.cookie = cookie
---> 13   self.organization_id = self.get_organization_id()

File ~/miniconda3/envs/truth/lib/python3.11/site-packages/claude_api.py:33, in Client.get_organization_id(self)
     31 response = requests.get(url, headers=headers,impersonate="chrome110")
     32 res = json.loads(response.text)
---> 33 uuid = res[0]['uuid']
     35 return uuid

KeyError: 0

Any ideas on how to mitigate this?

This solves the issue for those who are also coming to this error:
class Client:
def init(self, cookie):
self.cookie = cookie
self.organization_id = self.get_organization_id()
def get_organization_id(self):
BASE_URL = "https://claude.ai"
API_ENDPOINT = "/api/organizations"
API_KEY = cookie

HEADERS = {
  "Content-Type": "application/json",
  "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0",
  "Authority": "claude.ai",
  "Accept": "*/*",
  "Accept-Language": "en-US,en;q=0.9",
  "DNT": "1",
  "Sec-Fetch-Dest": "empty",
  "Sec-Fetch-Mode": "navigate",
  "Sec-Fetch-Site": "same-origin",
  "Upgrade-Insecure-Requests": "1",
  "Connection": "keep-alive"
}

url = BASE_URL + API_ENDPOINT

headers = HEADERS.copy()
headers["Cookie"] = f"sessionKey={API_KEY}"

response = requests.get(url, headers=headers)

if response.status_code == 200:

      orgs = response.json()

      for org in orgs:
            return org["uuid"]

      else:
            print("Error:", response.text)

@gullpavon  It seems not work for me

Just log in again and get the cookies

This solves the issue for those who are also coming to this error:
class Client:

  def __init__(self, cookie):
    self.cookie = cookie
    self.organization_id = self.get_organization_id()

  def get_organization_id(self):

    BASE_URL = "https://claude.ai"
    API_ENDPOINT = "/api/organizations"
    API_KEY = cookie

    HEADERS = {
      "Content-Type": "application/json",
      "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0",
      "Authority": "claude.ai",
      "Accept": "*/*",
      "Accept-Language": "en-US,en;q=0.9",
      "DNT": "1",
      "Sec-Fetch-Dest": "empty",
      "Sec-Fetch-Mode": "navigate",
      "Sec-Fetch-Site": "same-origin",
      "Upgrade-Insecure-Requests": "1",
      "Connection": "keep-alive"
    }

    url = BASE_URL + API_ENDPOINT

    headers = HEADERS.copy()
    headers["Cookie"] = f"sessionKey={API_KEY}"

    response = requests.get(url, headers=headers)

    if response.status_code == 200:

          orgs = response.json()

          for org in orgs:
                return org["uuid"]

          else:
                print("Error:", response.text)

It works well!!!! Thanks!!!!! ┭┮﹏┭┮

This solves the issue for those who are also coming to this error:
class Client:

  def __init__(self, cookie):
    self.cookie = cookie
    self.organization_id = self.get_organization_id()

  def get_organization_id(self):

    BASE_URL = "https://claude.ai"
    API_ENDPOINT = "/api/organizations"
    API_KEY = cookie

    HEADERS = {
      "Content-Type": "application/json",
      "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0",
      "Authority": "claude.ai",
      "Accept": "*/*",
      "Accept-Language": "en-US,en;q=0.9",
      "DNT": "1",
      "Sec-Fetch-Dest": "empty",
      "Sec-Fetch-Mode": "navigate",
      "Sec-Fetch-Site": "same-origin",
      "Upgrade-Insecure-Requests": "1",
      "Connection": "keep-alive"
    }

    url = BASE_URL + API_ENDPOINT

    headers = HEADERS.copy()
    headers["Cookie"] = f"sessionKey={API_KEY}"

    response = requests.get(url, headers=headers)

    if response.status_code == 200:

          orgs = response.json()

          for org in orgs:
                return org["uuid"]

          else:
                print("Error:", response.text)

thank, this is helpful to me but at first you guy need a new claude cookie


Update demo

Problem
React-Admin CRM demo was updated
Solution
Update ra-supabase demo

All issues found in the demo should be fixed now! :white_check_mark: