Use taskkill for win32

What problem did you solve?
Follows #936

Thanks.

The same with previous build, R.exe cannot work

So far I tried two configurations:

"r.rpath.windows": "... rterm path ..."
without "r.rpath.windows" set (no presence in settings)

I followed the usual workflow: loaded R files, edited R files, executed lines, created a R terminal, loaded R workspace, quit from R terminal, switched from R project (intending with R files) and a Nodejs (intending with no R files) and back, and so on.
I don't see leaked processes when I close vscode. So for me it works.
@renkun-ken What are your steps to reproduce the issue?

It seems magic.
I run a session with configuration 2 (without "r.rpath.windows" set), almost without any action, and the processes have not been closed. Sigh.
I rerun a new session with configuration 1 (with "r.rpath.windows" set) and I give you a feedback.

I used the/path/of/R.exe in my r.rpath.windows setting, and restarted vscode and opened a file with .R extensionn, then closed the vscode window. we can checked the task manager and found a lot of R.term were not killed.

And we changed the setting r.rpath.windows to use the/path/of/Rterm.exe, it can work well

Agreed.
We have three configurations:

"r.rpath.windows": "... RTerm path ..." - the fix works
without "r.rpath.windows" set (no presence in settings) - the fix doesn't work, but it is hard to reproduce it
"r.rpath.windows": "... R path ..." - the fix doesn't work

Again, with configuration 1, after 1 hour of vscode inactivity, while I was in a few video calls, I closed vscode and I opened process explorer (I said myself just in case). Sigh, I noticed those "frontend" processes. Really I have no idea what it is happening.

Looks like in your cases, the taskkill command is never triggered since the R.exe process exits on its own before we got a chance to kill its tree.
The latest commit adds a log file at ~/.vscode-R/diagnostics.log which shows every step from spawning to terminating. Please try the latest build and put your log here and let's see if taskkill is actually triggered in your cases.

Installed. Thanks a lot.
Tried with Rterm in "r.rpath.windows". It is ok.

Ok here the log without any set for r.path.windows and with R for Windows... processes left after closing vscode:
Process [C:\woot\opt\r\bin\R.exe] (4060) spawned
Process [C:\woot\opt\r\bin\R.exe] (5164) spawned
Process [C:\woot\opt\r\bin\R.exe] (4060) disposing, connected: false, exitCode: null, signalCode: null, killed: false, running: true
Process [C:\woot\opt\r\bin\R.exe] (5164) disposing, connected: false, exitCode: null, signalCode: null, killed: false, running: true
Process [C:\woot\opt\r\bin\R.exe] (5164) terminating
Process [C:\woot\opt\r\bin\R.exe] (5164) disposed, connected: false, exitCode: null, signalCode: null, killed: false, running: true
Process [C:\woot\opt\r\bin\R.exe] (5220) spawned
Process [C:\woot\opt\r\bin\R.exe] (13272) spawned
Process [C:\woot\opt\r\bin\R.exe] (5220) disposing, connected: false, exitCode: null, signalCode: null, killed: false, running: true
Process [C:\woot\opt\r\bin\R.exe] (13272) disposing, connected: false, exitCode: null, signalCode: null, killed: false, running: true
Process [C:\woot\opt\r\bin\R.exe] (13272) terminating
Process [C:\woot\opt\r\bin\R.exe] (13272) disposed, connected: false, exitCode: null, signalCode: null, killed: false, running: true
Process [C:\woot\opt\r\bin\R.exe] (16056) spawned
Process [C:\woot\opt\r\bin\R.exe] (9144) spawned
Process [C:\woot\opt\r\bin\R.exe] (16056) disposing, connected: false, exitCode: null, signalCode: null, killed: false, running: true
Process [C:\woot\opt\r\bin\R.exe] (9144) disposing, connected: false, exitCode: null, signalCode: null, killed: false, running: true

Ok. Next try. I manually killed all the processes (R.exe and Rterm.exe) and started again vscode, played a bit and closed it.
Here the log and the process.
Process [C:\woot\opt\r\bin\R.exe] (10760) spawned
Process [C:\woot\opt\r\bin\R.exe] (15664) spawned
Process [C:\woot\opt\r\bin\R.exe] (10760) disposing, connected: false, exitCode: null, signalCode: null, killed: false, running: true
Process [C:\woot\opt\r\bin\R.exe] (15664) disposing, connected: false, exitCode: null, signalCode: null, killed: false, running: true
Process [C:\woot\opt\r\bin\R.exe] (15664) terminating
Process [C:\woot\opt\r\bin\R.exe] (15664) disposed, connected: false, exitCode: null, signalCode: null, killed: false, running: true

The processes 10144 and 15424 left behind are not in the log. Interesting.

Another day, other thoughts. ;)
Using R.exe, it is very easy to reproduce the issue: start vscode, edit a an R file, close vscode and give a look at details of process explorer: there are leaked R.exe and Rterm.exe processes.
If taskkill approach is correct (I think so), just putting apart the comments about RTerm.exe, I was wondering if the problem is not HOW to dispose the resource, but WHEN. I don't know the lifecycle of a vscode extension, but is it plausible to say there is some sort of race condition when closing a project (or switching to another project), that the disposing function have no time to complete the task?

Thanks for the thorough testing!
In my testing, the log looks quite weird:
Process [C:\Program Files\R\R-4.1.2\bin\R.exe] (10340) spawned
Process [C:\Program Files\R\R-4.1.2\bin\R.exe] (11000) spawned
Process [C:\Program Files\R\R-4.1.2\bin\R.exe] (10340) disposing, connected: false, exitCode: null, signalCode: null, killed: false, running: true
Process [C:\Program Files\R\R-4.1.2\bin\R.exe] (11000) disposing, connected: false, exitCode: null, signalCode: null, killed: false, running: true

The terminating and disposed status never appear, but the R.exe and other processes are indeed terminated. Looks like they are not killed by the taskkill code but something else?

Here trying to debug the extension, just to understand why log lines, terminating and disposed, are not printed.
I am sorry for my limited knowledge.
If an extension needs to dispose a resource, it needs to add the disposing function to the so called extension subscriptions; then, when vscode exit, or the extension is unloaded, those functions are called and the objects are disposed.
Basically we have in util.ts:

asDisposable function to associate the object to dispose to a function and to add it to the subscriptions
exec function to execute a spawned process
const disposable = asDisposable(proc, () => { inside exec function as disposing handler

Eventually exec is called in languageService.ts to start R language server.
If I set a breakpoint inside the disposing handler (where there are the log lines), for example just after the taskkill line, the execution is halted at the breakpoint. So far it is ok. What I noticed after a few seconds the execution is halted and, maybe, while I had been exploring the values of the variables, suddenly the debugger is stopped and the whole process exits.
So I was wondering if there is an high-order logic about disposing lifecycle, limiting the time where the disposing function in the subscription is executed. I tried to give a look at vscode docs, but, weirdly, I didn't find any relevant info about this topic.
I say it in the wrong way: like when the callback of a promise is not executed until the end, because the main process is suddenly terminated.
Also this would explain the different behaviours for the different users (R.exe vs. Rterm, the printed log lines): it depends how fast the local machines are and it depends on the context of the language server (loaded files and so on).
Basically I am afraid the disposing logic is not robust, but I don't see the failure point (in the extension or in vscode).

Thanks for opening the issue in the "upstream".
Just after a few hours, it finally seems it is not an isolated case.
Anyway my next steps are starting from the hello world extension and adding the basic disposing logic, hoping to highlight some difference with this case.

Breaking News. The fix is "easy". Use the sync version of spawn for taskkill:
cp.spawnSync('taskkill', ['/pid', proc.pid.toString(), '/f', '/t']);
otherwise the dispose function is terminated, when quitting vscode, before the spawned taskkill killed all the subprocesses.
Details later.

Thanks for the fix! I tested it and it works magically 👍

Perfect! Thanks @renkun-ken for this awesome extension.
It works with all scenarios of testing (switching project, creating terminals, opening editors and so on) and configurations (with or without r.path.windows set). No orphans processes in sights.
At least for me, I hope other users can confirm it.
TLDR; Troubleshooting History
Starting from these quotes:
From @renkun-ken

Looks like in your cases, the taskkill command might be never triggered as the R.exe process exits on its own before we got a chance to kill its tree.

From @albertosantini

I say it in the wrong way: like when the callback of a promise is not executed until the end, because the main process is suddenly terminated.

I cloned the hello world extension  and replaced the content of extension.ts fil with this code:
// The module 'vscode' contains the VS Code extensibility API
// Import the module and reference it with the alias vscode in your code below
import * as vscode from 'vscode';

import * as cp from 'child_process';

let extensionContext: vscode.ExtensionContext;

// this method is called when your extension is activated
// your extension is activated the very first time the command is executed
export function activate(context: vscode.ExtensionContext) {
    // Use the console to output diagnostic information (console.log) and errors (console.error)
    // This line of code will only be executed once when your extension is activated
    console.log('Congratulations, your extension "helloworld-sample" is now active!');

    // assign extension context to global variable
    extensionContext = context;

    // The command has been defined in the package.json file
    // Now provide the implementation of the command with registerCommand
    // The commandId parameter must match the command field in package.json
    const disposable = vscode.commands.registerCommand('extension.helloWorld', () => {
        // The code you place here will be executed every time your command is executed

        // Run R executable
        startR();
    });

    context.subscriptions.push(disposable);
}

function startR() {
    const command = 'C:\\woot\\opt\\r\\bin\\R.exe';
    // const command = 'C:\\woot\\opt\\r\\bin\\x64\\R.exe';
    // const command = 'C:\\woot\\opt\\r\\bin\\x64\\Rterm.exe';
    const proc = cp.spawn(command, ["-q", "-e", "Sys.sleep(300)"]);

    vscode.window.showInformationMessage(`Proc started: ${proc.pid}`);

    asDisposable(proc, () => {
        cp.spawnSync('taskkill', ['/pid', proc.pid.toString(), '/f', '/t']);
    });
}

function asDisposable<T>(toDispose: T, disposeFunction: (...args: unknown[]) => unknown): T & vscode.Disposable {
    type disposeType = T & vscode.Disposable;
    (toDispose as disposeType).dispose = () => disposeFunction();
    extensionContext.subscriptions.push(toDispose as disposeType);

    return toDispose as disposeType;
}

My workflow was starting a debugging session of the extension, opening the command palette, executing Hello World command, which started an R command (sleeping for 5 minutes); then I closed the session and with the task manager opened I saw the results: orphans or not orphans processes.
I played with different approach: with spawn or with exec, with libraries killing processes (fkill, tree-kill, taskkill, etc.), with another native kill executable (pskill). Just to understand if the problem was related with R.exe (32bit or 64bit) or Rterm.exe, with the way I killed the processes, with the disposing logic of the extension or vscode.
During the journey, I learned you cannot use any of the libs out there, to kill a tree of processes in the disposing function, because all use the async approach. Generally speaking, in the disposing function you need to use only a sync approach.

Great finding and the details!
If @Yunuuuu could confirm it also works, we could merge this PR (again).

Perfect! Thanks for the excellent method! I have tested it can work for both Rterm.exe and R.exe


CentOS 6.6 Tmux

Not sure if this is possible on 6.6 but I'm attempting to start a GMOD server. I get the following error:
[ FAIL ] Starting gmod-server: Tmux not installed
* Tmux is required to run this server.
* Please see the the following link.
* http://gameservermanagers.com/tmux-not-found
But when I attempt to install TMUX it tells me that it is already installed as follows:
Loaded plugins: fastestmirror, refresh-packagekit
Setting up Install Process
Loading mirror speeds from cached hostfile

base: centos.mirrors.ovh.net
extras: centos.mirrors.ovh.net
updates: centos.mirrors.ovh.net
No package tmux available.

Is it simply not possible for me or am I being a fool? Thanks for any help.

Hi,
Sorry to hear that.
 It seems like you did everything right to try diagnosing it.
Did you ever have a server running well previously ? If yes,could it possibly be related to this ?
https://github.com/dgibbs64/linuxgsm/issues/627
Could you please give me the output of that command ?
command -v tmux

I didn't actually have a server running before so it's not related to that post you linked. In terms of that command, I'm not sure what that command actually is or how to run it. When it comes to SSH and Linux I'm rather new so I'm unsure what you're talking about, could you perhaps give me the exact command to input?

I'll give you a few commands to test, just to see what it answers. These commands are just meant to see if tmux is installed, and are initially runned by LGSM. Maybe LGSM could detect that tmux isn't installed while it is.
All the commands i'll copy/paste you are those at the line of the code i linked, they're looking like that : [ ..test-tmux-code... ] || [ ...test-tmux-2nd-code... ]
You can actually try them in your ssh command prompt and copy/paste here what each one answers.
If you don't get it i'll give you the commands to try tomorrow but right now i'm on my smartphone so it's pretty hard to do :p

source /etc/profile
See if you just aren't loading the Bash profile. Then
rpm -qa | grep -i tmux
and see what you have installed. You can use rpm -ql $PACKAGE to get a full list of what it installs, which would be your missing executable. If you don't see it, run
yum whatprovides */tmux
See what packages are there.

So in the end my question was just : Do you actually have tmux installed ?
As it's been 10 days without an answer, i'm closing this for now. Feel free to re-open if needed, or just post your solution to the issue to hopefully help other poeple.

Sorry for the delayed response been rather busy lately. In the end I couldn't get it to work at all so I just did it the regular way and made a new user installed steamcmd etc and just setup a server through steam commands and managed to get it all working.

Your error message sounds like tmux isn't available for you more than already installed. I'd suggest posting in centos forums to get it sorted out.
Could either be a mirror issue or an apt config issue, definitely not a script issue in my opinion.
BTW, have a nice admin time !

Not a clue, all I know is I followed the steps for this and for whatever reason it didn't work but when I did it through steamcmd it did. Thanks for all the support you've provided anyway though :)

The only issue with using steamcmd is if you're not using any kind of container as tmux or screen. Then either you won't be able to see your console and to control it, or you will be trapped, not being able to quit your console without closing the server. Hopefully you're using screen.
You're welcome :)


NIFI-8024 Added null claim check to EncryptedFileSystemRepository.read()

Description of PR
NIFI-8024 Added check for null ContentClaim in EncryptedFileSystemRepository.read() to avoid EOFException in following method implementation.  The standard FileSystemRepository.read() handles null ContentClaims, but the Encrypted implementation attempts to decrypt the contents.  This update brings the encrypted implementation in line with the standard implementation and adds a unit test for verification.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[X] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[X] Has your PR been rebased against the latest commit within the target branch (typically main)?

[X] Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

[X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[X] Have you written or updated unit tests to verify your changes?
[ ] Have you verified that the full build is successful on JDK 8?
[X] Have you verified that the full build is successful on JDK 11?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.

Thanks for fixing that @exceptionfactory! Code change makes sense to me. Looks good. Unit tests verify behavior. +1 merged to main.


rand_vms.c mod for VMS V8.3 and earlier

[.openssl-1_1_1b.crypto.rand]rand_vms.c
rand_pool_add_nonce_data()
#if __CRTL_VER >= 80400000
    sys$gettim_prec(&data.time);
#else
    sys$gettim((void*)&data.time);
#endif

need to be extended to the use of
sys$gettim_prec(&data.time);

in rand_pool_add_additional_data()

Strange. Why hasn't this been noticed when your issue #7229 ws fixed by pull request #7230?
Hang on for a second, fix is on the way...

Please try #8488 and let me know whether it fixes your issue.


Issue 241

Issue: #241
Approach

Generate a static code.json file in the _data directory with the script src\_plugins\code_json_generate_once
added the script src\_plugins\data_page_generator.rb taken from https://github.com/avillafiorita/jekyll-datapage_gen this script will generate individual pages per project
Updated navigation
Created a projects page

Please review

@david-saeger Why did we close this PR? Do you think it is not viable as an option to resolve the user-friendly project inventory?


kodi: fix build

Description of changes

while working on #210802 i noticed that our kodi build is failing
i did a quick search and found this as a resolution
seeing as the next major release of kodi this patch won't be needed for long
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

cc @dasj19 @minijackson


[microTVM] [WIP] Optimized assembly schedules for Cortex-M convolution

I originally intended this to be a standalone PR, but I'm probably going to turn this work into a schedule for quantized convolution.

Closing this because its changes are incorporated by #13242.


Add workspace (state?) to HAL command buffers?

Request description
CUDA graphs are built around CUgraph (host memory DAG of commands) and CUgraphExec (executable graph instantiated on device), and cuGraphInstantiate is an expensive oepration. CUDA has APIs for updating CUgraphExec with a new CUgraph (cuGraphExecUpdate) and a family of cuGraphExec...Set APIs for updating individual nodes in the executable graph.
However in current HAL command buffer instantiated into CUgraphExec, and discarded when command buffer is destroyed, which makes efficient implementation impossible.
Let's  add an optional workspace to command buffers?

util.global @_cmd_buf_worskspace:  !hal.command.buffer.workspace

func.func @main() {
%w = util.global.load @_cmd_buf_worskspace : !hal.command.buffer.workspace
%cmd_buf = hal.command_buffer.create_with_workspace(%w)
}

When lowering from Stream to HAL for backends that support command buffer workspaces compiler will add one workspace for each created command buffer, and HAL implementation will be free to use it to persist any state it needs between invocation. In CUDA CUgraphExec + some metadata about nodes can be persisted in the workspace, so that any new command buffer recording can use cuGraphExecHostNodeSetParams to efficiently update parameters.
What component(s) does this issue relate to?
Runtime
Additional context
No response

As discussed in Discord chat the correct solution is to use command buffers stored in globals and binding tables.


Property encrypt.key in version Dalston.SR2+ is not effect

when I update spring cloud version from Dalston.SR1 to Dalston.SR2,
curl localhost:9010/encrypt/status
{
    "description":"No key was installed for encryption service",
    "status":"NO_KEY"
}

I have set encrypt.key=test.
May be it is a bug

@Configuration
@EnableConfigurationProperties(KeyProperties.class)
public class EncryptionAutoConfiguration {

	@ConditionalOnMissingBean(TextEncryptor.class)
	protected static class DefaultTextEncryptorConfiguration {

		@Autowired
		private KeyProperties key;  #   **=> here is null in Dalston.SR2**

		@Autowired(required = false)
		private TextEncryptorLocator locator;

		@Bean
		public TextEncryptor defaultTextEncryptor() {
			if (locator != null) {
				return new LocatorTextEncryptor(locator);
			}
			if (StringUtils.hasText(this.key.getKey())) {
				return new EncryptorFactory().create(this.key.getKey());
			}
			return Encryptors.noOpText();
		}

	}

...

If you put it in bootstrap.properties does it work?

Also, there were issues fixed around this in SR3, so please try that.

@ryanjbaxter put it in bootstrap.properties ， it also does not work.
This way KeyProperties.key is correct. But this.encryptor in  EncryptionController  is null.
@RestController
@RequestMapping(path = "${spring.cloud.config.server.prefix:}")
public class EncryptionController {
...
   private void checkEncryptorInstalled(String name, String profiles) {
   	if (this.encryptor == null
   			|| this.encryptor.locate(this.helper.getEncryptorKeys(name, profiles, ""))
   					.encrypt("FOO").equals("FOO")) {
   		throw new KeyNotInstalledException();
   	}
   }
...
}

@spencergibb update to SR3 also has this problem


As a developer, I want failed brakeman notices to fail my travis tests

I'll pick this up.

Hi @pachonk is this the one? https://github.com/presidentbeef/brakeman

That's the one.


Sim crashes on program with picture

Describe the bug

Assigning the values for the variable "hase" in the loop for further processing works only once.

To Reproduce
Steps to reproduce the behavior:

Go to https://lab.open-roberta.org --> Calliope mini
Code the example program
Start the progam and process it more then two times
See error

Expected behavior

Assigning the variable "hase" should work all the time. No matter how many times the loop is run..

Screenshots

Device information

Type Laptop
OS macbook
Browser Chorme

Indeed its SIM-related!
Program as .txt file ;-)
Debug_Hase_Prog.txt

looks like with the third iteration an variable "k0" is added, which has no value, so the error occurs ...

While working on this problem, it turned out that we have a similar problem with the mbot image simulation. But here the hardware behaves like the simulation so the question is whether we should change both (hardware and simulation for mbot) to make it similar to Calliope?


Add accounts middleware

Check with ocis-accounts what rolves and permissions an authenticated user has and

add roles or
deny access

@butonic this is an old issue that was transferred from the archived ocis-pkg repo a few weeks ago.
Please close if no longer relevant.


build a model runtime simulator to help measure elastic scaling efficiency with little system resource.

measure elastic scaling efficiency
A set of composite metrics including QPS(query per second), Error, ART(average response time) and so on. So we can determine the new optimization is valid or not.
runtime simulator
the runtime load the yaml  file as model. this yaml file declare the resource that model should consume. blew is a example of yaml file.
resource:
  memory: 1M
  cpu: 5
time:
  load: 5 # second
  predict: 5 # second

@meijies it looks like this is a general model-mesh related issue. You opened it in the adapter repo which is one specific component, so I'll move it to the main repo.


DefaultViewLocator doesn't resolve IViewFor

According to the comments in DefaultViewLocator, it looks for IViewFor for a ViewModel of type FooBarViewModel. This doesn't appear to be the case, so the test below fails
[Fact]
public void ResolveByInterfaceType()
{
    var resolver = new ModernDependencyResolver();

    resolver.InitializeSplat();
    resolver.InitializeReactiveUI();
    resolver.Register(() => new FooBarView(), typeof(IViewFor<IFooBarViewModel>));

    using (resolver.WithResolver()) {
        var fixture = new DefaultViewLocator();
        var vm = new FooBarViewModel(null);

        var result = fixture.ResolveView(vm);
        this.Log().Info(result.GetType().FullName);
        Assert.IsType<FooBarView>(result);
    }
}

namespace Foobar.ViewModels
{
    public interface IFooBarViewModel : IRoutableViewModel {}

    public class FooBarViewModel : ReactiveObject, IFooBarViewModel
    {
        public string UrlPathSegment { get { return "foo"; } }
        public IScreen HostScreen { get; private set; }

        public FooBarViewModel(IScreen hostScreen)
        {
            HostScreen = hostScreen;
        }
    }
}

namespace Foobar.Views
{
    using ViewModels;

    public class FooBarView : IViewFor<IFooBarViewModel>
    {
        object IViewFor.ViewModel { get { return ViewModel; } set { ViewModel = (IFooBarViewModel) value; } }
        public IFooBarViewModel ViewModel { get; set; }
    }
}

I've had a crack at fixing it, but the only way I can see is to attempt to resolve an IViewFor for every interface the ViewModel implements and that seems a little ugly (it might be reasonable to restrict that to interfaces that implement IRoutableViewModel which is a little better though?).

I'm confused as to what you're trying to do differently than this test. https://github.com/reactiveui/ReactiveUI/blob/master/ReactiveUI.Tests/RxRouting.cs#L66-L82
That one passes, why doesn't yours?

That test is resolving a view model by building a name for BazViewModel, it puts an I on the front, and replaces ViewModel with View, rather than doing anything more intelligent with the view types/implemented interfaces
There are three cases listed in the ViewLocator (https://github.com/reactiveui/ReactiveUI/blob/master/ReactiveUI/ViewLocator.cs#L49-L51)

IFooBarView that implements IViewFor - what the above test cases covers, and what https://github.com/reactiveui/ReactiveUI/blob/master/ReactiveUI/ViewLocator.cs#L60-L63 does
IViewFor - what this issue is about, that doesn't seem to be handled anywhere
IViewFor (the original behavior in RxUI 3.1) - this is handled by https://github.com/reactiveui/ReactiveUI/blob/master/ReactiveUI/ViewLocator.cs#L65-L67 but doesn't have any test coverage

I've expanded the example a little to make it clearer, but the particular case that isn't working for me is that

I have a ViewModel that implements an IViewModel
I have a view that implements IViewFor
The view is registered with Splat/the service locator
The view isn't resolved for an instance of the ViewModel

Thanks for the high quality bug report @olevett!
We have been talking in slack about it, specifically the fact you submitted a failing test as part of the initial bug report and that you proceeded to resolve the issue via a PR.
You should come and join us in Slack, see the README.md for details - for now I've sent you an invitation to join the ReactiveUI community which when accepted will provide the ability to display the ReactiveUI badge on your github profile.

RxUI 7 Navigation Bug
I Have attached a repo of this BUG, the DefaultViewLocator fails to resolve an IViewFor<INTERFACE>
Included in the repo is a potential fix or another way to try and resolve the view I have named it SplatViewLocator.
I have also included an extension method that might be useful when using Routing I have Extended from IScreen however it may be more flexable to extend from the RoutingState therefore allowing multiple routers within an IScreen.
Open the solution and run to see the exception
Open the AppBootstrapper and change UseDefaultViewLocator to false to run with the fix.
DefaultViewLocatorTest.zip

I think the fix I had is on master but not the RxUI7 branch, not sure exactly why that is however...

Do we have any info on when this will be released?


incompatible-pointer-types in c_generated_functions.c

GCC 14 complains about incompatible pointer types:
[   13s] + dune installed-libraries
[   13s] bigarray               (version: 4.14.2)
[   13s] bigarray-compat        (version: 1.1.0)
[   13s] bytes                  (version: 4.14.2)
[   13s] compiler-libs          (version: 4.14.2)
[   13s] compiler-libs.bytecomp (version: 4.14.2)
[   13s] compiler-libs.common   (version: 4.14.2)
[   13s] compiler-libs.optcomp  (version: 4.14.2)
[   13s] compiler-libs.toplevel (version: 4.14.2)
[   13s] ctypes                 (version: 0.23.0)
[   13s] ctypes-foreign         (version: 0.23.0)
[   13s] ctypes.stubs           (version: 0.23.0)
[   13s] ctypes.top             (version: 0.23.0)
[   13s] dynlink                (version: 4.14.2)
[   13s] integers               (version: 0.7.0)
[   13s] integers.top           (version: 0.7.0)
[   13s] ocamldoc               (version: [distributed with OCaml])
[   13s] raw_spacetime          (version: 4.14.2)
[   13s] seq                    (version: 4.14.2)
[   13s] stdlib                 (version: 4.14.2)
[   13s] stdlib-shims           (version: 0.3.0)
[   13s] str                    (version: 4.14.2)
[   13s] threads                (version: 4.14.2)
[   13s] threads.none           (version: 4.14.2)
[   13s] threads.posix          (version: 4.14.2)
[   13s] uchar                  (version: 4.14.2)
[   13s] unix                   (version: 4.14.2)

[   15s] File "_build/.dune/default/src/c/dune", line 12, characters 9-30:
[   15s] 12 |   (names c_generated_functions helpers)
[   15s]               ^^^^^^^^^^^^^^^^^^^^^
[   15s] Command [137] exited with code 1:
[   15s] $ (cd _build/default/src/c && /usr/bin/gcc-14 -O2 -fno-strict-aliasing -fwrapv -pthread -Werror=implicit-function-declaration -Werror=return-type -Wno-deprecated-declarations -fPIC -pipe -D_FILE_OFFSET_BITS=64 -O2 -fno-strict-aliasing -fwrapv -pthread -Werror=implicit-function-declaration -Werror=return-type -Wno-deprecated-declarations -fPIC -pipe -g -I /usr/lib64/ocaml -I /usr/lib64/ocaml/bigarray-compat -I /usr/lib64/ocaml/ctypes -I /usr/lib64/ocaml/integers -I /usr/lib64/ocaml/stdlib-shims -I /usr/lib64/ocaml/threads -I . -o c_generated_functions.o -c c_generated_functions.c)
[   15s] c_generated_functions.c: In function ‘luv_stub_156_luv_get_getnameinfo_trampoline’:
[   15s] c_generated_functions.c:1177:4: error: initialization of ‘void (*)(uv_getnameinfo_t *, int,  const char *, const char *)’ {aka ‘void (*)(struct uv_getnameinfo_s *, int,  const char *, const char *)’} from incompatible pointer type ‘luv_getnameinfo_cb’ {aka ‘void (*)(struct uv_getnameinfo_s *, int,  char *, char *)’} [-Wincompatible-pointer-types]
[   15s]  1177 |    luv_get_getnameinfo_trampoline();
[   15s]       |    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[   15s] c_generated_functions.c: In function ‘luv_stub_157_luv_getnameinfo’:
[   15s] c_generated_functions.c:1189:43: error: passing argument 3 of ‘luv_getnameinfo’ from incompatible pointer type [-Wincompatible-pointer-types]
[   15s]  1189 |    int x954 = luv_getnameinfo(x947, x948, x949, x950, x951);
[   15s]       |                                           ^~~~
[   15s]       |                                           |
[   15s]       |                                           void (*)(uv_getnameinfo_t *, int,  const char *, const char *) {aka void (*)(struct uv_getnameinfo_s *, int,  const char *, const char *)}
[   15s] In file included from c_generated_functions.c:8:
[   15s] helpers.h:177:64: note: expected ‘luv_getnameinfo_cb’ {aka ‘void (*)(struct uv_getnameinfo_s *, int,  char *, char *)’} but argument is of type ‘void (*)(uv_getnameinfo_t *, int,  const char *, const char *)’ {aka ‘void (*)(struct uv_getnameinfo_s *, int,  const char *, const char *)’}
[   15s]   177 |     uv_loop_t *loop, uv_getnameinfo_t *req, luv_getnameinfo_cb getnameinfo_cb,
[   15s]       |                                             ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~

Thanks!
For GCC 14, it looks like Luv will have to upgrade to ctypes 0.23.0 to use its ability to generate const-correct code, or work around this otherwise, perhaps with some more casts. See:

https://github.com/yallop/ocaml-ctypes/issues/134
https://github.com/yallop/ocaml-ctypes/pull/780
https://github.com/yallop/ocaml-ctypes/pull/782

Even more tricky, this is likely caused by ctypes 0.23.0 now translating string as const char*, which is the more correct behavior, but it makes the wrappers we currently have for handling ctypes' previous behavior of translating string as char* not only redundant, but harmful, causing these new warnings when we try to bind to the previous wrappers, whose whole purpose was to cast char* to const char*. I will delete them and require ctypes 0.23.0.

Would you be able to try this with the Luv commit ad7f953fccb8732fe4eb9018556e8d4f82abf8f2, from branch getnameinfo-constness, which is linked above? It makes the build warning-clean again on my machine, with ctypes 0.23.0.

ad7f953fccb8732fe4eb9018556e8d4f82abf8f2 does indeed work. Thank you.

Thanks! Let me know if you'd like a release to opam.

Well, if you don't mind: release early, release often.
With a new release I would be able to point the Haxe people to this issue, so that they can start porting their code to 0.5.13, or newer.

https://github.com/ocaml/opam-repository/pull/26501


build error  in terget linux

I tried build Release unix/ibus/ibus.gyp:ibus_mozc  server/server.gyp:mozc_server gui/gui.gyp:mozc_tool renderer/renderer.gyp:mozc_renderer
I occurred error, but i dont understand "what file is lacked?"
please tell me.
Traceback (most recent call last):
  File "/usr/lib/python2.7/pdb.py", line 1314, in main
    pdb._runscript(mainpyfile)
  File "/usr/lib/python2.7/pdb.py", line 1233, in _runscript
    self.run(statement)
  File "/usr/lib/python2.7/bdb.py", line 400, in run
    exec cmd in globals, locals
  File "<string>", line 1, in <module>
  File "build_mozc.py", line 1257, in <module>
    main()
  File "build_mozc.py", line 1244, in main
    BuildMain(cmd_opts, cmd_args)
  File "build_mozc.py", line 875, in BuildMain
    BuildWithNinja(options, targets)
  File "build_mozc.py", line 849, in BuildWithNinja
    RunOrDie([ninja, '-C', build_arg] + ninja_targets)
  File "build_tools/util.py", line 92, in RunOrDie
    process = subprocess.Popen(argv)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
Uncaught exception. Entering post mortem debugging

Can you check if the following command succeeds or not?
ninja --version

If it doesn't success, it's probably ninja that your environment missed.

I don't have ninja and apt-get install ninja,so I moved on next step! thanks!
but build was looped.
Is this behave build success?
INFO: Running: ninja -C out_linux/Release ibus_mozc mozc_emacs_helper mozc_server mozc_tool mozc_renderer
log: ninja version 0.1.3 initializing
log: magic group: gid=0 (root)
log: entering main loop
log: generating initial pid array..
log: now monitoring process activity

and out_linux/Release is existed obj/ dir and many *.ninja files.

I am writing this to inform you that the problem is now solved. Thank you for your help.
apt-get install ninja-build

Thank you for your update.  Seems that no further action is needed on Mozc upstream side and we can close this thread.


gzip models during download

Only gzip models, to be conservative. We could decide to gzip more routes (or all the routes) in the future.
The middleware takes care of not gzipping small payloads (<200b), not gzipping the payload if the the client doesn't support it, not gzipping if the response is already compressed, etc
The client (requests.get) already adds Accept-Encoding: gzip,deflate, and decompresses accordingly if the response has Content-Encoding: gzip. So we don't need to do anything client-side.

You can check that the response is actually encoded by doing a HTTP request manually with curl (and adding Accept-Encoding: gzip), or by adding this to utils.py
enc = response.headers.get('Content-Encoding', 'unknown')
logger.info(f'Response encoding: {enc}')

tested with substra-tests: :heavy_check_mark:

need to fix unit tests :roll_eyes:


Calendar does not show days off

The calendar lets you add days off, but it doesn't show them to you on the calendar. This is straight up broken. Or useless.

Hi Alex,
I'm sorry you ran into this issue. We have just published a fix for what may be the issue you were seeing. Can you please try again and see if your days off appear?
Thanks,
Derrick

Problem solved, thank you so much for the quick turn around.
Alex
On Mon, Feb 13, 2017 at 11:31 AM, dmarcey<EMAIL_ADDRESS>wrote:

Closed #130 https://github.com/Microsoft/vsts-team-calendar/issues/130.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vsts-team-calendar/issues/130#event-960100217,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABt6JmRw0cOX_ATzKQwATyFJlSxBNCHtks5rcKGegaJpZM4L_ULX
.

--
Alex


[Tree Details - Design] Maintenance Delete Maintenance Performed (Accidents)

The solution I'm thinking of is
if we are going to go with a quick fix before we change the tree details container:

if maintenance has already occurred, the maintenance button text changes from "maintenance" to "modify maintenance" or something like that.

This would only be active for 24 hours? Then it will go back to saying "maintenance" for all users?

or we add a button for the tree history to modify session. This would then allow the user to delete or confirm changes?

if we want to add it into the new design:

once maintenance has been submitted, the tree maintenance icons would close and a message saying "thanks for caring for "TREE NAME". Under that text we have a button or linked text that would reopen the tree maintenance icons to modify the session. There would also be an additional button saying "delete session".

If "delete session" is selected, the user would then get to a screen asking "Do you really want to delete this session" which would lead to two buttons, "Yes delete" or "No, I want to keep this session".

Or we have it be modifiable from the tree history with a button stating "modify session". Which would lead to the same steps.
Or we only allow one of the two to be modifiable since they do the same function.

Maintenance Button to change to "Modify Maintenance" when session completed for 24 hours. After that the button changes back to to "maintenance" with a tree.


Add capability for FIP products

We need to add in-flight icing products for RRFS parallels. These products need no additional computation and show up in the RRFS output as:
ICPROB = icing probability
SIPD = Supercooled Large Droplet Icing
ICSEV = ICPROB separated into several thresholded categories
They're on flight levels, every 500 feet from 500 to 30000, hourly from lead times 1hr onward (the calculations need hourly accumulated precipitation). There is sample data on Jet, at the end of PRSLEV.GrbF06:
JET: /lfs4/BMC/wrfruc/Samuel.Trahan/for-graphics-team
Examples of these maps from AWC can be found here: https://www.aviationweather.gov/icing/fip


Create a new PolicyTopology React component for rendering Graphvis graphs via data in the CR. Add this component to our plugin.

Build a new React component to render Graphviz graphs via dotfiles in CRs.
Can likely use this full-featured library for actual graph rendering:
https://github.com/ts-graphviz/ts-graphviz
or vis.js:
https://viz-js.com/api/
https://viz-js.com/api/#viz
If we find we also want to look at functionality for filtering, we may also want to look at dotfile parsing:
https://github.com/dagrejs/graphlib-dot
Important: this component should have, ideally, no hooks into elsewhere in OCP console. We want this to be shippable standalone so we can embed into other UI (E.g. a standalone policy toplogy UI that users can install on any Kube flavour they like). The component should have one concern: to render dotfiles

https://github.com/Kuadrant/react-policy-topology

TODO: publish to npm

Next: add to console-plugin. Won't be able to hookup to cluster state until config-map available.


WFLY-5815  Read resource of deployment with faulty datasource fails

deployment JPA resource should marked as runtime = true, so validation works properly
jpa & undertow runtime resources should properly handle failed deployment.

@scottmarlow are you fine with JPA changes?
They are mostly just fixes for potential NPEs
and change that missing EMF doesn't throw exception but handles it gracefully.

Thanks for addressing the change @ctomc :-)
JPA changes looks good to me.

this got eap acked.


[Snyk] Security upgrade mdast from 2.3.2 to 3.0.0

Snyk has created this PR to fix 7 vulnerabilities in the npm dependencies of this project.
Snyk changed the following file(s):

package.json
package-lock.json

Vulnerabilities that will be fixed with an upgrade:

Issue

Regular Expression Denial of Service (ReDoS) SNYK-JS-ANSIREGEX-1583908

Uncontrolled resource consumption SNYK-JS-BRACES-6838727

Missing Release of Resource after Effective Lifetime SNYK-JS-INFLIGHT-6095116

Inefficient Regular Expression Complexity SNYK-JS-MICROMATCH-6838728

Regular Expression Denial of Service (ReDoS) SNYK-JS-TRIM-1017038

Prototype Pollution SNYK-JS-UNSETVALUE-2400660

Regular Expression Denial of Service (ReDoS) npm:braces:20180219

[!IMPORTANT]

Check the changes in this PR to ensure they won't cause issues with your project.
Max score is 1000. Note that the real score may have changed since the PR was raised.
This PR was automatically created by Snyk using the credentials of a real user.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Regular Expression Denial of Service (ReDoS)
🦉 Prototype Pollution

snyk-fix-17e9eb8df272b79143087d4fe61f06aa


Restrict computation of CHG and PCHG to post-baseline records only

Background Information
CHG and PCHG computation have been updated in templates and vignettes of {admiral} in order to compute them only for post- baseline records. This is what is usually expected for these 2 parameters. A note has also been added to notify that this computation rule is left to the producer choice. See issue #2484 and it's corresponding PR
Definition of Done
Procced with the same in {admiralpeds} template and vignette:
Use restrict_derivation() while deriving CHG and PCHG and add a note explaining the rational of this update.

@remikk this is a good issue for you to help with. you can follow similar to what was done for admiral core package as linked above. let me and Fanny know if any issue

@Fanny-Gautier @rossfarrugia Sure, thank you for the assignment. I will take care of the issue.

thanks @remikk - you can tag us both as reviewers once the PR is ready


Running generate_kustomization.sh fails on mac

I downloaded the helm chart via the suggested command:
helm pull timescale/timescaledb-single --untar

And then attempted to run generatore_kustomization.sh on my mac.
The following errors occured:
sed: ./kustomize/example/kustomization.yaml: No such file or directory
tr: Illegal byte sequence
tr: Illegal byte sequence
tr: Illegal byte sequence
Generating a RSA private key
..............................................................................................................++++
............++++

No only is the example directory missing but macs don't like tr.  To get around the example error I had to download the example directory manually.  And to get around the tr error, I had to add this to the generate_kustomization.sh scriptr:
export LC_CTYPE=C

This was merged before you raised the issue:
https://github.com/timescale/timescaledb-kubernetes/blob/master/charts/timescaledb-single/generate_kustomization.sh#L9
Could you tell us which version you're using?

This was already fixed, but must surely be fixed with the latest release:
https://github.com/timescale/timescaledb-kubernetes/releases/tag/v0.6.2
Can you confirm?


hipify-clang doc out of data

Seems I can't get hipify-clang work by referencing the doc here:
https://github.com/ROCm-Developer-Tools/HIP/blob/master/hipify-clang/README.md

opt/rocm/hip/bin/hipify-clang memory.cu -x cuda --cuda-path=/usr/local/cuda
LLVM ERROR: CommonOptionsParser: failed to parse command-line arguments. [CommonOptionsParser]: hipify-clang: Unknown command line argument '-x'.  Try: '/opt/rocm/hip/bin/hipify-clang -help'
hipify-clang: Did you mean '-o'?
hipify-clang: Unknown command line argument '--cuda-path=/usr/local/cuda'.  Try: '/opt/rocm/hip/bin/hipify-clang -help'
hipify-clang: Did you mean '-extra-arg=/usr/local/cuda'?

/opt/rocm/hip/bin/hipify-clang memory.cu
Error while trying to load a compilation database:
Could not auto-detect compilation database for file "memory.cu"
No compilation database found in /rocm/nwchem/src/tce/ccsd_t or any parent directory
fixed-compilation-database: Error while opening fixed database: No such file or directory
json-compilation-database: Error while opening JSON database: No such file or directory
Running without flags.
warning: /rocm/nwchem/src/tce/ccsd_t/memory.cu.hipify-tmp: 'linker' input unused [-Wunused-command-line-argument]
warning: argument unused during compilation: '-std=c++11' [-Wunused-command-line-argument]
error: unable to handle compilation, expected exactly one compiler job in ''
Error while processing /rocm/nwchem/src/tce/ccsd_t/memory.cu.hipify-tmp.

Do we use hipify-perl or hipify-clang nowadays?

In case you were wondering: You're using the wrong string.
opt/rocm/hip/bin/hipify-clang memory.cu -- -x cuda --cuda-path=/usr/local/cuda instead of
opt/rocm/hip/bin/hipify-clang memory.cu -x cuda --cuda-path=/usr/local/cuda should do the trick.

Thanks, my mistake didn't see the "--" separator.


Reading property scope from an AccessToken throws a TypeError

Bug Report
When reading the property scope from an AccessToken received with the OAuth Authorization Code Flow it throws an error.
The Twitch API returns an array for the scope property while the code assumes it's a string and runs scope.split(' ') which results in an error because the function split doesn't exist on arrays.
EDIT: After looking a bit into the API Docs, it seems like Twitch always returns an array except for the case when a token is refreshed. I did not test out the refresh case yet, but it should not be overlooked when fixing this.
EDIT2: Just tested the refresh case. Also returns an array of scopes for me. So the Twitch docs are also not 100% up to date in that case.
Code
const express = require('express');
const twitch = require('twitch');

const app = express();

const clientID = '<Your client ID>';
const clientSecret = '<Your client Secret>';
const redirectUri = 'http://localhost:8381/twitch/authorize/callback'

const scopes = ['user_read', 'channel_editor'];
// Also try with no scopes. Gives another error
// const scopes = [];

app.get('/twitch/authorize', (req, res) => {
	res.redirect(`https://id.twitch.tv/oauth2/authorize?client_id=${clientID}&redirect_uri=${redirectUri}&response_type=code&scope=${scopes.join(' ')}&force_verify=true`)
});

app.get('/twitch/authorize/callback', async (req, res) => {
	const token = await twitch.default.getAccessToken(clientID, clientSecret, req.query.code, redirectUri);

	try {
		res.send(token.scope);
	} catch (e) {
		console.log(e);
		res.send(e.stack);
	}
});

app.listen(8381);
console.log('Started. Open http://localhost:8381/twitch/authorize');

Expected behavior
It returns the array of scopes that was returned by the Twitch API.
Actual Behavior
When reading the property scope from an AccessToken received with the OAuth Authorization Code Flow it throws the following error:
TypeError: this._data.scope.split is not a function
    at AccessToken.get [as scope] (/home/<redacted>/Workspaces/<redacted>/node_modules/twitch/lib/API/AccessToken.js:63:37)

Environment

Version: 3.4.4
Node version: 10.15.1
Operating system: Linux (Ubuntu 18.04.3 LTS)

Fix released with 3.5.0.

When I did a code search there were some places where it was still initialized as a string. You might want to check those too and change them to an empty array.

I already fixed these other places in the following commits. TypeScript catches these errors and I can't release a version that doesn't build.

Ahh okay, was only seeing that one commit and assumed that TS should have already complained and you might have forgotten about it.


Add forward_missing_to @object to Colorize

Feature Request
I have submitted a PR:  https://github.com/crystal-lang/crystal/pull/13005
The origin of the PR was an effort to enhance Benchmark::IPS to make more visually engaging reports when there are a lot of items being benchmarked. Sidenote -- I will also be dropping a PR and a discussion issue for this work, once I clean up some of the hacky-bits of what I did.
i.e. it lets one do this:

Without any significant changes to the code.
Importing Ary's comment on the PR into this discussion thread:

In my mind colorize is the last thing you call before moving that object into the output. If you need to do some things with the original string, I would preserve that as a separate variable.
I think adding forward_missing_to would make things a bit harder to understand.

The impetus for adding this was for places, like the usage with Benchmark::IPS, where a string is being provided as an argument, where it would be nice for a just directly support the use of Colorize.
      ips.separator "    Base58 Version #{Base58::VERSION}".colorize.light_yellow

Of course, alternatively, one can do this:
      ips.separator "    Base58 Version #{Base58::VERSION}".colorize.light_yellow.to_s

Which obviates the need for adding forwarding to colorize.
It's definitely a judgement call on which is preferable -- the change to Colorize or the dangling to_s.
I'll adjust my patch to Benchmark in accordance with whichever way this goes.

What about having separator call to_s on its argument?

Right. The key, at least for me when I did this, was that I wanted to alter the code of Benchmark::IPS as little as possible, and adding the forwarding to colorize let me avoid dangling to_s in the benchmark script, while modestly reducing the lines that I was touching within Benchmark::IPS; I was trying to make as few changes there as possible.
If adding this forwarding seems like a bad idea, though, I am more than happy to accept that judgement, as the reality is that my additions to Benchmark::IPS, while not materially changing anything, do modestly touch many pieces of the implementation, and dropping a couple to_s in the appropriate spots is pretty minor.

I have gone ahead and yanked the dependence on it from my WIP for Benchmark::IPS. I will close this issue, and also cancel the PR. Thanks for the feedback!


Read cache internal usage

For one of our use cases we have a file that we only read from in a very random manner (its graphical data), typically we have a table at the start with information on width, height of icons and offset into the file of the icon. For best performance we have found that setting a read cache of 1 (i.e. disabled) gives the best performance.
However we have found that some internal aspects of littlefs slow down significantly when using a cache of this size (e.g. opening and closing files). Specifically the lfs_cache_cmp and lfs_cache_crc functions slow down without a read cache.  If we tweak these functions to use a separate cache we get good performance for our data, and good open and close performance (e.g. 250ms to open and close 100 files goes down to 25ms).
It might be useful if this internal read cache size could be configurable in the standard release if you feel it doesn't over complicate things.

Ah, that is interesting! Thanks for raising the issue, this feedback is really helpful.
Originally, I thought that you would always see a performance improvement as you increased your cache size, since you would have a better chance to reuse read operations. But it turns out this was a mistake. Larger caches == larger read operations, and for linked lists this can turn into reading a large amount of useless data. This is especially notable on seek operations and is probably what you're seeing.
Unfortunately right now, there's no difference between the physical read size and the cache size, so first we'll need to add a separate cache size configuration option.
But I am planning on add something to fix this when I get the chance.

I think it would be nice to have two separately defined read caches - one for the internal operations and one for the user operations.  Mainly to help with lfs_cache_cmp and lfs_cache_crc as currently they will read a byte at a time, even though they know the block size being crc'd.

The trade off there is code size vs runtime, the byte-based loops relying on the general purpose caching layer may be slower but they also require less code to implement.
When you also consider block devices where reads are bigger than 1 byte and need to be aligned appropriately, the caching can get quite complicated.
(But to be fair I haven't tried out any alternatives, I'm open to being proven wrong)


iterrows turns some columns from int64 to float64

>>> import pandas as pd
>>> pd.version.version
'0.15.2'
>>> df = pd.DataFrame({u'A': {123.0: 2.0}, u'B': {123.0: 1}})
>>> df
     A  B
123  2  1
>>> df['B'].dtype
dtype('int64')
>>> type(df.iterrows().next()[1]['B'])
<type 'numpy.float64'>

This is simply a consequence of putting together a cross-sectional that goes across dtypes and is as expected.
Anytime you slice a row from a frame and turn it into a Series this will happen. All dtypes are upcast to the common-dtype to hold them all.
In [9]: df = pd.DataFrame({u'A': {123.0: 2.0}, u'B': {123.0: 1}, 'C' : 'foo'})

In [10]: list(df.iterrows().next())[1]
Out[10]:
A      2
B      1
C    foo
Name: 123.0, dtype: object

Sorry for reopening, is there a way of iterating over rows and keeping dtype?

@iosonofabio You can use itertuples for that, if you really need to iterate over rows. There is an updated section in the docs about this: http://pandas-docs.github.io/pandas-docs-travis/basics.html#iteration (not yet in stable docs)

Thanks!
(I find it funny how much emphasis is put in pandas documentation about avoiding iterations. Of course it is slow, but people do not only use pandas for efficiency but also for simple bookkeeping :smile: )

Because people do complain about it being slow :-) And also quite some people use loops for things that are really not necessary, and even more complex to do with a loop (eg adding a number to all elements, I've seen this a lot when teaching pandas to beginners).
But you're right, in certain cases, and if you know what you're doing, nothing wrong with it.


next -> tower

Write the next pipe file (tower.pipe.json)
In the same folder
on the next_bucket

What data should it take as input?
How to feed that into a sample sheet?
Can we brute force that into quilt.pipe.json?

Tower.pipe.json kicks off the existing job targeting VIVOS-production
Creates a Quilt package first
Auto-generates spreadsheet against package
Params pull from
package


Don't know how to handle the case when connection is refused by a mail server.

Q
A

Bug report?
yes

Feature request?
no

RFC?
no

How used?
Symfony

Swiftmailer version
3.0

PHP version
7.3.16

Observed behaviour

My code just uses:  send($message, $errors);
There are 3 mail addresses set as receipents, correctly set to headers as "To".
Each receipent should receive just a 1 message, so in a summary: there are 3 messages to be sent.
Even that the connection is refused(just because there is not mail server on localhost) method send() returns 1 as a result... why?
Expected behaviour

It should allow to handle Connection refused error, or return this as a string message in $errors array.
Example

Because by default, SwiftMailer uses SpoolTransport, which implements Transport interface on its own, there is no place in StackTrace to handle an exception thrown by StreamBuffer then AbstractSmtpTransport, just because the AbstractSmtpTransport is not in the same trace as the SpoolTransport instance: one do not inherit after the second one.
Thus, I suggest to throw it higher so the users can handle it on their own or to pass the information in $errors array.
P.S: I guess I may be wrong, and please correct me if I am mistaken, thus if so, I am very sorry about wasting your time.

Hi @Arkasian i'm looking for the same solution. There's no way to check if there's any problem with the server. I found that https://swiftmailer.symfony.com/docs/plugins.html#logger-plugin but I still don't know how to use it and if it's what we're looking for.

Sure, it can help sysops because it will log all problems with server from all around the SwiftMailer, but for us, I guess it has no real usage, as we want to avoid parsing logs to receive just as simple information as that the connection was refused or something. In my opinion they should just pass that information somehow higher in errors array or just leave the exception for user to be caught.

I'm using Symfony so i could feedback the errors via Slack with swiftmailer-bundle

Tell me more about it, I don't know that, and I am using Symfony too. 3.4/4.4 with SwiftMailer.

Hi @Arkasian:
Do you have this configuration in swiftmailer declaration?
spool:          { type: memory }
If true the mails are sent at the end of the request so Symfony use this listener to send.
I had this configuration in my project but I removed it and I refactored the code to send mails directly when send method is called. This way if it fails an Exception is thrown and you can handler the error.
In my case i send this error to slack with monolog. I both cases you can send the error to slack.
Hope this helps you.

As far as I remember there is such configuration done in my project. I have to try it out. Thanks for a clue @cluster28 . I will get back to you if it will work for me too.


Remove .characters because of deprecation

@alirp88 Vapor 2 still needs to work with Swift 3 so we can't break the codebase by removing the .characters property. The only way to do it would be to add both sets of code wrapped in #if swift(>=4) macros

@tanner0101 We happy with this fix for removing the deprecation warnings?

@0xTim I will fix other PRs too if this will be ok for you guys

let me fix this, it has error!

Cool if you can get the build to go through that would be awesome. I'd hold off on the other PRs until @tanner0101 comes online just to make sure he is happy with the way of doing it, just in case there is a reason that it hasn't been done yet. Though I suspect this is because the focus has been on Vapor 3

Fixed now,

we'll fix ci when we break out the vapor 3 packages. merge and testing locally for now. thanks guys!


math_rem test fails on the i686 target

When testing jaq 2.0.0 for Void Linux, the test run fails on the i686 platform: https://github.com/void-linux/void-packages/actions/runs/12129949574/job/33819327834?pr=50494

Backtrace after recompiling tests in debug:
---- math_rem stdout ----
thread 'math_rem' panicked at /builddir/jaq-2.0.0/jaq-core/src/lib.rs:221:9:
assertion failed: out.eq(ys)
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::panic
   3: jaq_core::Filter<F>::yields
             at /builddir/jaq-2.0.0/jaq-core/src/lib.rs:221:9
   4: funs::common::yields
             at ./tests/common/mod.rs:14:5
   5: funs::common::give
             at ./tests/common/mod.rs:22:5
   6: funs::math_rem
             at ./tests/funs.rs:134:5
   7: funs::math_rem::{{closure}}
             at ./tests/funs.rs:82:14
   8: core::ops::function::FnOnce::call_once
             at /builddir/rust-1.82.0/library/core/src/ops/function.rs:250:5

Looks like this test https://github.com/01mf02/jaq/blob/1ac39f8bec824fc5851512f8f87e02113fc8b965/jaq-json/tests/funs.rs#L134-L138 hmm guessing<PHONE_NUMBER> overflows etc 32 bit system? catches error message with different number?

I also suspected that, but<PHONE_NUMBER> should fit in 32 bits just fine, and the previous tests also deal with this value without failure;
so it seems this is about the logic behind div-by-zero error catching?

jaq seems to use isize so i think<PHONE_NUMBER> won't fitt signed 32 bit, that might be the issue. Could you try change<PHONE_NUMBER> to something smaller? (and also change in the error string)

@tranzystorekk, I would be happy to accept a PR from you with the smaller numbers.

I'm wondering if switching the int repr in jaq-json to i64 would be a viable option (I saw the comment that it allows ints to easily index arrays, but it seems the trade-off isn't as good if int is also tested against arithmetic operations)

@GaetanLepage does a smaller number work?

@GaetanLepage does a smaller number work? could try #245

Well, after more testing, I found out that the problematic line was give(json!(null), "2.1 % 0 | isnan", json!(true));.
Commenting this one made the whole test succeed.

Oh ok, what does cargo run -- -n '2.1 % 0 | ., isnan, isinfinite' say?

Whereas it is
null
true
false

on aarch64-darwin

Huh, is this macOS or some other darwin based os?
@01mf02 any idea what is going on? known float behaviour?

Huh, is this macOS or some other darwin based os?

MacOS, running on an aarch64 M2 CPU, but with rosetta2 emulation of x86.

@01mf02 any idea what is going on? known float behaviour?

I'm sorry, I have no idea what is happening here. If it helps, the documentation for what should happen is here: https://doc.rust-lang.org/std/primitive.f64.html#impl-Rem-for-f64. Perhaps, @GaetanLepage, could you tell us what 2.1 / 0 | ., isnan, isinfinite yields on both architectures that you tested?

Hmm confusing, i got i M3 macbook so i guess i could install rosetta somehow. I wonder what other language implementation does, could you try this with rosetta?
$ node -p '2.1 % 0'
NaN
$ node -p '2.1 / 0'
Infinity

Perhaps, @GaetanLepage, could you tell us what 2.1 / 0 | ., isnan, isinfinite yields on both architectures that you tested?

In the meantime, our darwin builder got updated to an M4 machine.
Here are the results on this new machine:
$ cargo run -- -n '2.1 % 0 | ., isnan, isinfinite

# aarch64-darwin
null
true
false

# x86_64-darwin
null
false
false

and
$ cargo run -- -n '2.1 / 0 | ., isnan, isinfinite

# aarch64-darwin
null
false
false

# x86_64-darwin
null
false
true

@GaetanLepage: Can you post the output of rustc -o test <(echo 'fn main() { dbg!(2.1 / 0.0); }') && ./test on your two platforms, just to make sure that the problem is on the Rust side and not on the jaq side? If they yield different outputs on your two platforms, could you perhaps file an issue on the Rust issue tracker, given that multiplication and division should be consistent across platforms?

This gives me inf on all x86_64-linux, aarch64-linux, x86_64-darwin and aarch64-darwin?

Dugg a bit more
With this float_cmp
fn float_cmp(left: f64, right: f64) -> Ordering {
    if left == 0. && right == 0. {
        Ordering::Equal
    } else {
        let lb = left.to_bits() as i64;
        let rb = right.to_bits() as i64;
        dbg!(&left, &right);
        dbg!(lb, rb);
        f64::total_cmp(&left, &right)
    }
}

I get this:
$cargo run --target x86_64-apple-darwin -- -n '0.0/0.0 == 0.0%0.0'
[jaq-json/src/lib.rs:892:9] &left = NaN
[jaq-json/src/lib.rs:892:9] &right = NaN
[jaq-json/src/lib.rs:893:9] lb = -2251799813685248 # NOTE: on aarch64 this is same value as below
[jaq-json/src/lib.rs:894:9] rb =<PHONE_NUMBER>041090560
false

So i wonder if one of the NaN-values gets "normalized" etc? ex the NaN payload/flags gets zeroed or something?
Still can't reproduce with same-ish code as simple rust code.

@wader, thanks for digging! Taking on your approach, may I ask you, @GaetanLepage, to post the output of the following?
rustc -o test <(echo 'fn main() { dbg!(f64::total_cmp(&(2.1 / 0.0), &f64::INFINITY)); }') && ./test

I hope that this will yield different outputs for different architectures ... that would explain jaq's behaviour, and we could file an issue at rust-lang.

One thought that stuck me: could compiling a simple program with rustc and building via cargo differ some way that changes float behaviour? some jaq crates dependency pokes at something? different compile optimization levels?

may I ask you, @GaetanLepage, to post the output of the following?

On all architectures, I get:
[/dev/fd/63:1:13] f64::total_cmp(&(2.1 / 0.0), &f64::INFINITY) = Equal

More digging
The two different NaNs as u64 are  these:
[jaq-json/src/lib.rs:890:5] lb =<PHONE_NUMBER>3895866368
[jaq-json/src/lib.rs:891:5] rb =<PHONE_NUMBER>041090560

If one inspect them using my favorite website they are:
https://float.exposed/0xfff8000000000000 signed NaN
https://float.exposed/0x7ff8000000000000 "positive" NaN
I don't really follow where and how in the jaq code we end up with a signed NaN but f64::total_cmp will see them as no equal so maybe a solution to this and possible other NaN-differences is to do:
fn float_cmp(left: f64, right: f64) -> Ordering {
    if (left == 0. && right == 0.) || (left.is_nan() && right.is_nan()) {
        Ordering::Equal
    } else {
        f64::total_cmp(&left, &right)
    }
}

Are the more places to look for this? and it is ok to normalize NaN values like this? maybe only consider signness difference equal?

One thought that struck me: could compiling a simple program with rustc and building via cargo differ some way that changes float behaviour? some jaq crates dependency pokes at something? different compile optimization levels?

I am not aware of such behaviour. On the other hand, for integers, I know that compiling with --release disables overflow checks.

Are the more places to look for this? and it is ok to normalize NaN values like this? maybe only consider signness difference equal?

It may be OK to normalize NaN values like this, but first, I would like to understand what is going on here. I'd really like to reproduce the platform-specific difference without jaq-related code.

Let's get the big cannon out. Can you post the output of the following program?
fn main() {
    let t: f64 = "2".parse().unwrap();
    let z: f64 = "0".parse().unwrap();
    let vals = [t / z, z / z, t % z, z % z];
    for v in vals {
        println!("{v} (bytes = {})", v.to_bits());
    }
    for v1 in &vals {
        for v2 in &vals {
            print!("{:?}\t", v1.total_cmp(v2));
        }
        println!();
    }
}

On my computer, this yields:
inf (bytes =<PHONE_NUMBER>227405312)
NaN (bytes =<PHONE_NUMBER>3895866368)
NaN (bytes =<PHONE_NUMBER>3895866368)
NaN (bytes =<PHONE_NUMBER>3895866368)
Equal	Greater	Greater	Greater
Less	Equal	Equal	Equal
Less	Equal	Equal	Equal
Less	Equal	Equal	Equal

On macbook M3
$ rustc --target x86_64-apple-darwin  -o test test.rs && ./test
inf (bytes =<PHONE_NUMBER>227405312)
NaN (bytes =<PHONE_NUMBER>3895866368)
NaN (bytes =<PHONE_NUMBER>041090560)
NaN (bytes =<PHONE_NUMBER>041090560)
Equal	Greater	Less	Less
Less	Equal	Less	Less
Greater	Greater	Equal	Equal
Greater	Greater	Equal	Equal

$ rustc --target aarch64-apple-darwin  -o test test.rs && ./test
inf (bytes =<PHONE_NUMBER>227405312)
NaN (bytes =<PHONE_NUMBER>041090560)
NaN (bytes =<PHONE_NUMBER>041090560)
NaN (bytes =<PHONE_NUMBER>041090560)
Equal	Less	Less	Less
Greater	Equal	Equal	Equal
Greater	Equal	Equal	Equal
Greater	Equal	Equal	Equal

(not exactly sure how x86_64 gets executed  on arm but i guess it's transparent thru rosetta, i get same result if do arch --x86_64 ./test which seems to the explicit way of doing it)

@wader: YES! I think we've got it. Just to be sure, can you check one last thing, namely whether replacing t and z with 2.0 and 0.0 yield the same results?

You mean like this?
    let t: f64 = 2.0;
    let z: f64 = 0.0;

hmm then i get:
$  rust_float rustc --target x86_64-apple-darwin  -o test test.rs && ./test
inf (bytes =<PHONE_NUMBER>227405312)
NaN (bytes =<PHONE_NUMBER>041090560)
NaN (bytes =<PHONE_NUMBER>041090560)
NaN (bytes =<PHONE_NUMBER>041090560)
Equal	Less	Less	Less
Greater	Equal	Equal	Equal
Greater	Equal	Equal	Equal
Greater	Equal	Equal	Equal

$  rust_float rustc --target aarch64-apple-darwin  -o test test.rs && ./test
inf (bytes =<PHONE_NUMBER>227405312)
NaN (bytes =<PHONE_NUMBER>041090560)
NaN (bytes =<PHONE_NUMBER>041090560)
NaN (bytes =<PHONE_NUMBER>041090560)
Equal	Less	Less	Less
Greater	Equal	Equal	Equal
Greater	Equal	Equal	Equal
Greater	Equal	Equal	Equal

Now i'm confused. After some poking around it seems this is what differs:
$ rustc --target x86_64-apple-darwin -o test <(echo 'fn main() { dbg!(("0".parse::<f64>().unwrap() / "0".parse::<f64>().unwrap()).to_bits()); }') && ./test
[/dev/fd/11:1:13] ("0".parse::<f64>().unwrap() / "0".parse::<f64>().unwrap()).to_bits() =<PHONE_NUMBER>3895866368

$ rustc --target x86_64-apple-darwin -o test <(echo 'fn main() { dbg!((0.0f64 / 0.0f64).to_bits()); }') && ./test
[/dev/fd/11:1:13] (0.0f64 / 0.0f64).to_bits() =<PHONE_NUMBER>041090560

but i don't see any difference in float etc:
$ rustc --target aarch64-apple-darwin -o test <(echo 'fn main() { dbg!("0".parse::<f64>().unwrap().to_bits()); }') && ./test
[/dev/fd/11:1:13] "0".parse::<f64>().unwrap().to_bits() = 0

$t rustc --target aarch64-apple-darwin -o test <(echo 'fn main() { dbg!(0.0f64.to_bits()); }') && ./test
[/dev/fd/11:1:13] 0.0f64.to_bits() = 0

Hmm is this the compiler doing some transformations etc?

@wader, great! So parsing seems to have an impact here.
I opened a bug report on rust-lang. Let's see what people have to say there.

The aarch64 and x86-64 CPUs use different NaNs.

This is concealed (LLVM normalizes to a particular NaN, irrespective of architecture) if you use a pattern which does not conceal the NaN. It is not if you don't.

I think this was misdiagnosed/mis-minimized, and the actual bug here is that jaq is using total_cmp. NaN generation in Rust is effectively nondeterminstic (https://rust-lang.github.io/rfcs/3514-float-semantics.html or https://doc.rust-lang.org/stable/std/primitive.f32.html#nan-bit-patterns) and since total_cmp cares about which NaN is being compared, you're going to have problems like this one.
I don't know what your floating point comparison semantics are supposed to be, but I think you want all NaNs to compare less than all NaNs, or something like that.

It seems you attempted to use total_cmp as a way of getting out of checking for NaN, but you should just do this:
fn float_cmp(left: f64, right: f64) -> Ordering {
    if left.is_nan() & right.is_nan() {
        Ordering::Equal
    } else {
        left == right
    }
}

I don't know what your floating point comparison semantics are supposed to be, but I think you want all NaNs to compare less than all NaNs, or something like that.

This is actually what happens in jq. I have just made a PR that proposes to adopt this behaviour after all.

@workingjubilee, thanks for your suggestion, that's what I settled on in the end.


chore(firestore-send-email) Content encoding

Fixes https://github.com/firebase/extensions/issues/69
Users can chose to opt-out of HTML-escapes by providing triple backslashes over double. Copy is from the handlebars documentation.

Closing, see https://github.com/firebase/extensions/issues/69#issuecomment-552965185Clos for more info.


Add docs

Hey there, this project seems interesting but it's hard to tell whether it is usable or how to use it. a README and documentation would be very helpful. Thanks!

Hey, thanks for the interest. Yeah, I've been meaning to get on that... I'll try to get something out explaining what's going on in the code this weekend!


[Favour.me] Add new extractor (Closes #9435)

Added extractor for favour.me. The extractor also works for likeafool.com, so it closes #9435.

Tests failed for Python 2.7 and 3.5. Did you really test before opening pull requests?
$ python2 test/test_download.py TestDownload.test_FavourMe{,_1}
[FavourMe] 13814: Downloading webpage
[info] Writing video description metadata as JSON to: 13814.info.json
[debug] Invoking downloader on u'http://video.favour.me/814/this-mom039s-experiment-proves-that-bottled-water-is-not-as-quothealthyquot-as-it-claims_13814.mp4'
[download] Destination: 13814.mp4
[download] 100% of 10.00KiB in 00:00
F[FavourMe] 13799: Downloading webpage
[info] Writing video description metadata as JSON to: 13799.info.json
[debug] Invoking downloader on u'http://video.favour.me/799/moose-unexpectedly-gives-birth-to-twins_13799.mp4'
[download] Destination: 13799.mp4
[download] 100% of 10.00KiB in 00:00
F
======================================================================
FAIL: test_FavourMe (__main__.TestDownload)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_download.py", line 209, in test_template
    expect_info_dict(self, info_dict, tc.get('info_dict', {}))
  File "/home/yen/Executables/Multimedia/youtube-dl/test/helper.py", line 177, in expect_info_dict
    expect_dict(self, got_dict, expected_dict)
  File "/home/yen/Executables/Multimedia/youtube-dl/test/helper.py", line 173, in expect_dict
    expect_value(self, got, expected, info_field)
  File "/home/yen/Executables/Multimedia/youtube-dl/test/helper.py", line 167, in expect_value
    'Invalid value for field %s, expected %r, got %r' % (field, expected, got))
AssertionError: Invalid value for field title, expected u'This Mom\'s Experiment Proves That Bottled Water Is Not As "Healthy" As It Claims', got u'This Mom\'s Experiment Proves That Bottled Water Is Not As "Healthy" As It Claims '

======================================================================
FAIL: test_FavourMe_1 (__main__.TestDownload)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_download.py", line 209, in test_template
    expect_info_dict(self, info_dict, tc.get('info_dict', {}))
  File "/home/yen/Executables/Multimedia/youtube-dl/test/helper.py", line 177, in expect_info_dict
    expect_dict(self, got_dict, expected_dict)
  File "/home/yen/Executables/Multimedia/youtube-dl/test/helper.py", line 173, in expect_dict
    expect_value(self, got, expected, info_field)
  File "/home/yen/Executables/Multimedia/youtube-dl/test/helper.py", line 167, in expect_value
    'Invalid value for field %s, expected %r, got %r' % (field, expected, got))
AssertionError: Invalid value for field title, expected u'Moose Unexpectedly Gives Birth To Twins', got u'Moose Unexpectedly Gives Birth To Twins '

----------------------------------------------------------------------
Ran 2 tests in 5.700s

FAILED (failures=2)

$ python3 test/test_download.py TestDownload.test_FavourMe{,_1}
[FavourMe] 13814: Downloading webpage
[info] Writing video description metadata as JSON to: 13814.info.json
[debug] Invoking downloader on 'http://video.favour.me/814/this-mom039s-experiment-proves-that-bottled-water-is-not-as-quothealthyquot-as-it-claims_13814.mp4'
[download] Destination: 13814.mp4
[download] 100% of 10.00KiB in 00:00
F[FavourMe] 13799: Downloading webpage
[info] Writing video description metadata as JSON to: 13799.info.json
[debug] Invoking downloader on 'http://video.favour.me/799/moose-unexpectedly-gives-birth-to-twins_13799.mp4'
[download] Destination: 13799.mp4
[download] 100% of 10.00KiB in 00:00
F
======================================================================
FAIL: test_FavourMe (__main__.TestDownload)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_download.py", line 209, in test_template
    expect_info_dict(self, info_dict, tc.get('info_dict', {}))
  File "/home/yen/Executables/Multimedia/youtube-dl/test/helper.py", line 177, in expect_info_dict
    expect_dict(self, got_dict, expected_dict)
  File "/home/yen/Executables/Multimedia/youtube-dl/test/helper.py", line 173, in expect_dict
    expect_value(self, got, expected, info_field)
  File "/home/yen/Executables/Multimedia/youtube-dl/test/helper.py", line 167, in expect_value
    'Invalid value for field %s, expected %r, got %r' % (field, expected, got))
AssertionError: 'This[16 chars]nt Proves That Bottled Water Is Not As "Healthy" As It Claims' != 'This[16 chars]nt Proves That Bottled Water Is Not As "Healthy" As It Claims '
- This Mom's Experiment Proves That Bottled Water Is Not As "Healthy" As It Claims
+ This Mom's Experiment Proves That Bottled Water Is Not As "Healthy" As It Claims
?                                                                                +
 : Invalid value for field title, expected 'This Mom\'s Experiment Proves That Bottled Water Is Not As "Healthy" As It Claims', got 'This Mom\'s Experiment Proves That Bottled Water Is Not As "Healthy" As It Claims '

======================================================================
FAIL: test_FavourMe_1 (__main__.TestDownload)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/test_download.py", line 209, in test_template
    expect_info_dict(self, info_dict, tc.get('info_dict', {}))
  File "/home/yen/Executables/Multimedia/youtube-dl/test/helper.py", line 177, in expect_info_dict
    expect_dict(self, got_dict, expected_dict)
  File "/home/yen/Executables/Multimedia/youtube-dl/test/helper.py", line 173, in expect_dict
    expect_value(self, got, expected, info_field)
  File "/home/yen/Executables/Multimedia/youtube-dl/test/helper.py", line 167, in expect_value
    'Invalid value for field %s, expected %r, got %r' % (field, expected, got))
AssertionError: 'Moose Unexpectedly Gives Birth To Twins' != 'Moose Unexpectedly Gives Birth To Twins '
- Moose Unexpectedly Gives Birth To Twins
+ Moose Unexpectedly Gives Birth To Twins
?                                        +
 : Invalid value for field title, expected 'Moose Unexpectedly Gives Birth To Twins', got 'Moose Unexpectedly Gives Birth To Twins '

----------------------------------------------------------------------
Ran 2 tests in 3.799s

FAILED (failures=2)

@yan12125 Was tired and pushed and squashed a few commits without checking them first. I'll remember to test before pushing next time.

There should be a standalone extractor for handling URLs like http://www.favour.me/video/embed/13814/this-mom039s-experiment-proves-that-bottled-water-is-not-as-quothealthyquot-as-it-claims

Does this look good?

There should be a standalone extractor

Maybe I was misleading. I mean there should be both FavourIE and something like FavourEmbedIE. The former handles http://www.favour.me/video/13814/this-mom039s-experiment-proves-that-bottled-water-is-not-as-quothealthyquot-as-it-claims, and the latter handles http://www.favour.me/video/embed/13814/this-mom039s-experiment-proves-that-bottled-water-is-not-as-quothealthyquot-as-it-claims.
By the way, http://www.favour.me/video/embed/13814/this-mom039s-experiment-proves-that-bottled-water-is-not-as-quothealthyquot-as-it-claims uses JWPlayer, so the generic extracor can handle it. It's OK to just delegate the extraction to GenericIE.

@yan12125 Thanks for feedback. I'll start working on that.

@yan12125 @dstftw Does this seem good?

Squash commits.

@dstftw I've squashed commits. Good to merge?

@dstftw @yan12125 Does this look good?


add method unary

Codecov Report

Merging #125 (54d150d) into main (8ef691c) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #125   +/-   ##
=======================================
  Coverage   96.86%   96.86%
=======================================
  Files         167      168    +1
  Lines        1688     1692    +4
=======================================
+ Hits         1635     1639    +4
  Misses         53       53

Impacted Files
Coverage Δ

ramda/__init__.py
100.00% <100.00%> (ø)

ramda/unary.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8ef691c...54d150d. Read the comment docs.


Add grid to pattern library

Description
Add grid to pattern library for better layout..
Screenshot

Duplicate issue... https://github.com/universelabs/meta/issues/54


Request for new term 'metabolic dysfunction-associated steatohepatitis'

Label
metabolic dysfunction-associated steatohepatitis
Synonyms
mash
Synonym type
related
Definition
Metabolic dysfunction-associated steatohepatitis (MASH, formerly known as nonalcoholic steatohepatitis or NASH) is a type of fatty liver disease. It often develops due to a metabolic disorder, such as obesity or diabetes, resulting in a toxic buildup of fat in the liver. It is the most severe form of metabolic dysfunction-associated steatotic liver disease (MASLD, formerly known as nonalcoholic fatty liver disease or NAFLD).
Parent term
MONDO:0004790 fatty liver disease, MONDO:0002251 hepatitis
Children term(s)
No response
ORCID Identifier
No response
Website URL
https://www.verywellhealth.com/non-alcoholic-steatohepatitis-nash-5196357#toc-symptoms
Additional comments
No response

HI @tzukuo-evotec - thanks for this ticket. As I noted on another ticket, we'll address this in or after November due to other priorities this month. Thanks for your patience!

Hi @nicolevasilevsky ,
Thank you for the message and efforts.

@tzukuo-evotec
I think we should rename MONDO:0007027 'non-alcoholic steatohepatitis' to the label you requested above.
Please let me know if you have any objections, thanks!

Hi @tzukuo-evotec, we'll keep the old name as a synonym, so people will be able to find this term. I'll take care of this now and it should be available in the next release, which will be next week.


The TS Client can ignore an error in a response and continue, missing messages

On the PGlite + Linearlite demo comments table has 500k rows, it will try and sync, the client will get a 200 response code, but then there is this error in the status column on the dev tools:
(failed)net::ERR_INCOMPLETE_CHUNKED_ENCODING

This is after downloading about 78mb of data.
At that point nothing from that download is emited by the stream but it then seems to go on to the next request as though everything was ok. This uses the offset in the header of the failed response and so we miss a chunk of data. In my case it's during an initial sync so we loose everything.

I was incorrect in my initial understanding of this when I was looking at it on Thursday (don't debug with a headache!)
It does throw an error Uncaught (in promise) TypeError: Failed to fetch and that shape stops trying to sync.
I'm going to close this issue as #2075 is the real problem - I will add some notes over there.


Reflection-less Index generation [Upto 70% improvement in performance]

While generating the Subscriber indices there is an opportunity to do away with reflection.
Using the interface SubscriberMethodInvoker anonymous classes are generated which access the public methods of the class directly.
Older implementation :

New Implementation :

@greenrobot-team  : The build seems to be failing with 1 lint issue though it passes for me. We have tried to fix the build breakage on the master here with these fixes as well. We do not have access to the travis logs for lint warnings for us to be able to action it.


Error in Lua script using redis clustering

I sometimes get an error thrown in my limiter.on('error', err => console.log(err)) handler:
ReplyError: ERR Error running script (call to f_c8e3e22936b90c2d0e919964acaf643f234b6146): @user_script:77: user_script:77: attempt to perform arithmetic on a nil value
When this error happens, jobs currently EXECUTING seem to disappear, and the code using the limiter hangs forever.
This is my setup code:
const bottleneck_options = {
    id: 'awql_throttler',
    maxConcurrent: 50,
    minTime: 30,
    datastore: 'redis',
    clearDatastore: false,
    clientOptions: {
        host: redis_host,
        auth_pass: redis_password,
    },
    timeout: 1000 * 60,
}

const throttler = new Bottleneck(bottleneck_options)

This is how I add jobs:
        throttler.wrap(slowFunction).withOptions(
            {
                expiration: 1000 * 60 * 2,
            },
            args
        )

and here are the arguments to the failing lua code:
command: 'EVALSHA',
  args:
   [ 'c8e3e22936b90c2d0e919964acaf643f234b6146',
     8,
     'b_awql_throttler_settings',
     'b_awql_throttler_job_weights',
     'b_awql_throttler_job_expirations',
     'b_awql_throttler_job_clients',
     'b_awql_throttler_client_running',
     'b_awql_throttler_client_num_queued',
     'b_awql_throttler_client_last_registered',
     'b_awql_throttler_client_last_seen',
<PHONE_NUMBER>251,
     'np7jc1nq29s',
     '5f3qr3bs7ok' ],

I'm not able to find a repeatable way to reproduce this error :(
We're using redis 2.8.4 on a host a few ms of latency away, with a dozen or so processes across different hosts using the same limiter, which is almost always at max capacity (50 jobs). The error happens several times a minute.
Thanks for any help with this!

I'm using Bottleneck v2.16.0
I think that it's been throwing with this error for more than 48 hours, with previous Bottleneck versions (at least v2.15)

Found it. It's a race condition that can only happen at relatively high load or in relatively large clusters. The race condition was already handled in one place, but not in a second one just a few lines later.
Basically, limiter 1 cleaned up an expired job while limiter 2 was still running it, then limiter 2's timer expires, it goes to mark the job as expired. It tries to compute how much capacity is now available to start new jobs, but it can't find its weight value (deleted by limiter 1) and throws an exception.
I'll release v2.16.1 with this fix later today.

Please try v2.16.1 and let me know if the problem persists.
By the way you should make sure your job expirations aren't longer than your cluster timeout value. I'll be adding a validation for that in the next release.
Out of curiosity, how many servers do you have in that cluster? And other than this error, is it working as expected? Thanks 😄

I've just released with this new version -- so far so good :)
I've also increased the cluster timeout value -- you're right that it doesn't make sense. Although in practice our cluster is never idle for more than a minute anyway.
We've got one redis server. As for the processes using Bottleneck, we've got 4 servers, each running roughly 4 node processes. They're all constantly competing for a rate-limited API resource, which is why we're using Bottleneck.
I think it's working as expected! But I have to admit that it's quite difficult to test. Sometimes it feels like things gradually slow down over the course of days, and restarting the processes or clearing the b_* keys makes things faster again. But I'm not sure whether it's an issue with Bottleneck or some other moving part.
Thanks for the super quick fix, it's very much appreciated! 👍

This is very valuable feedback, thank you. It's hard to get that kind of Clustering feedback since most usage is in commercial/closed source software.
Next time it starts slowing down, could you please look into the following for me?

Do you only have to restart the Node processes to make it faster again? Or only delete the b_* keys? Or is it still slow until you do both things?
When the system is slow, is the CPU usage higher on the Node processes than when it's fast?
When the system is slow, is the network usage (from Bottleneck) higher on your Redis server? When you've reset everything and it's fast again, is the network usage back to normal?

Tip: the servers will recreate the keys automatically if they're missing.
Important: when you delete the b_* keys make sure your servers are not running, because they might be in the middle of running jobs and they would corrupt the data.
Thank you in advance, this will help me figure out if it's a Bottleneck issue that I need to reproduce and fix, or if it's an issue in your system.

The slowdown you're noticing might be due to clock drift between your servers. When things start slowing down, please take a look and make sure they're all in agreement. I have plans to use the Redis server's clock time instead, but it will require Redis 3.2 or higher. Redis 5 recently came out and even 3.2 is not supported anymore. However, since you and many other people are still using 2.8, I'm waiting a little bit longer before making this change.
I'm still interested to hear more about this slowdown next time it happens, answers to the questions in my previous post would greatly help.

It's possible that clocks could be the culprit. Although we use the standard ubuntu NTP setup, which I think should keep things fairly synced up.
Besides a potential re-write of the clock functionality, are there any other reasons why we should we should upgrade our redis instance for Bottleneck? Today, we're on v2.8 only because that's what came installed by default...
As for the slowdown issue, it seems to have been very solid since your last fix.

I haven't cleared the b_* keys since (and I had only started doing it because of this bug you fixed). I consider the clearing of keys in redis a closed affair.
Slowdowns that are fixed by restarting the server could really be caused by any number of things, and honestly I don't think that bottleneck is one of the prime suspects. But the next time it happens (and that I think it might be caused by bottleneck), I'll give you a proper breakdown of what's going on, including CPU usage and redis network usage.

Besides a potential re-write of the clock functionality, are there any other reasons why we should we should upgrade our redis instance for Bottleneck? Today, we're on v2.8 only because that's what came installed by default...

Not really, Redis is so reliable there is little reason to upgrade. When I decide to improve the cluster performance and accuracy by making this Time change, users with a version <3.2 will get a helpful error message upon first connection.

As for the slowdown issue, it seems to have been very solid since your last fix.

🎉 🎉 🎉

Slowdowns that are fixed by restarting the server could really be caused by any number of things, and honestly I don't think that bottleneck is one of the prime suspects. But the next time it happens (and that I think it might be caused by bottleneck), I'll give you a proper breakdown of what's going on, including CPU usage and redis network usage.

Perfect. I'm going to close this issue, please open another one if needed.


keosd-provider-timeout=5 default is quite aggressive

I've seen some missed blocks using the OOB signing process. Both for production blocks and "counter-signatures". Error looks like this upon production:
1853505ms thread-0   producer_plugin.cpp:1006      maybe_produce_block  ] 10 assert_exception: Assert Exception
!ec: Failed to read response: Connection timed out
    {"message":"Connection timed out"}
    thread-0  http_client.cpp:300 post_sync

And like this upon counter-signature:
2650583ms thread-0   controller.cpp:150            emit                 ] 10 assert_exception: Assert Exception
!ec: Failed to read response: Connection timed out
    {"message":"Connection timed out"}
    thread-0  http_client.cpp:300 post_sync

Some signatures take a lot more than 5ms.  This doesn't seem to interrupt the in-process signing mechanism, but it does truncate and drop the block if the OOB signing process takes too long.
Some blocks might loop through multiple sigs to find a "canonical" signature.. and I've seen the process take up to 15ms on my laptop (using https://github.com/eoscanada/eos-blocksigner ).
I'm going through http_client.cpp and am unsure how it considers connection keep-alives. If it were to drop the TCP connection between each signature, that'd add to latency. I'll try to check deeper if this is the case.
I don't have debug logging enabled in production, so can't easily tell the perceived time from nodeos.. I'll see what I can get.
If all of this adds to the latency, and is an important security measure, then https://github.com/EOSIO/eos/issues/3683 becomes all the more important.

We haven't received any update for this. This timeout parameter is configurable. If you still have the problem and wish to discuss further, please post it in https://eosio.stackexchange.com.


The access token has been obtained from wrong audience or resource 'https://batch.core.windows.net'. It should exactly match (including forward slash) with one of the allowed audiences 'https://batch.core.windows.net/'

Please follow the issue template below. Failure to do so will result in a delay in answering your question.

Library

[x]<EMAIL_ADDRESS>
Framework
Description
By msal.acquireToken({ scopes: ['https://batch.core.windows.net/user_impersonation'] }), I successfully got an access token with Payload including
    "aud": "https://management.azure.com"

But when I use this access token in the header for Azure Batch API, a response with status 401 would return with payload:
{
  "odata.metadata":"https://cpproto1batch.eastus.batch.azure.com/$metadata#Microsoft.Azure.Batch.Protocol.Entities.Container.errors/@Element","code":"AuthenticationFailed","message":{
    "lang":"en-US","value":"Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly.\nRequestId:a2e5f376-b529-43d0-a0ab-f9bf55716c70\nTime:2020-06-03T18:21:40.2674126Z"
  },"values":[
    {
      "key":"AuthenticationErrorDetail","value":"The access token has been obtained from wrong audience or resource 'https://batch.core.windows.net'. It should exactly match (including forward slash) with one of the allowed audiences 'https://batch.core.windows.net/'"
    }
  ]
}

Expected behavior
I have use the same approach: acquireToken({ scopes: ['https://management.azure.com/user_impersonation'] }), the access token it returened includes payload of
"aud": "https://management.azure.com",

no forward slash also, but it works fine when access API like https://management.azure.com/subscriptions.
I believe Azure Batch API should behave the same. This issue could be more of an API backend issue. And it has be there for quite a while. Is there any work around msal can do? like define audience when acquire access token?
Browsers/Environment

[ ] Chrome
[x] Firefox
[ ] Edge
[ ] Safari
[ ] IE
[ ] Other (Please add browser name here)

I tried the client credentials flow too, without msal. The access token it return got no slash in aud too. And again it failed to call api like https://{account-name}.{region-id}.batch.azure.com/pools.
Here could to be a wrong place to report this BUG. but where is the right place?

I double checked this and yeah, msal isn't affecting anything on that audience string, it looks like a bug from the API itself. A workaround would be to potentially code in the extra slash at the end of the audience string until this is fixed.
You can try to report the issue on both the report an issue page and the batch feedback page. We'll pass this along internally as well. Thanks for the information!

Thanks @jmckennon , I know it is more of an Azure batch API bug than an MSAL issue.
I have already issued a support ticket. After 6 days, they supply a workaround to change the scope from https://batch.core.windows.net/user_impersonation to https://batch.core.windows.net//.default (yes. there are double slashes).
This approach works for now. For someone who ran into the same problem, I hope this issue would supply some help.
As a developer myself, I always prefer opening issues on github rather than mailing back and forth for days explaining I am not a idiot. Just wish all you official contributors could have confirmed the BUG sooner.


Move minikube image to SUSE repo

https://github.com/SUSE/kubecf/blob/8568fb2b937e532789c56924c574592fb7ec5993/dev/minikube/def.bzl#L47 references an ISO from a private repo. It should be hosted in a SUSE repo.

@mook-as can you please add details on how this is built?

Run make build-in-docker to build the iso for use with minikube (for bazel run //dev/minikube:start).