It shouldn't be possible to (easily) create script named ".gd"

Godot version:
3.0.6, master as of 8c435a343e9739f30cb2e347df95835c91c1ff1a
OS/device including version:
Linux
Issue description:
if you choose to a create new gdscript file  in the script  editor, by default it will be named ".gd"  (dot G D)
When you press Ctrl-S after that, it will be saved as (unsurprisingly) ".gd" at which point you'll have to find the script in filesystem dock and rename it to something sensible.
Expected behaviour:
Editor should either refuse to create a new script until its name is given(right now it doesn't even show a warning),
or ask for the name when Ctrl-S is pressed (as most text editors do)
Steps to reproduce:
In script editor File->New, click Create without changing anything, Press Ctrl-S

I can't reproduce it in the master branch (c7fc3dcab):

It was fixed by 69c09ab2fdcfcc9de42ddcb3b7580c118b2182a3, which closed #13965 (which this issue duplicates). I'll push a fixup though as it's still possible to name a script     .gd.


Ryven udates

Hey, I'm currently trying to move towards a pre-release of a new Ryven version (see Ryven dev) to integrate a largely reworked ryvencore (docs here) which I wrote throughout last year. There are many breaking API  changes.

Stronger separation of node functionality and GUI. I'd like node packages to be compatible with headless mode by default (no GUI dependencies, no Qt). And I think the new system makes that refreshingly easy.
A conceptual change in nodes programming: any data sent over node ports must be wrapped in a Data object which defines how the data can be de-/serialized (i.e. when the object sent is not pickle serializable by default - like pythonocc types - then the package developer has to define de-/serialization manually). This finally enables Ryven to efficiently and correctly re-establish the old graph state when loading a project, even when stateful nodes are used, which was not the case before!
And many little API improvements which I included because why not if I'm already breaking stuff.

I thought I might use this package to 1. find the pain points of upgrading, and 2. working out the details of the API changes. So I just did that. Some important points:

A Node subclass class can now reference a custom NodeGUI subclass which implements everything UI related now (color, display title, widgets, right-click actions, etc.).
dtypes don't doesn't exist anymore. I added another system for frequently used input widgets in Ryven. Normally you would now write something likeclass MyNodeGui(NodeGUI):
    # register MyQtWidgetClass as input widget class for the node of this gui
    input_widget_classes = {'mywidget': SomeInputWidgetClass}
    # instantiate the input widget for the first initial input
    init_input_widgets = {0: 'mywidget'}

And SomeInputWidgetClass could be either manually defined, or one of the default provided ones by Ryven (replacing dtypes). Because there is little variation of input widgets in this package, I implemented a decorator function which automatically adds this here, so we don't have to write it for every node explicitly.
Packages like this one could substantially benefit from the new data sharing approach. One could now implement explicit de-/serialization for all the pythonocc types that are sent from nodes to other nodes. I defined a dummy OCCData(Data) class which currently does nothing - so it does not implement de-/serialization, but that might be fine here, after loading the user just needs to update the nodes that have no input connections and provide everything else with data. But that is just an idea, we might also provide proper de-/serialization. See comment in the code.

Basic node functionality is already working again. I will also try to write some sort of script that translates old project files so they can be loaded with the new version. I'll try to wrap this up soon.

Wow it seems to be some interesting features here, thanks!
I can't make the dev branch running on my computer though, I tried to reinstall using pip install .' but I get this error :  ERROR: No matching distribution found for ryvencore==0.4.*`

Yeah, you need to install ryvencore and ryvencore-qt from the corresponding dev branches as well. You can clone Ryven, ryvencore, and ryvencore-qt, and in each checkout the dev branch and pip install ..

Ok thank you.
but I still can't make Ryven works : I have several Error, I tried to comment the lines cuasing troubles but there are a lot.
Example :
check_connection_validity_request = Signal((NodeOutput, NodeInput), bool)
TypeError: pyqtSignal() argument expected to be sequence of types

when commenting the line (line 41 in ryvencore_qt\src\flows\FlowView.py) The startup dialog is displayed, when I click OK without changing anything I get :
File "...\ryven\gui\uic\ui_main_window.py", line 25, in setupUi
    MainWindow.setSizePolicy(sizePolicy)
TypeError: arguments did not match any overloaded call:
  setSizePolicy(self, a0: QSizePolicy): argument 1 has unexpected type 'PySide2.QtWidgets.QSizePolicy'
  setSizePolicy(self, hor: QSizePolicy.Policy, ver: QSizePolicy.Policy): argument 1 has unexpected type 'PySide2.QtWidgets.QSizePolicy'

When I comment the line and rerun I get:
 File "C:\ProgramData\Miniconda3\envs\kuka_iiwa_python-env\lib\site-packages\ryven\gui\uic\ui_main_window.py", line 26, in setupUi
    self.actionImport_Nodes = QAction(MainWindow)
TypeError: 'PySide2.QtWidgets.QAction' called with wrong argument types:
  PySide2.QtWidgets.QAction(MainWindow)
Supported signatures:
  PySide2.QtWidgets.QAction(PySide2.QtGui.QIcon, str, typing.Optional[PySide2.QtCore.QObject] = None)
  PySide2.QtWidgets.QAction(typing.Optional[PySide2.QtCore.QObject] = None)
  PySide2.QtWidgets.QAction(str, typing.Optional[PySide2.QtCore.QObject] = None)

It's too specific for my knowledge of Qt framework unfortunately.
One comment on your startup dialog : we can save config file but we can't load one, what is the use of it? It will be loaded when loading a .json session file?

check_connection_validity_request = Signal((NodeOutput, NodeInput), bool)
TypeError: pyqtSignal() argument expected to be sequence of types

This seems to be a PyQt error, but PyQt shouldn't be used at all, it's not supported. Do you have PyQt installed? If so can you uninstall it and try again?

  setSizePolicy(self, a0: QSizePolicy): argument 1 has unexpected type 'PySide2.QtWidgets.QSizePolicy'

So this is clearly a contradiction, or the two types come from different libraries. Maybe try creating a fresh virtual environment and installing the dependencies again (the Ryven libs from sources and PySide2, pyqt, and 'python-occ`)?

One comment on your startup dialog : we can save config file but we can't load one, what is the use of it? It will be loaded when loading a .json session file?

You can certainly load config files, the default config file ~/.ryven/config.cfg is loaded automatically, and you can specify further config files on the command line, e.g. ryven @~/.ryven/custom.cfg.

Indeed I had modules conflicts, it works with a new env, thank you.
I had to change some imports to make it work, example :
ModuleNotFoundError: No module named 'ryvencore.addons.default'
-->
from ryvencore.addons.Variables import VarsAddon

I can now load pythonocc nodes but can only display nodes without input :

Hmm, interesting. Sorry for the struggle, there's a few things to consider, I should probably figure out a better way to make everything work together.

Are you sure you checked out and pulled dev in the ryven repositories and installed from there? I just pushed some changes that might make it work for you so please pull the dev branches again.
Make sure to uninstall all ryven libraries (ryvencore, ryvencore-qt, ryven) before installing from source.

you can verify using conda list which lists all installed libraries

You need to install ryvencore first, then ryvencore-qt, and then Ryven.

Please let me know if it still doesn't work.

Yes, I've uninstalled and repull the dev branch, it still doesn't work.
Also to be able to run Ryven I had to change de setup.cfg file from Ryven from
install_requires =
    ryvencore-qt ==0.4.*

to
install_requires =
    ryvencore-qt

as the env is a new one and I've installed the packages in the correct order it doesn't change anything in our problem.
Just to say that I think that ryvencore-qt/dev has not the 0.4 version specified (that's a change that I had also made in my earlier tests).

My bad, VsCode displayed "dev" evant after repulling but without success, deleting the dir and cloning/checkout dev had make it appeared. I still can't add nodes with inputs unfortunately

What's the error? Starting Ryven as ryven -v will give verbose output to the console, including all error messages.

I don't get any messages drag/droping these inputs nodes (ex: Point).
For Point0 I get:
--> INFO:  setting output 0 in Point0
When I right click and choose one by double clicking on it I get:
Traceback (most recent call last):
  File "C:\Users\tanneguydv\.conda\envs\ryven_dev-env\lib\site-packages\ryvencore_qt\src\flows\FlowView.py", line 794, in create_node__cmd
    self._push_undo(
  File "C:\Users\tanneguydv\.conda\envs\ryven_dev-env\lib\site-packages\ryvencore_qt\src\flows\FlowView.py", line 260, in _push_undo
    cmd.activate()
  File "C:\Users\tanneguydv\.conda\envs\ryven_dev-env\lib\site-packages\ryvencore_qt\src\flows\FlowCommands.py", line 34, in activate
    self.redo()
  File "C:\Users\tanneguydv\.conda\envs\ryven_dev-env\lib\site-packages\ryvencore_qt\src\flows\FlowCommands.py", line 40, in redo
    self.redo_()
  File "C:\Users\tanneguydv\.conda\envs\ryven_dev-env\lib\site-packages\ryvencore_qt\src\flows\FlowCommands.py", line 98, in redo_
    self.node = self.flow.create_node(self.node_class)
  File "C:\Users\tanneguydv\.conda\envs\ryven_dev-env\lib\site-packages\ryvencore\Flow.py", line 233, in create_node
    self.add_node(node)
  File "C:\Users\tanneguydv\.conda\envs\ryven_dev-env\lib\site-packages\ryvencore\Flow.py", line 262, in add_node
    self.node_added.emit(node)
  File "C:\Users\tanneguydv\.conda\envs\ryven_dev-env\lib\site-packages\ryvencore\Base.py", line 77, in emit
    cb(*args)
  File "C:\Users\tanneguydv\.conda\envs\ryven_dev-env\lib\site-packages\ryvencore_qt\src\flows\FlowView.py", line 817, in add_node
    item.initialize()
  File "C:\Users\tanneguydv\.conda\envs\ryven_dev-env\lib\site-packages\ryvencore_qt\src\flows\nodes\NodeItem.py", line 112, in initialize
    self.add_new_input(inp)
  File "C:\Users\tanneguydv\.conda\envs\ryven_dev-env\lib\site-packages\ryvencore_qt\src\flows\nodes\NodeItem.py", line 189, in add_new_input
    item = InputPortItem(self.node_gui, self, inp, input_widget=widget)
  File "C:\Users\tanneguydv\.conda\envs\ryven_dev-env\lib\site-packages\ryvencore_qt\src\flows\nodes\PortItem.py", line 113, in __init__
    if self.port.load_data is not None and self.port.load_data['has widget']:
AttributeError: 'NodeInput' object has no attribute 'load_data'

I don't get any messages drag/droping these inputs nodes (ex: Point).

true, that's a bug I still need to fix, I remember.
It still doesn't seem to have dev pulled correctly, I added the fix for that this morning and commited it to dev, see this commit.

Hey,
sorry for the pulling trouble that I had!
I have been able to re-do the torus example with success (except for the final fillet operation, don't know why) :

As you can see we have a result, thank you!
Some notes :

Nodes's color are all yellow, even though it's specified in guis
Some links doesn't refresh when we unplug it (example in screenshot)
I cannot add input to nodes (List node for example), so I had to create several List nodes with only 2 input per nodes and then add them together and flatten them
Perhaps an autosave function in .json format could be useful, mainly with PythonOCC where a wrong plug on input can causes crashes (as there are no try/except on nodes for the moment)

I have tried to reload the saved file in the .json format. the graph does appear well but, as you said above, the nodes are not filled with values, I need to refresh it and I can't find a way. Even node without input, such as AxZ doesn't output any values. I have tried the right-click "update shape" without success. But it is still a great improvement regarding previous loading causing crash without any display of any kind.

Hey,
sorry for the pulling trouble that I had!
I have been able to re-do the torus example with success (except for the final fillet operation, don't know why) :

As you can see we have a result, thank you!
Some notes :

Nodes's color are all yellow, even though it's specified in guis
Some links doesn't refresh when we unplug it (example in screenshot)
I cannot add input to nodes (List node for example), so I had to create several List nodes with only 2 input per nodes and then add them together and flatten them
Perhaps an autosave function in .json format could be useful, mainly with PythonOCC where a wrong plug on input can causes crashes (as there are no try/except on nodes for the moment)

I have tried to reload the saved file in the .json format. the graph does appear well but, as you said above, the nodes are not filled with values, I need to refresh it and I can't find a way. Even node without input, such as AxZ doesn't output any values. I have tried the right-click "update shape" without success. But it is still a great improvement regarding previous loading causing crash without any display of any kind.

I have been able to re-do the torus example with success

Great!

Nodes's color are all yellow, even though it's specified in guis
I cannot add input to nodes (List node for example), so I had to create several List nodes with only 2 input per nodes and then add them together and flatten them

Thanks, fixing it now.

* Some links doesn't refresh when we unplug it (example in screenshot)

Thanks.

* Perhaps an autosave function in` .json` format could be useful, mainly with PythonOCC where a wrong plug on input can causes crashes (as there are no `try/except` on nodes for the moment)

Autosave would be neat but the implementation is a bit more difficult than it sounds. In any case there shouldn't be crashes, or the nodes package is malfunctional (which renders autosave useless anyway). The nodes cannot crash the application on update by raising an exception, exceptions should be catched by Ryven. When does this happen for you, and what is the output? (run ryven --verbose to get full output on console)

I have tried to reload the saved file in the .json format. the graph does appear well but, as you said above, the nodes are not filled with values, I need to refresh it and I can't find a way.

The way I did this now implies that all output values will be None after loading. In cases where there are few sources of truth (like a couple of val nodes and all other nodes depend on them directly or indirectly, so the whole graph will be re-executed once these val nodes update) the user can easily bring the graph back into the original state by updating those. But when there are nodes without inputs this cannot work, of course.
The default approach would be to define de-/serialization. I saw that PythonOCC doesn't support any serialization by itself, and it might be some work. Do you know anything about this?

Autosave would be neat but the implementation is a bit more difficult than it sounds. In any case there shouldn't be crashes, or the nodes package is malfunctional (which renders autosave useless anyway). The nodes cannot crash the application on update by raising an exception, exceptions should be catched by Ryven. When does this happen for you, and what is the output? (run ryven --verbose to get full output on console)

I have tried to reload the saved file in the .json format. the graph does appear well but, as you said above, the nodes are not filled with values, I need to refresh it and I can't find a way.

The way I did this now implies that all output values will be None after loading. In cases where there are few sources of truth (like a couple of val nodes and all other nodes depend on them directly or indirectly, so the whole graph will be re-executed once these val nodes update) the user can easily bring the graph back into the original state by updating those. But when there are nodes without inputs this cannot work, of course.
The default approach would be to define de-/serialization. I saw that PythonOCC doesn't support any serialization by itself, and it might be some work. Do you know anything about this?

Ok thank you very much!
I have other projects where serialization is needed for Pythonocc, I will try do do something about it as soon as I can, i keep you informed.

Quick update: I finished all major Ryven adaptions and the new versions (Ryven v3.4, ryvencore-qt v0.4, ryvencore v0.4) are available as pre-release on PyPI. In a fresh virtual environment, you can run:
pip install --pre ryven

Hey, any progress on this? We will need to define serialization and de-serialization for all the PythonOCC types passed between nodes. Is there anything I can help with that doesn't require PythonOCC knowledge? Or is there anyone who has experience with this task and could maybe help out?

#19 @leon-thomm

#19 @leon-thomm

not sure how your PR solves the serialization issue described here. there is no point as long as that's not solved I think


bbci_apply_sendControl.m to "Processing"

Test if bbci_apply_sendControl.m can connect via UDP to the visualization software "Processing".

BBCI Feedback with processing was done for the BBCI Spraying Game. The communication works simply with the provided UDP interface:
bbci.feedback.receiver= 'udp';
see also the run script in (the non-public folder):
bbci_private/acquisition/scripts/teaching/2014ss-BCIPJ-Team2
Receiving the signals in processing is also very simple, see BCIListenForClassifier.pde in
bbci_private/feedbacks/processing/SprayYourBBCIFeedback
As far was I remember, you need to install the hypermedia package manually.


Rounding error fix for page calculation

There was an error in the calculation of the page based on the scrollview.
As long as you have a full width slideshow everything is fine, but when your slideshow frame is smaller than the screen you can have page calculation errors due to rounding of CGFloat values, resulting in page counting as for example 1,1,3,3,5,6,6,....
Note: Swift CGFloat operations are not the same as ObjC CGFloat ones. The original code did probably work for ObjC projects, but in Swift you must alter some calculations.

Hi @mjrehder, good catch! Thanks for your contribution.


os: StartProcess ETXTBSY race on Unix systems

Modern Unix systems appear to have a fundamental design flaw in the interaction between multithreaded programs, fork+exec, and the prohibition on executing a program if that program is open for writing.
Below is a simple multithreaded C program. It creates 20 threads all doing the same thing: write an exit 0 shell script to /var/tmp/fork-exec-N (for different N), and then fork and exec that script. Repeat ad infinitum. Note that the shell script fds are opened O_CLOEXEC, so that an fd being written by one thread does not leak into the fork+exec's shell script of a different thread.
On my Linux workstation, this program produces a never-ending stream of ETXTBSY errors. The problem is that O_CLOEXEC is not enough. The fd being written by one thread can leak into the forked child of a second thread, and it stays there until that child calls exec. If the first thread closes the fd and calls exec before the second thread's child does exec, then the first thread's exec will get ETXTBSY, because somewhere in the system (specifically, in the child of the second thread), there is an fd still open for writing the first thread's shell script, and according to modern Unix rules, one must not exec a program if there exists any fd anywhere open for writing that program.
Five years ago this bit us because cmd/go installed cmd/cgo (that is, copied the binary from a temporary location to somewhere/bin/cgo) and then executed it. To fix this we put a sleep+retry loop around the fork+exec of cgo when it gets ETXTBSY. Now (as of last week or so) we don't ever install cmd/cgo and execute it in the same cmd/go process, so that specific race is gone, although as I write this cmd/go still has the sleep+retry loop, which I intend to remove.
Last week this bit us again because cmd/go updated a build stamp in the binary, closed it, and executed it. The resulting flaky ETXTBSY failures were reported as #22220. A pending CL fixes this by not updating the build stamp in temporary binaries, which are the main ones we execute. There's still one case where we write+execute a program, which is go test -cpuprofile x.prof pkg. The cpuprofile flag (and a few others) cause cmd/go to leave the pkg.test in the current directory for debugging purposes but also run the test. Luckily running the test is currently the final thing cmd/go does, and it waits for any other fork+exec'ed programs to finish before fork+exec'ing the test. So the race cannot happen in this case.
In general this race is going to happen every time anyone writes a program that both writes and executes a program. It's easy to imagine other build systems running into this, but also programs that do things like unzip a zip file and then run a program inside it - think a program supervisor or mini container runtime. As soon as there are multiple threads doing fork+exec at the same time, and one of them is doing fork+exec of a program that was previously open for write in the same process, you have a mysterious flaky problem.
It seems like maybe Go should take care of this, if possible. We've now hit it twice in cmd/go, five years apart, and at least this past time it took the better part of a day to figure out. (I don't remember how long it took five years ago, in part because I don't remember anything about discovering it five years ago. I also don't want to rediscover all this five years from now.)
There are a few hacks we could use:

In os.StartProcess, if we see ETXTBSY, sleep 100ms and try again, maybe a few times, up to say 1 second of sleeping. In general we don't know how long to sleep.
Arrange with a locking mechanism that close must never complete during a fork+exec sequence. The end of the fork+exec sequence needs to be the point where we know the close-on-exec fds have been closed. Unfortunately there is no portable way to identify that point.

If the exec fails and the child tells us and exits, we can wait for the exit. That's easy.
If the exec succeeds, we find out because the exec closes the child's end of the status pipe, and we get EOF.

If we know that an OS does close-on-exec work in increasing fd order, then we could also track the maximum fd we've opened and move the status pipe above that. Then seeing the status pipe close would mean all other fds are closed too.
If the OS had a "close all fds above x", we could use that. (I don't know of any that do, but it sure would help.)

It may not be OK to block all closes on a wedged fork+exec (in general an exec'ed program may be loaded from some slow network server).

Note that vfork(2) is not a solution. Vfork is defined as the parent does not continue executing until the child is no longer using the parent's memory image. In the case of a successful exec, at least on Linux, vfork releases the memory image before doing any of the close-on-exec work, so the parent continues running before the child has closed the fds we care about.

None of these seem great. The ETXTBSY sleep, up to 1 second, might be the best option. It would certainly reduce the flake rate and in many cases would probably make it undetectable. It would not help exec of very slow-to-load programs, but that's not the common case.
I wondered how Java deals with this, and the answer seems to be that Java doesn't deal with this. https://bugs.openjdk.java.net/browse/JDK-8068370 was filed in 2014 and is still open.
#include <pthread.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <errno.h>
#include <stdint.h>

void* runner(void*);

int
main(void)
{
	int i;
	pthread_t pid[20];

	for(i=1; i<20; i++)
		pthread_create(&pid[i], 0, runner, (void*)(uintptr_t)i);
	runner(0);
	return 0;
}

char script[] = "#!/bin/sh\nexit 0\n";

void*
runner(void *v)
{
	int i, fd, pid, status;
	char buf[100], *argv[2];

	i = (int)(uintptr_t)v;
	snprintf(buf, sizeof buf, "/var/tmp/fork-exec-%d", i);
	argv[0] = buf;
	argv[1] = 0;
	for(;;) {
		fd = open(buf, O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0777);
		if(fd < 0) {
			perror("open");
			exit(2);
		}
		write(fd, script, strlen(script));
		close(fd);
		pid = fork();
		if(pid < 0) {
			perror("fork");
			exit(2);
		}
		if(pid == 0) {
			execve(buf, argv, 0);
			exit(errno);
		}
		if(waitpid(pid, &status, 0) < 0) {
			perror("waitpid");
			exit(2);
		}
		if(!WIFEXITED(status)) {
			perror("waitpid not exited");
			exit(2);
		}
		status = WEXITSTATUS(status);
		if(status != 0)
			fprintf(stderr, "exec: %d %s\n", status, strerror(status));
	}
	return 0;
}

/cc @aclements @ianlancetaylor @crawshaw

Userspace workarounds seem flawed or less than ideal.  This is a kernel
problem, like O_CLOEXEC.  Perhaps lobby for a O_CLOFORK that's similar
but close on fork instead.  The writer would open, write, close, fork,
exec so wouldn't make use of it, but any other thread that forks
wouldn't carry the FD with it so the writer's close would succeeding in
nailing the sole, final, reference to the "open file description", as
POSIX calls it.

O_CLOFORK is a good idea.  Does anybody want to suggest that to the Linux kernel maintainers?  I expect that if someone can get it into Linux it will flow through to the other kernels.
I'm going to repeat a hack I described elsewhere that I believe would work for pure Go programs.

record the highest file descriptor returned by syscall.Open, syscall.Socket, syscall.Dup, etc.
add a new RWMutex in syscall: forkMutex
during syscall.Close, acquire a read lock on forkMutex
in syscall.forkAndExecInChild acquire a write lock on forkMutex, and
open a pipe in the parent (as we already do if UidMappings is set), and
in the child, loop through the descriptors up to the highest one,
closing each one that is marked close-on-exec, then close the pipe to the parent
in the parent, when the pipe is closed, release the forkMutex lock

The effect of this should be that when syscall.Close returns, we know for sure that there is no forked child that has an open copy of the descriptor.
The disadvantages are that all forks are serialized, and that all forks waste time closing descriptors that will shortly be closed anyhow.  Also, of course, forks temporarily block closes, but that is unlikely to be significant.

O_CLOFORK is a good idea. Does anybody want to suggest that to the Linux kernel maintainers?

I'm happy to have a go, but I'm a nobody on that list. I was assuming folks here might have the ear of a Google kernel developer or two in that area that would vet the idea and suggest it to the list if worthy. :-)

during syscall.Close, acquire a read lock on forkMutex

And syscall.Dup2 and Dup3 as they may cause newfd to close.
Do syscall.Open et al also synchronise with forkMutex somehow? I'm wondering if they can be creating more FDs, either above or below the highwater mark, whilst forkAndExecInChild is looping, closing close-on-exec ones.

Is there a place to file a feature request against the Linux kernel?  I know nothing about the kernel development process.  I hear it uses git.
Agree about Dup2 and Dup3.
As far as I can see it doesn't matter if syscall.Open and friends create a new FD while the child is looping, because the child won't see the new descriptor anyhow.

@ianlancetaylor thanks, yes, the explicit closes would solve the problem with slow execs, which would be nice. That might make this actually palatable. You also don't even need the extra pipe if you use vfork in this approach.
I agree with @RalphCorderoy that there's a race between the "maintain the max" and "fork", in that Open might create a new fd, then fork runs in a different thread before Open can update the max. But since fds are created lowest-available, it should suffice for the child to assume that max is, say, 10 larger than it is.
Also note that this need not be an RWMutex (and for that matter the current syscall.ForkMutex need not be an RWMutex either). It just needs to be an "either-or" mutex. An RWMutex allows N readers or 1 writer. The mutex we need would allow N of type A or N of type B, just never a mix. If we built that (not difficult, I don't think), then programs that never fork would not serialize any of their closes, and programs that fork a lot but don't close things would not serialize any of their forks.
O_CLOFORK would require having fcntl F_SETFL/F_GETFL support for that bit too, and it would complicate fork a little more than it already is. An alternative that would be equally fine for us would be a "close all fd's above" or "tell me the maximum fd of my process" syscall. I don't know if a new bit or a new syscall is more likely.

I should maybe also note that macOS fixes this problem by putting #if 0 around the ETXTBSY check in the kernel implementation of exec. That would be a third option for Linux although probably less likely than the other two.

I've emailed<EMAIL_ADDRESS>Will reference an archive once it appears.
If they're unpersuaded, then there's the POSIX folks at Open Group; they have a bug tracker.

linux-kernel mailing-list archive of post: https://marc.info/?l=linux-kernel&m=150834137201488

What's the plan here for Go 1.10?
@RalphCorderoy, looks like you never got a reply, eh?

Looks like Solaris and macOS and OpenBSD have O_CLOFORK already.  Hopefully it will catch on further.

I'm currently running into this (I think?) on Ubuntu, using Go 1.13.5, calling ioutil.WriteFile to write a binary, immediately followed by exec.Command.

A colleague pointed me to this bug in context of a wider discussion about O_CLOFORK. When each fork is expected to proceed to exec (as is the case here), it is possible to solve the problem via open file description locks in 4 extra syscalls, without requiring any cooperation between threads.
The high-level algorithm for writing a file for execution is as follows:

open an fd with O_WRONLY | O_CLOEXEC
write into fd
place open file description lock on the fd
close the fd
open a new fd with O_RDONLY | O_CLOEXEC (same path as step 1)
place open file description lock on it
close the fd

If an fd opened in step 1 leaked to another process as a result of concurrent thread issuing a fork(), we wait for it to be closed at step 6. An fd opened at step 5 may also leak, but won't cause ETXTBUSY as it is open read-only.
The diff to the program shown in the opening comment would be just:
@@ -41,6 +44,20 @@ runner(void *v)
                        exit(2);
                }
                write(fd, script, strlen(script));
+               if (flock(fd, LOCK_EX) < 0) {
+                       perror("flock");
+                       exit(2);
+               }
+               close(fd);
+               fd = open(buf, O_RDONLY|O_CLOEXEC, 0777);
+               if(fd < 0) {
+                       perror("open (readonly)");
+                       exit(2);
+               }
+               if (flock(fd, LOCK_SH) < 0) {
+                       perror("flock (readonly)");
+                       exit(2);
+               }
                close(fd);
                pid = fork();
                if(pid < 0) {

@amonakov Thanks for the comment.  That is an interesting suggestion.
I guess that to make this work automatically in Go we would have to detect when an executable file is opened with write access.  Unfortunately this would seem to require an extra fstat system call for every file opened for write access.  That is not so great.  Perhaps we could restrict it to only calls that use O_CREATE as that is likely the most common case that causes problems.
But then there seems to be a race condition.  The fork can happen at any time.  If the fork happens after we call open but before we call flock, then it seems that the same problem can occur.  In the problematic case the fork doesn't know anything about the file that we are writing.  The problem is that the file is held open by the child process.  Using the flock technique makes this much less likely to be a problem, but I don't think it completely eliminates the problem.

... make this work automatically in Go ...

I don't think that would work: permission bits could be changed independently after close(). In any case, my solution has two assumptions, that file was opened with O_CLOEXEC, and that long-lived forks do not appear. For that reason I'd say it's not appropriate to roll it up into some standard function. It could live as a separate close-like function where the purpose and requirements could be clearly documented.

But then there seems to be a race condition. The fork can happen at any time. If the fork happens after we call open but before we call flock, then it seems that the same problem can occur.

No, forked child shares the open file description with the parent, so a later flock in the parent still affects it.

@amonakov Thanks.
For what it's worth, all files opened using the Go standard library have O_CLOEXEC set.  And Go doesn't support long-lived forks, as fork doesn't work well with multi-threaded programs, and all Go programs are multi-threaded.  So I don't think those are issues.
That said, personally I would not want to add new API to close an executable file.  That seems awkward and hard to understand.  I'd much rather persuade kernels to support O_CLOFORK.  Of course any particular program can use your technique.

Relatively recent (sad) thread re: linux O_CLOFORK: https://lore.kernel.org/lkml/20200525081626.GA16796@amd/T/

Thanks to reading the linux-kernel thread @spoerri mentions above, I see POSIX has added FD_CLOFORK and O_CLOFORK: https://www.austingroupbugs.net/view.php?id=1318

I see that some of the Linux kernel developers are pretty skeptical about the need for this; is anybody reading this issue able to point them to the problem described here?  It's not a common problem but it's not at all specific to Go.  Thanks.

Hi Ian, Yes, I had a go yesterday by telling the linux-kernel list about this Go issue and the Java one to show it wasn't system(3) specific and has a wide long-standing impact. Matthew Wilcox, who implies he's a Googler, has replied so far:

The problem is that people advocating for O_CLOFORK understand its value, but not its cost. Other google employees have a system which has literally millions of file descriptors in a single process. Having to maintain this extra state per-fd is a cost they don't want to pay (and have been quite vocal about earlier in this thread).

Perhaps the first thing is to get agreement it's the kernel's issue to fix and then move on to an implementation with a cost they find acceptable. At the moment, the kernel seems to be faulty but fast.

I observed what I believe is an equivalent race for pipes when investigating #36107.
The race can cause a Write call to a pipe to spuriously succeed (instead of returning EPIPE) after the read side of the pipe has been closed.
The mechanism is the same:

The parent creates the pipe and (with syscall.ForkLock held) sets it to be CLOEXEC.
The parent process calls os.StartProcess, which forks a child process.

(The child process inherits a copy of the file descriptors for the pipe.)

Before the child process has reached its exec, the parent process closes the read side of the pipe.

(However, the FD for that side remains open in the child process.)

The parent process calls Write on the write side, causing the kernel to buffer the write (since the pipe FD is still open in the child). The Write succeeds.
Finally, the child process reaches its exec call, closing its copy of the read FD and dropping the buffered bytes.

This race can be observed by running os_test.TestEPIPE concurrently with a call to os.StartProcess.

If the OS had a "close all fds above x", we could use that. (I don't know of any that do, but it sure would help.)

Since this was written, Linux and FreeBSD added a close_range syscall (about fall 2020 - Linux kernel 5.9, FreeBSD 12.2, should be identical semantics). The reason for close_range instead of closefrom (which also exists in a few OSes) is so that if you want to have a child process inherit, say, fds 0, 1, 2, and 1000, you can do two syscalls instead of 998.
Assuming that the set of open fds is sparse, you can also do a pretty good job of this on many OSes, including older Linux versions, by using /dev/fd, /proc/self/fd, or equivalent. From a quick Google, https://github.com/cptpcrd/close_fds seems to be a pretty thorough attempt at finding the best OS-specific way.
Also, just to make sure I understand correctly - the idea here is that you would have some sort of process-wide two-sided lock where calls to close require a critical section around the close call alone and calls to fork require a critical section on the other side from before fork until unwanted file descriptors are closed. (It's not exactly an rwlock because you can have multiple closes at once or multiple forks at once, you just can't have one of each. Also note the lock has to work properly across fork.) And the reason a "close all fds above x" operation would help is that it would allow you to reduce the second critical section to be [fork, close_range] instead of [fork, exec triggering close-on-exec], which is helpful because exec could potentially be slow because of the filesystem, and also because if you're waiting for a self-pipe to close, it might get closed before other FDs are closed. Do I have all of that right?
By the way, there is one mildly exciting complication here, which is that closing the FD can also potentially be slow because of the filesystem. At least on Linux, closing a file descriptor calls the filesystem's flush operation on the underlying file description - even if there are other file descriptors open to that file description which will eventually be closed too! So it's possible that if thread A opens a file on a slow filesystem, thread B forks a child for exec, and thread A then closes the file, both thread A and the child will be slowed down. This isn't any worse than the status quo, to be clear, where the close happens on exec. But it is a really good argument in favor of O_CLOFORK, and that it should be implemented in the form where the FD is never copied to the new process's FD table in the first place instead of actually being closed on fork.
(Another related subtlety - close can return errors! The file might fail to write for I/O reasons or the filesystem might check quota only when the writes is flushed. This can cause actual data loss if you assume it succeeded. On Linux, close returns the error from the filesystem's flush operation and I believe always gets rid of the FD, and I guess it's up to the filesystem whether a second flush on another FD to the same open file description also returns the same error, or if it's lost state at that point. And close_range and of course close-on-exec do not report errors back to userspace the way close does. So that's another really good argument for O_CLOFORK implemented as never-inherit, to ensure that errors from close successfully get to the right close call.)

Note that vfork(2) is not a solution. Vfork is defined as the parent does not continue executing until the child is no longer using the parent's memory image. In the case of a successful exec, at least on Linux, vfork releases the memory image before doing any of the close-on-exec work, so the parent continues running before the child has closed the fds we care about.

The bigger problem is that vfork (at least on Linux and FreeBSD) only suspends the calling thread, and other threads continue to run, so it doesn't solve the problem at all. (If it weren't for that, I think vfork + close_range would avoid this problem, since the parent is still suspended during close_range.)

@geofft Thanks for pointing out close_range.  Unfortunately I'm not sure how to use it in a fully reliable way, as Go programs can of course turn off the "close-on-exec" flag for a file descriptor.  I think that what we want in the absence of O_CLOFORK is an efficient way to close all descriptors marked close-on-exec, while leaving the non-close-on-exec descriptors alone.

I've worked around this issue on a piece of code that writes to a file that will be executed shortly after by locking syscall.ForkLock for reading while the file is open.
If my understanding is correct, the runtime always locks syscall.ForkLock for writing before forking, so locking it for reading before opening a file prevents "leaking" its file descriptor to forked processes, as that fork cannot happen while syscall.ForkLock is locked.
There is of course not a general solution, as locking that mutex for every file that is open for writing would incur massive performance penalties, but if I'm not mistaken this is a viable workaround for pieces of code which can assume that a file they are writing will be executed later.

FYI, Linux 6.11 (just released today) has dropped the ETXTBSY "feature" entirely. The commit making this change links to this issue, and also has an argument that the security purpose of this feature didn't actually work. See also this 2021 LWN discussion of the issue and previous changes to reduce the cases that trigger ETXTBSY.
It'll be months/years before the distros that real-world users are actually using get to 6.11, but it's good to know that at least on Linux this issue will completely go away at some point.
Maybe we can also use this to argue to Oracle, Apple, and the BSDs that they should make the equivalent change.


When a new release?

V2.6.2 Missing zrevrangebylex command.
When a new release?

2.7.0 will be released shortly (this weekend probably). Branches are frozen so if you really need the functionality you can checkout the branch and build the jar manually.

OK. Thank you!


support configmap volume items API

Example:
apiVersion: v1
kind: ConfigMap
metadata:
  name: hello-world-wasi-rust
data:
  myval: "cool stuff"
  loremIpsum: |
    Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut
    labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco
    laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in
    voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat
    non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
---
apiVersion: v1
kind: Pod
metadata:
  name: hello-world-wasi-rust
spec:
  containers:
    - name: hello-world-wasi-rust
      image: webassembly.azurecr.io/hello-world-wasi-rust:v0.2.0
      env:
        - name: FOO
          value: bar
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: CONFIG_MAP_VAL
          valueFrom:
            configMapKeyRef:
              key: myval
              name: hello-world-wasi-rust
      volumeMounts:
        - name: storage
          mountPath: /mnt/storage
  volumes:
    - name: storage
      configMap:
        name: hello-world-wasi-rust
        items:
          - key: loremIpsum
            path: lorem_ipsum.txt
  tolerations:
    - key: "krustlet/arch"
      operator: "Equal"
      value: "wasm32-wasi"
      effect: "NoExecute"

Here's the important bit:
        items:
          - key: loremIpsum
            path: lorem_ipsum.txt

The current behaviour will mount the whole ConfigMap into the WASI runtime. The items API tells the runtime that only certain keys should be mounted, and at what path.
The example above should be available at /mnt/storage/lorem_ipsum.txt in the runtime, and myval should not be present.

more info: https://kubernetes.io/docs/concepts/storage/volumes/#configmap

If this is something we'd like to do, and if nobody else is already doing it, I can put my hand up for it.

@itowlson have you already picked this up? I have been looking to get involved and this seems like a good place if you haven't already started.

Go for it @EItanya - I've gotten distracted by other issues.  Thanks and welcome!

@EItanya Are you still working on this?  Let me know if not and I can pick it up.

@EItanya Are you still working on this? Let me know if not and I can pick it up.

Hey @itowlson sorry for the late reply, things got crazy on my end and I haven't had time to pick work on it. Feel free to pick it back up


Allow checkboxes to be marked as disabled

Fixes #207188

Adopts this in Manage Trusted Extensions picker.
Also exposes the Manage Trusted Extensions flow via the command palette.

Do you plan to make this available for extensions to use?

@gjsjohnmurray not this iteration but I would like to light this up as I know some folks would like something like this.
Would you mind opening an issue with your scenario?


[Broken Windows Build] MutableActionConflictException

C:\src\reclient>bazelisk build  --action_env PATH="C:\Python312;C:\tools\msys64\usr\bin;C:\tools\msys64\bin;C:\Windows;C:\Windows\System32;C:\Windows\System32\WindowsPowerShell\v1.0;C:\tools\msys64\mingw64\bin;C:\Program Files\Git\cmd" --verbose_failures  --features=fastbuild  //cmd/...

FAILED: Build did NOT complete successfully (120 packages loaded, 13292 targets configured)

I don't have a lot of context but the following appears to resolve the issue:
+#copy_file(
+#    name = "scandeps_server_pdb",
+#    src = ":scandeps_server_dbg_pdb",
+#    out = "scandeps_server.pdb",
+#    tags = ["manual"],
+#    target_compatible_with = ["@platforms//os:windows"],
+#)

@bentekkie any thoughts on this?

Working on a fix

https://github.com/bazelbuild/reclient/commit/bce2a83c95ef7f2b32115defcb26606e87da2d94 should fix this, can you verify?

bce2a83 should fix this, can you verify?

yep, that's the fix I used locally to get past this. I need to get clang installed on my build before I can test further but I will report back if there's anything else still broken. thanks!

Closing as this is fixed


Add LongPress gesture handling

I needed long press support, so I simply replicated the current tap support.
It works, but the duplication isn't pretty. I can come up with a better solution
if you agree this functionality is worth adding and dislike the duplication. For
example, using a single handler with an enum (.tap, .longPress) as an argument.
Let me know your thoughts.

Regarding the failing test: that test fails for me on a fresh checkout, without any of my changes.

I just realized that the long press handler isn't very useful without the gesture state.  I'm going to think about a better way, and close this for now.


Documentation enhance for Calibration algorithm

Type of Change
documentation
API not changed
Description
Documentation enhance for Calibration algorithm
Expected Behavior & Potential Risk
calibration.md should be easy to read
How has this PR been tested?
CI test
Dependency Change?
no

We need to add the entry point somewhere guide user to this docs. Maybe some where in https://github.com/intel/neural-compressor/blob/master/docs/source/quantization.md, or somewhere directly here https://intel.github.io/neural-compressor/latest/docs/source/user_guide.html#advanced-topics?

We need to add the entry point somewhere guide user to this docs. Maybe some where in https://github.com/intel/neural-compressor/blob/master/docs/source/quantization.md, or somewhere directly here https://intel.github.io/neural-compressor/latest/docs/source/user_guide.html#advanced-topics?

Add link to https://intel.github.io/neural-compressor/latest/docs/source/user_guide.html#advanced-topics


Offworld Trading Company (Unity 5.6.3p1) near-instacrash with HUD enabled.

I'm getting immediate crashes with DXVK_HUD=1 in Offworld Trading Company.
The game seems to work pretty well without the HUD enabled, though.
Software information
Offworld Trading Company (Unity 5.6.3p1)
System information

GPU: GTX 970
Driver: prop 387.34
Wine version: Wine 2.21-staging

Apitrace file(s)

Offworld.trace.gz (gzipped not for size but to trick GitHub into hosting the bloody thing)

Log files

d3d11.log: d3d11.log
dxgi.log: dxgi.log

Also, fun in dmesg:
[229371.058294] NVRM: Xid (PCI:0000:0b:00): 69, Class Error: ChId 004b, Class 0000b197, Offset 00001168, Data 00000600, ErrorCode 00000005

Rendering issue gone, appears to renders correctly as of 8f134ba.

Good to hear.
Could you bisect the exact commit that fixed it? Knowing what exactly fixed the issue might help fixing similar issues in the future.


Improve the CONTRIBUTING.md

The goal of this issue is to make the CONTRIBUTING.md less "heavy" for the contributors. Indeed, the more the file is big, the less they are going to read it.
Here is what I suggest:

the Release Process section was in the Development Workflow: I suggest to move it to a separated section and to specify this section concerns only the Administrators, I mean the Meili team. If the contributors are curious, they can read. If they don't care, they know they can skip this section.
Add a Bors related section, but the section is really small and redirects to the Bors guide I created: the goal is to mention it but not to make the CONTRIBUTING file too heavy. As this section only concerns the Meili team, adding a redirection is not impactful.
Make the Automated Changelogs section smaller: as for the Bors section, the purpose is to reduce the CONTRIBUTING.md file and to redirect to the release-drafter guide. See #49.

I created a PR so that you can approve on it! https://github.com/meilisearch/integration-guides/pull/50
I know a lot other things could be improved, this is just the first steps of a general change 😉

Done!


Missing documentation for MatAutocompleteTrigger

Missing instruction on how to implement the following solution:
I have a material autocomplete field as follows:
<mat-form-field class="example-full-width">     <input matInput placeholder="State" aria-label="State" [matAutocomplete]="auto" [formControl]="stateCtrl" (keyup.Esc)="onEscape($event)">     <mat-autocomplete #auto="matAutocomplete">       <mat-option *ngFor="let state of filteredStates | async" [value]="state.name">                  {{state.name}} |         Population: {{state.population}}       </mat-option>     </mat-autocomplete>   </mat-form-field>
the onEscape is defined in the .ts file as follows:
onEscape(event): void {
// what to go here ???
}
What I would like to happen is if the suggestion panel is open and/or the user has selected an option, at that point I would like to allow the Escape key to programmatically return the auto complete to the state it would be in when the component loads and the mouse clicks on the input field.
There are several, what appear to be useful properties and methods, one that stands out is MatAutocompleteTrigger._handleKeydown
But no documentation anywhere on how to implement this.
I have a hunch, maybe I should try something like this:
ngAfterViewInit(){
this.trigger.panelClosingActions.subscribe(options => {
// but what to put here ? - I'm a noob with rxjs stuff :(
});
}
There are jo examples anywhere on how to make use of MatAutocompleteTrigger.
Things I've already tried in the onEscape method:

this.stateCtrl.setValue(''); - this correctly blanks the field but doesn't cause the blur and re-focus behaviour that you normally get from clicking outside the control then back in it .

Solved :)
onEscape(): void {
this.trigger.openPanel();
this.stateCtrl.setValue('');
}


Updates to Filter UI

Describe requirements
I would like to request the following 2 enhancement to the filter UI in Manager:

Remove the Zoom to button (and row). The button doesn't add much to the app as its in a modal and hard to see what happens when you click it. Also the row adds a lot of white space to the UI, so hiding/removing would improve the spacing,
There is a grey box around the Reset and Close buttons that looks weird because there is also white background border around this section. The grey line above also doesn't stretch the full width of the modal window.

I made the following changes through the browser debugger to hide the zoom to row and to remove the padding around the filter component in the modal. I think it looks better, but it would be nice if we could add a little padding on the left and right side of the elements inside the filter. I am not sure if this would be a change to the Filter component to support it.

Alternatives
No response
Other Relevant Info
No response
Impact
No response

first two issues are fixed, no zoom button is showing and the grey line stretches the full width of the modal window, only issue remaining is adding a bit of padding on the left and right, may be also vertically align the user input dropdowns with the Reset and Close buttons

@jmhauck
It seems like this needs to be fixed in instant-apps-filter-list component, could you please confirm if any changes required from our side

@sumitzarkar yes...changes for this will need to happen in the instant app component

@sumitzarkar is this fixed in the latest code drop?

@azizaparveen
We don't have any changes for this in manager component. This needs to be fixed in Instant app component
FYI @jmhauck

Logged an issue in IA template repo https://devtopia.esri.com/WebGIS/arcgis-portal-app-templates/issues/10082

Closing this one and tracking the padding issue in https://github.com/Esri/solutions-components/issues/1041


get multi store instance with same structure and no unique id

What problem is this solving
We have a complicated component which would be instanced many times.
Each instance should have its own store. But these stores also have the same structure.
Proposed solution

Then they shouldn’t be stores, there should be entries in your list array


Can't run Kahoot-flood or Kahoot-rand

I have fully reinstalled the latest Go version and latest gorilla and also the newest push of Kahoot-Hack.
Problem: Cant run the Go program
example:
$ go run /usr/local/go/src/github.com/unixpickle/kahoot-hack/kahoot/kahoot-flood/main.go
command-line-arguments
/usr/local/go/src/github.com/unixpickle/kahoot-hack/kahoot/kahoot-flood/main.go:12: imported and not used: "github.com/unixpickle/kahoot-hack/kahoot" as kahoothack
/usr/local/go/src/github.com/unixpickle/kahoot-hack/kahoot/kahoot-flood/main.go:29: undefined: kahoot in kahoot.NewConn
ALSO I tried this:
$ cd /usr/local/go/src/github.com/unixpickle/kahoot-hack/kahoot/kahoot-rand/
Hidden:kahoot-rand$ go run main.go
command-line-arguments
./main.go:17: imported and not used: "github.com/unixpickle/kahoot-hack/kahoot" as kahoothack
./main.go:65: undefined: kahoot in kahoot.NewConn
./main.go:87: undefined: kahoot in kahoot.NewQuiz
./main.go:99: undefined: kahoot in kahoot.QuestionAnswers

Can someone please tell me what I am doing wrong?
Thanks all

I don't know if it is related but in the past week or so it seems that Kahoot have changed their protocol.

Are there any pull requests to fix it?
On Monday, October 10, 2016, msemple1111<EMAIL_ADDRESS>wrote:

I don't know if it is related but in the past week or so it seems that
Kahoot have changed their protocol.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/unixpickle/kahoot-hack/issues/45#issuecomment-252748984,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALvwGuE1LScZS2A9UAJNeIqzY6ACy4dGks5qyqupgaJpZM4KIILJ
.

--
Sent from Grant IPhone

I have a python port of kahoot-hack and this also doesn't not work. From the errors you can see that the responses are not what they should be. I am going to look into it more on the weekend when I've got some free time. When i find what they have changed, ill submit an issue here with what need to be fixed.

Ok thanks
On Monday, October 10, 2016, msemple1111<EMAIL_ADDRESS>wrote:

I have a python port of kahoot-hack and this also doesn't not work. From
the errors you can see that the responses are not what they should be. I am
going to look into it more on the weekend when I've got some free time.
When i find what they have changed, ill submit an issue here with what need
to be fixed.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/unixpickle/kahoot-hack/issues/45#issuecomment-252753775,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALvwGlG467K1CTLmnfiU_-PASdNdx08hks5qyrEIgaJpZM4KIILJ
.

--
Sent from Grant IPhone

This seems like an instance of someone failing to use Go properly, rather than a problem with the kahoot-hack repository itself. Thus, I am closing the issue.


OLM metadata for 0.12.2

Fixes #
Proposed Changes

OLM metadata for 0.12.2

Release Note

NONE

OperatorHub PR is already merged: https://github.com/operator-framework/community-operators/pull/1280
Note: we had to skip 0.12.1 as it had a bug that was causing some problems with OperatorHub verification tests

/assign @houshengbo

/lgtm
/approve

/lgtm
/approve


[Android] Blocking on runInTransaction

Hi,
I'm trying to initially populate a database in the MainActivity.
The observerable containing the insert operations are defined in the CreateLotteries helper. Since the data comprises entries in different tables, they should run in an transaction.
However on executing the transaction (or subscribing to it), the app blocks and logs W/SQLiteConnectionPool: The connection pool for database '/data/user/0/app/databases/default' has been unable to grant a connection.
The population seems to work, if the insert is done without the transaction.
I attached an excerpt of the datamodel in a gist.
Am I doing something wrong, my conversion of the transaction parameter looks a little bit strange to me.
Thanks in advance.

Hi if you remove the observeOn(Schedulers.io()) do you still encounter the same problem?

Yes, the behaviour stay the same with observable.subscribe().

When testing different transaction modes, I found out that

TransactionMode.NONE inserts the data as excepted (but without transactions)
TransactionMode.MANAGED results in a java.lang.NoClassDefFoundError: Failed resolution of: Lio/requery/sql/ManagedTransaction;
TransactionMode.AUTO leads to above blocking behaviour

The thread dump in blocking state, looks like a deadlock. Maybe one transaction wants to start before another one is finished?
"main@4348" prio=5 tid=0x1 nid=NA waiting
  java.lang.Thread.State: WAITING
	 blocks main@4348
	  at java.lang.Object.wait(Object.java:-1)
	  at java.lang.Thread.parkFor$(Thread.java:2127)
	  - locked <0x14bd> (a java.lang.Object)
	  at sun.misc.Unsafe.park(Unsafe.java:325)
	  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:161)
	  at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:840)
	  at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
	  at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
	  at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:203)
	  at rx.internal.util.BlockingUtils.awaitForComplete(BlockingUtils.java:49)
	  at rx.singles.BlockingSingle.value(BlockingSingle.java:85)
	  at io.requery.rx.SingleEntityStoreFromBlocking$17$1.call(SingleEntityStoreFromBlocking.java:335)
	  at io.requery.rx.SingleEntityStoreFromBlocking$17$1.call(SingleEntityStoreFromBlocking.java:329)
	  at io.requery.sql.EntityDataStore.runInTransaction(EntityDataStore.java:525)
	  at io.requery.sql.EntityDataStore.runInTransaction(EntityDataStore.java:535)
	  at io.requery.rx.SingleEntityStoreFromBlocking$17.call(SingleEntityStoreFromBlocking.java:329)
	  at io.requery.rx.SingleEntityStoreFromBlocking$17.call(SingleEntityStoreFromBlocking.java:326)
	  at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
	  at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
	  at rx.Observable.subscribe(Observable.java:10296)
	  at rx.Observable.subscribe(Observable.java:10263)
	  at rx.Observable.subscribe(Observable.java:10038)
	  at MainActivity.populateDatabase(MainActivity.java:121)

"pool-1-thread-1@5305" prio=5 tid=0x79 nid=NA waiting
  java.lang.Thread.State: WAITING
	 blocks pool-1-thread-1@5305
	  at java.lang.Object.wait(Object.java:-1)
	  at java.lang.Thread.parkFor$(Thread.java:2127)
	  - locked <0x14c7> (a java.lang.Object)
	  at sun.misc.Unsafe.park(Unsafe.java:325)
	  at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:201)
	  at android.database.sqlite.SQLiteConnectionPool.waitForConnection(SQLiteConnectionPool.java:670)
	  at android.database.sqlite.SQLiteConnectionPool.acquireConnection(SQLiteConnectionPool.java:348)
	  at android.database.sqlite.SQLiteSession.acquireConnection(SQLiteSession.java:894)
	  at android.database.sqlite.SQLiteSession.beginTransactionUnchecked(SQLiteSession.java:310)
	  at android.database.sqlite.SQLiteSession.beginTransaction(SQLiteSession.java:298)
	  at android.database.sqlite.SQLiteDatabase.beginTransaction(SQLiteDatabase.java:507)
	  at android.database.sqlite.SQLiteDatabase.beginTransactionNonExclusive(SQLiteDatabase.java:442)
	  at io.requery.android.sqlite.SqliteConnection.ensureTransaction(SqliteConnection.java:56)
	  at io.requery.android.sqlite.BaseConnection.setAutoCommit(BaseConnection.java:188)
	  at io.requery.sql.ConnectionTransaction.begin(ConnectionTransaction.java:81)
	  at io.requery.sql.ThreadLocalTransaction.begin(ThreadLocalTransaction.java:66)
	  at io.requery.sql.ThreadLocalTransaction.begin(ThreadLocalTransaction.java:48)
	  at io.requery.sql.TransactionScope.<init>(TransactionScope.java:39)
	  at io.requery.sql.TransactionScope.<init>(TransactionScope.java:33)
	  at io.requery.sql.EntityDataStore.insert(EntityDataStore.java:195)
	  at io.requery.sql.EntityDataStore.insert(EntityDataStore.java:189)
	  at io.requery.rx.SingleEntityStoreFromBlocking$1.get(SingleEntityStoreFromBlocking.java:83)
	  at io.requery.rx.SingleOnSubscribeFromSupplier.call(SingleOnSubscribeFromSupplier.java:34)
	  at io.requery.rx.SingleOnSubscribeFromSupplier.call(SingleOnSubscribeFromSupplier.java:23)
	  at rx.Single.subscribe(Single.java:1798)
	  at rx.Single$15$1.call(Single.java:1870)
	  at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55)
	  at rx.internal.schedulers.ExecutorScheduler$ExecutorSchedulerWorker.run(ExecutorScheduler.java:107)
	  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
	  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
	  at java.lang.Thread.run(Thread.java:761)

Thanks for the update, working on reproducing it

I stripped down the app to a minimal scenario reproducing the issue.

Hi thanks for the detailed example. There were some significant issues with the original implementation that are now fixed. However on Android you have to extra careful to use a single threaded rx scheduler (e.g. not Schedulers.io()) for all write operations to prevent the connection not available problem.

Great to hear that you resolved the issue to fast. I will be glad to test this with single threaded scheduler, Schedulers.from(Executors.newSingleThreadExecutor()). Thanks for the hint!


Is there a way of disabling automatic identification?

Hi, is there a property or something like that to disable the automatic identification?

@jzabala There currently isn't in this package, if you don't want to do any automatic identification and only automatically track page views. Remove this package and use the base package auto-analytics that this package depends on.
e.g -
meteor remove okgrow:analytics
meteor npm install @okgrow/auto-analytics --save

Thanks for your response.
I ended doing that.

@jzabala I should have mentioned if you want to see how to set it up in a Meteor App, check out the React Router example here


feat: new function - useQRCode

Looking good! One little thing is that it's better to be named as useQRCode.

PS. updating the readme link is not required, I will handle that. If you want to do it, don't do it by hand, just yarn prepare ;)

Never mind, I will do that! Thanks!

Changes:

Renamed to useQRCode
Accept a second arg for options passed to QRCode

Released in beta.32.
Thanks!


Roaring

Coverage increased (+0.4%) to 77.675% when pulling b7275800abee1d1f3639b5fb32e5a1e200f3c394 on roaring into 33703267b3b70d8a9b810aca653f90bf945fe57e on master.

Coverage increased (+0.4%) to 77.675% when pulling b7275800abee1d1f3639b5fb32e5a1e200f3c394 on roaring into 33703267b3b70d8a9b810aca653f90bf945fe57e on master.

Coverage increased (+0.4%) to 77.675% when pulling b7275800abee1d1f3639b5fb32e5a1e200f3c394 on roaring into 33703267b3b70d8a9b810aca653f90bf945fe57e on master.

Coverage increased (+0.5%) to 77.717% when pulling 4e400c8aea6873df4294816a8143c74514230e3f on roaring into 33703267b3b70d8a9b810aca653f90bf945fe57e on master.


Initial setup of Authentication Frontend repo

Description
We wish to build a frontend for authentication with similar solutions as access management
(https://github.com/Altinn/altinn-access-management-frontend), to keep a consistent profile for the two authorization teams.
Additional Information
We plan to transfer much code from the access management solution, and modify it to our needs.
### Tasks
- [ ] https://github.com/Altinn/altinn-authentication-frontend/issues/1
- [x] Transfer code
- [x] Modify code to run in new repo
- [x] Test code by running on Mac and PC

The frontend sub-repo is now well established.


Add web interface to add/remove registered dataverses

Check dataset urls to see if the dataverse itself is registered

Example of fail message

Now mapping is disabled:


Add test cases for the parking controller

[x] Add examples using 1p and 4p kite model
[x] implement a well defined disturbance, e.g. fly to the right with 10% steering for one second
[x] plot the disturbance response

Now implemented in the examples parking_1p.jl and parking_4p.jl


Windows installation fails: libvips 8.2.0 no longer downloadable

Sharp 0.12.2 produces the following error on Windows installations:

ERROR: https://dl.bintray.com/lovell/sharp/libvips-8.2.0-win.tar.gz status code 404

It appears that the only available version via bintray is 8.3.1.

Please see https://github.com/lovell/sharp/issues/424 (and https://github.com/lovell/sharp/issues/449).


delete unused variable

I have deleted the unused variable and fix the issue

Hi @ChangweiZhang, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

@NeerajW


Added error handling when receiving errors from ifttt

this should fix issue #11, and prevents any further issues when parsing the error object returned from ifttt

Coverage decreased (-5.1%) to 42.857% when pulling ee76f587eca2144deea57dda48e4e52d26ad09c0 on SenseTecnic:fix-json-error into 4e59a0b86071aa699ddbfd8d2cd447f069f39f41 on diegopamio:master.

Can you see what broke the Travis tests? Also it's hard committing new code without tests - even though it's essentially bug-fix code.

I think the errors were all related to javascript lint...... other than that, it seems like the test file ifttt-spec.js still have "TODO: Fix nock tests" for most of the test cases. Not sure what would be the correct way to add test code for now.

Coverage decreased (-6.6%) to 41.379% when pulling ceb1df15ec1fab046784d8ee71e854a6b4ff4d57 on SenseTecnic:fix-json-error into 4e59a0b86071aa699ddbfd8d2cd447f069f39f41 on diegopamio:master.

This looks good - thanks for your work!

Published to NPM as 0.1.0.  Still not 1.x worthy because too many NPM install warnings about outdated libraries.


Progressbar display loss nan.

Progress bar display loss nan.
When start training, progress bar display loss: nan. I tested more than 10 models but, all model's loss displayed nan.
I made a test file and took a log to see if the loss was really nan, and the loss was displayed normally here.
Below are some of the areas that I have implemented that may be relevant to this. If there is anything strange, please give me feedback.
class Model(pl.LightningModule):
    def configure_optimizers(self) -> Dict[str, Union[torch.optim.Optimizer, object, str]]:
        r"""
        Configure optimizer for training.

        Returns:
            dict: dictionary contains `optimizer`, `scheduler`, `monitor`
        """
        SUPPORTED_OPTIMIZERS = {
            "adam": Adam,
            "adamp": AdamP,
            "radam": RAdam,
            "adagrad": Adagrad,
            "adadelta": Adadelta,
            "adamax": Adamax,
            "adamw": AdamW,
            "sgd": SGD,
            "asgd": ASGD,
            "novograd": Novograd,
        }
        assert self.configs.model.optimizer in SUPPORTED_OPTIMIZERS.keys(), \
            f"Unsupported Optimizer: {self.configs.model.optimizer}\n" \
            f"Supported Optimizers: {SUPPORTED_OPTIMIZERS.keys()}"

        self.optimizer = SUPPORTED_OPTIMIZERS[self.configs.model.optimizer](
            self.parameters(),
            lr=self.configs.lr_scheduler.lr,
        )
        self.scheduler = SCHEDULER_REGISTRY[self.configs.lr_scheduler.scheduler_name](
            optimizer=self.optimizer,
            configs=self.configs,
        )

        return {
            'optimizer': self.optimizer,
            'scheduler': self.scheduler,
            'monitor': 'metric_to_track',
        }

    def optimizer_step(
            self,
            epoch=None,
            batch_idx=None,
            optimizer=None,
            optimizer_idx=None,
            optimizer_closure=None,
            on_tpu=None,
            using_native_amp=None,
            using_lbfgs=None,
    ) -> None:
        self.optimizer.step()
        self.optimizer.zero_grad()
        if (isinstance(self.scheduler, ReduceLROnPlateauScheduler)
            or isinstance(self.scheduler, WarmupReduceLROnPlateauScheduler)) \
                and self.trainer.global_step % self.trainer.val_check_interval == 0:
            self.scheduler.step(self.current_val_loss)
        else:
            self.scheduler.step()

    def configure_criterion(self, criterion_name: str) -> nn.Module:
         ...

Hi @sooftware, thanks for the issue :smiley: Could you provide a reproducible script, ideally using our BoringModel, showing the bug in action? Without that it will be difficult for us to assist you.


Curious page breaks in tables

I'm on the way migrating some Latex projects to AsciiDoctor and have some issues.
One of my documents is a questionaire.
My tables for a questionaire are defined as
|===
2.5+|Q-02 18+|What is the correct answer?
| [A] 17+|Option A is the correct answer
| [B] 17+|Option B is the correct answer
| [C] 17+|Option C is the correct answer
| [D] 17+|Option D is the correct answer
|===
For some reasons the [cols=10,5,90] didn't work together with column-spans, so I found the solution above which works fine for me.
When I place several question-tables in the document, I got the result like the attached document (minimal example)
(First column on one page, rest of the table on the next page; in some cases the text of the question disappeared)
Is there a trick? I already tried [options=unbreakable], which did't change the behavior. The manual mentioned, that it only would work with DocBook.
Should I change my workflow to asciidoc -> docbook -> pdf?
Regards Frank
test.pdf

This is a known bug in Prawn table. Asciidoctor PDF doesn't actually handle laying out the table. That is up to the Prawn table library. This is a duplicate of https://github.com/asciidoctor/asciidoctor-pdf/issues/403 and has been reported upstream.

Asciidoctor PDF does not support unbreakable tables. One reason is that it's far too expensive. The second reason is that the table layout is handled by Prawn table and it really needs to be added there first before we can use it.

Related to #2022

In the upcoming release of Asciidoctor PDF 2.0, you will be able to prevent an orphaned table caption by assigning the option breakable to a table.
.Optional table caption/title
[%breakable]
|===
|Content |Content

|Content |Content
|===

Note that the table will still break if it flows from the current page to the next page. The breakable option only prevents the table caption (and ID, if one is assigned) from being orphaned. (We decided it was best if people choose to opt-in to this behavior due to the potential performance costs in some cases.)
Also, you'll be able to assign the unbreakable option to a table if you don't want it to break and flow to the next page when it doesn't fit on the current page.
.Optional table caption/title
[%unbreakable%header,cols="2*"]
|===
|Content
|Content

|Content
|Content
|===


Wrong pagination total count with global scope

Following my ticket in scout package repository: https://github.com/laravel/scout/issues/345
I have an issue with pagination, it shows wrong total pages count Post::search($request->get('q'))->paginate(); returns total 9 posts while while actual posts count in the collection is just 4.
Looks like the issue is caused by global scope, applied to a model.
Global scope is applied to get, but not count. For example Post::search($request->get('q'))->get(); returns 4 records, but Post::search($request->get('q'))->count() returns 9.
"laravel/framework": "5.7.*",
"laravel/scout": "^6.1",
"babenkoivan/scout-elasticsearch-driver": "^3.9",

I would imagine that you are updating models without the global scope enabled so docs are getting added that wouldn't normally be imported.
Doing the collection before pagination would break a lot of implementations as mysql cannot handle passing thousands of ids to select before doing the limit. So its just getting the ids for that page of the pagination and then your global scope is further limiting. I would recommend trying to mimic that scope in your scout search to reduce the difference or using withoutSyncingToSearch(function () {     // Perform model actions... }); when working on models with global scopes disabled

@vedmant, could you please provide a Minimal, Complete, and Verifiable example?

We have a similar issue with the TNTSearch driver, so my guess it's Scout doing some odd things.


Update createRecord.md to note images in offline

Making a note in the createRecord to mention that images are not supported in offline mode.

Responded via email and closing.


print_rich not working

Godot version
daa055d479916b9fdb9b6cf3bc6aa822f35a2be6
System information
Windows 11
Issue description
Using that function in the code has no effect on the output eg the provided input:
	print_rich("[code][b]Hello world![/b][/code]")

Simply prints out  [code][b]Hello world![/b][/code] without effects which should be applied.

Color and other tags are not working for me too.
Steps to reproduce
Create a script - paste code with print_rich and run the project
Minimal reproduction project
Too easy to reproduce.

I'll quickly check this when I have time, when I rebased I saw it working within the terminal. I think I know what the issue may be.

When running the project it prints as expected in the terminal, however, it does not do so inside the Godot editor.
In print_string.cpp on line 172 the while loop get's skipped for some reason not passing the bool for rich text. When I made the print_rich() function I had this problem in the beginning as well, but then I started making the script inside of the Godot editor bigger to print more different kinds of rich text and it started working as expected in the Godot editor as well. My test project works find, but indeed when only creating a program with a single print_rich, then it only gets printed correctly in the terminal.
Will try to fix this asap!

Was able to fix it. Wasn't easy as vs code was not working along nicely (as usual). Just passing through the correct type was all that was needed.

@Calinou Sorry about this, fixed it as soon as I could.

No worries :slightly_smiling_face: Thanks for looking into fixing this!


Latest mapnik at 98c26bc

Changes:

mapnik 3.1.x at https://github.com/mapnik/mapnik/commits/98c26bc
mason version recent enough to have the mapnik 98c26bc
starts pulling the mason version consistently via the scripts/setup.sh like node-cpp-skel does
updates a few test fixtures to allow the tests to pass with this mapnik version (unclear what changed, but looks non-visual).

Motivation here is to get #817 done.

updates a few test fixtures to allow the tests to pass with this mapnik version (unclear what changed, but looks non-visual).

I think those little changes are caused by https://github.com/mapnik/mapnik/commit/f3cf1ad38836de315c51da83b0aa192d0499b83d.


Unable to open CircularFloatingActionMenu programatically

getting error when performing click on FAB using perform click
java.lang.RuntimeException: All of the sub action items have to be independent from a parent.

Hey, this is a workaround that works.
Actually, I tried with only actionMenu.open(true); and actionMenu.toggle(true); and both returned  "All of the sub action items have to be independent from a parent."
So, after firing this call a couple of milliseconds later, it works.
  final FloatingActionMenu actionMenu = new FloatingActionMenu.Builder(getActivity())
                .setEndAngle(360)
                .setStartAngle(0)
                .setRadius(getResources().getDimensionPixelSize(R.dimen.radius_large))
                .addSubActionView(view1)
                .addSubActionView(view2)
                .attachTo(mCircleMenu)
                .build();

        mCircleMenu.postDelayed(new Runnable() {
            @Override
            public void run() {
                actionMenu.open(true);
            }
        }, 200);


3.0-M1 generates invalid empty parameters section

Specs 2.0 seem to expect no parameters rather than empty json array (parameters : [ ]).
Once trying to generate 2.0 spec using 3.0-M1 version of the plugin, having:
@ApiOperation(value = "Returns foo", response = Foo.class)
@GET
@Produces("application/json")
public Foo get() {

generates invalid chunk:
    "produces" : [ "application/json" ],
    "parameters" : [ ],
    "responses" : {

where valid would be:
    "produces" : [ "application/json" ],
    "responses" : {

some mixed feelings here, as they seem to be fixes:
in: https://github.com/swagger-api/swagger-editor/issues/395
and in: https://github.com/apigee-127/swagger-tools/issues/189
still, it works the way it's implemented right now in the 3.0-M1 build and I would like not to see it reopened again for whatever outdated component (either generation part, or swagger-ui part).


Docs are failing

The added wurlitzer dependency is causing docs to fail. Should be a similar fix to numpy?

Fixed by https://github.com/manodeep/Corrfunc/commit/0dce74053bcf5e58914b9fcb113c7d9aede4279e


The counter example, with Dwarf debugging (breakpoints, single stepping in vscode and the browser)

Hi,
I've recently been looking into if it's possible to single step / debug wasm rust within the browser or vscode
It turns out it is possible although there doesn't seem to be much in the way of examples or documentation on getting it to work.
So I've put together an example of the counter that also can do debugging.

Readme lists what's needed to get it to work.
index.html modified slightly.
I've included examples for the .vscode files for this one since it's main focus is on how to get debugging to work easily

So far I've only really tried this under the windows environment.

Cool! I am wondering if this belongs in examples or in projects — I am trying to move more things into projects over time because the maintenance burden for examples is too high with the number we have in there right now, but it's great to be able to point to things saved somewhere.

I've now moved this to projects, it's not perfect in that you can add breakpoints within chrome and visual studio code which is pretty cool, but so far I can't see what's inside variables with the example setup (so no variable inspection).

Awesome — Thanks! The write-up in the README is really helpful.


Potential security issue

Hey there!
I belong to an open source security research community, and a member (@rudloff) has found an issue, but doesn’t know the best way to disclose it.
If not a hassle, might you kindly add a SECURITY.md file with an email, or another contact method? GitHub recommends this best practice to ensure security issues are responsibly disclosed, and it would serve as a simple instruction for security researchers in the future.
Thank you for your consideration, and I look forward to hearing from you!
(cc @huntr-helper)

Done
https://github.com/Athlon1600/youtube-downloader/blob/master/SECURITY.md

Great, we sent you an e-mail! 👍
You can also view the report directly here:
https://huntr.dev/bounties/e6f2d10e-be4d-4d0d-9068-fc7a6dc0e2fc/


[examples/chat] Why do peers have to be their own friends?

Context
Asymmetry in friend set
Suppose I have the following network of 2 friends:

Bootstrap peer / friend 1
cargo run --release -- --me=1@9001 --friends=1,2

Friend 2
cargo run --release -- --me=2@9002 --friends=1 --bootstrappers=1@<IP_ADDRESS>:9001

Then every 3 seconds or so, friend 2 will croak and restart

[10/03 07:23:29|DEBUG] commonware_p2p::authenticated::actors::tracker::actor => failed to handle bit vector (error="BitVecExtraBit")
[10/03 07:23:29|INFO] commonware_p2p::authenticated::actors::spawner::actor => peer shutdown (error="PeerKilled("ff87a0..
[10/03 07:23:29|DEBUG] commonware_p2p::authenticated::actors::router::actor => peer released (peer="ff87a0..

Friend 1 will also report this as debug messages:

[10/03 07:24:29|INFO] commonware_p2p::authenticated::actors::spawner::actor => peer shutdown (error="ReceiveFailed(StreamClosed)" peer="191fc38f134aaf1b7f
[10/03 07:24:29|DEBUG] commonware_p2p::authenticated::actors::router::actor => peer released (peer="191fc38f134aaf1b7fdb1f86330b9d03e94bd4ba884f490389de96

Thanks to retries, the two friends will re-connect after each failure of friend 2.
Symmetry
Interestingly, if the 2 peers are symmetric in their friend set, then the error no longer appears. By symmetric I mean that:

peer 1 is friend with 2 (not with itself)
peer 2 is friend with 1 (not with itself)

Bootstrap peer / friend 1
cargo run --release -- --me=1@9001 --friends=2

Friend 2
cargo run --release -- --me=2@9002 --friends=1 --bootstrappers=1@<IP_ADDRESS>:9001

No error noticed in logs. The chat works as expected as far as I can tell.
Questions

Why does peer 2 have to have itself as a friend to avoid the error? And why does the error disappear when peer 1 is not his own friend?
Unrelated to the scenarios above -- does commonware have support to discover new peers joining the network? More specifically: Suppose that peer 2 uses peer 1 for bootstrap, and that peer 3 uses peer 2 for bootstrap, then can commonware provide to peer 1 dialing information related to peer 3 (and provide to peer 3 dialing information related to peer 1)?

Unrelated to the scenarios above -- does commonware have support to discover new peers joining the network? More specifically: Suppose that peer 2 uses peer 1 for bootstrap, and that peer 3 uses peer 2 for bootstrap, then can commonware provide to peer 1 dialing information related to peer 3 (and provide to peer 3 dialing information related to peer 1)?

I think I found the answer is: This is only possible if peer 1 knows the pubkey of peer 2 and peer 3 (and the other two peers respectively each know the pub keys of the others). I think this model makes sense. It assumes (valid assumption) that peers know some authentication information (and a bootstrap node) for a network they want to join. The model was not intuitive to me, but as I understand it now it makes a lot of sense.

During startup, commonware-chat registers the specified "friends" (i.e. peers) at index=0:  https://github.com/commonwarexyz/monorepo/blob/ee0b909129c75153a5f12ef062ed9cae9e29475a/examples/chat/src/main.rs#L161-L165
As documented, peer discovery relies heavily on the assumption that peers are known at a given index: https://github.com/commonwarexyz/monorepo/blob/ee0b909129c75153a5f12ef062ed9cae9e29475a/p2p/src/authenticated/mod.rs#L91-L133
I failed to mention in those docs, however, that it is also assumed that the list of peers at a given index is synchronized. I'll add that now.

Interestingly, if the 2 peers are symmetric in their friend set, then the error no longer appears.

Regarding your point on symmetry, all that we can check on a bit vector is that its length is expected. In your example, the length is expected and your directly dial a peer (2 dialing 1) that explicitly whitelists you. To prevent this issue, we could add a hash to these messages (over all members at an index to detect this).

peer 1 is friend with 2 (not with itself)
peer 2 is friend with 1 (not with itself)

As for why this works, a peer will always accept a connection from someone that is in the set of allowed peers (even if it itself is not). I'm not sure if this should be considered a bug or not as running some process on commonware-p2p that is meant to connect to nothing seems useless (but semantically may be incorrect). Will ponder this.

Unrelated to the scenarios above -- does commonware have support to discover new peers joining the network? More specifically: Suppose that peer 2 uses peer 1 for bootstrap, and that peer 3 uses peer 2 for bootstrap, then can commonware provide to peer 1 dialing information related to peer 3 (and provide to peer 3 dialing information related to peer 1)?

Yes, you can run a chat example where you can see this working (where you bootstrap each terminal window of a new peer).
commonware-p2p::authenticated is documented extensively here: https://github.com/commonwarexyz/monorepo/blob/ee0b909129c75153a5f12ef062ed9cae9e29475a/p2p/src/authenticated/mod.rs#L1-L229

I think I found the answer is: This is only possible if peer 1 knows the pubkey of peer 2 and peer 3 (and the other two peers respectively each know the pub keys of the others). I think this model makes sense. It assumes (valid assumption) that peers know some authentication information (and a bootstrap node) for a network they want to join. The model was not intuitive to me, but as I understand it now it makes a lot of sense.

As mentioned earlier, yeah it is assumed in commonware-p2p::authenticated that all participants know the public keys of the other peers they want to connect to (not necessarily their IPs). As I also mentioned above, it assumes synchronization of this list at each registered index (and I had not mentioned that clearly).

Thanks for diving in and asking some good questions!


chore: update author in npm file

I'm not sure if we have an npm organisation yet. I will make one!

Okay!


[FEATURE]

Feature Description
This feature will allow the Patients to change their personal data.
Definition of Done
The patient can change their personal data.
Only the patient that is logged in can update their personal data.

Duplicated feature


bicep keep complaining about SecureString required even though parameter is marked with @secure

Bicep version
PS C:\Users\fmoeller\Desktop\RepositoryVorlage> bicep --version
Bicep CLI version 0.18.4 (1620479ac6)

Describe the bug
If I define a parameter as @secure` and string` I expect that I can assign it to a password field without warning.
This works for SQL Servers, for VMs. However, it does not worked for SSIS runtimes of ADF.
To Reproduce
Steps to reproduce the behavior:
Enter the following code
param dataFactoryName string
param location string
param ssisSqlServerName string = 'myserver'

@secure()
param SqlServerPassword string

resource dataFactoryName_resource 'Microsoft.DataFactory/factories@2018-06-01' = {
  name: dataFactoryName
  location: location
  identity: {
    type: 'SystemAssigned'
  }
}

resource ssis 'Microsoft.DataFactory/factories/integrationRuntimes@2018-06-01' = {
  parent: dataFactoryName_resource
  name: 'ssis'
  properties: {
    type: 'Managed'
    typeProperties: {
      computeProperties: {
        location: location
        nodeSize: 'Standard_D2_v3'
        numberOfNodes: 1
        maxParallelExecutionsPerNode: 8
        vNetProperties: {
          publicIPs: [

          ]
        }
      }
      ssisProperties: {
        licenseType: 'BasePrice'
        catalogInfo: {
          catalogAdminPassword: SqlServerPassword
          catalogAdminUserName: 'sqladmin'
          catalogPricingTier: 'Basic'
          catalogServerEndpoint: '${ssisSqlServerName}${environment().suffixes.sqlServerHostname}'
        }
      }
    }
  }
}

In line 36 it shows:
The property "catalogAdminPassword" expected a value of type "SecureString | null" but the provided value is of type "string". If this is an inaccuracy in the documentation, please report it to the Bicep Team.bicephttps://aka.ms/bicep-type-issues

How do I resolve the warning? In fact the password right now is not even deployed ...
Additional context
Add any other context about the problem here.

catalogAdminPassword needs to be an object:
resource ssis 'Microsoft.DataFactory/factories/integrationRuntimes@2018-06-01' = {
  ...
  properties: {
    typeProperties: {
      ...
      ssisProperties: {
        catalogInfo: {
          catalogAdminPassword: {
            type: 'SecureString'
            value: SqlServerPassword
          }
          ...
        }
      }
    }
  }
}

This one took a bit of investigation to figure out! The type declared by Microsoft.DataFactory is confusingly named because 'SecureString' is a valid type in ARM, but in this case it's a custom object defined by the resource provider.

@jeskew thank you so much. Awesome, this has solved my problem.
Your quick suppport is very apppreciated. Have a nice day.


Align langs repo variable

The alignment should be fixed as part of other changes, to reduce commit history noises.


[e2e]: container-based schedule main registry-username-secret

Repo: https://github.com/slsa-framework/example-package/tree/main
Run: https://github.com/slsa-framework/example-package/actions/runs/8548682428
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.container-based.schedule.main.registry-username-secret.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.container-based.schedule.main.registry-username-secret.yml
Trigger: schedule
Branch: main
Date: Thu Apr  4 03:08:34 UTC 2024

Repo: https://github.com/slsa-framework/example-package/tree/main
Run: https://github.com/slsa-framework/example-package/actions/runs/8564396577
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.container-based.schedule.main.registry-username-secret.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.container-based.schedule.main.registry-username-secret.yml
Trigger: schedule
Branch: main
Date: Fri Apr  5 03:14:46 UTC 2024
Tests are passing now. Closing this issue.


fix: fixing docs setup

Notes for Reviewers
This PR fixes #7792
Signed commits

[ ] Yes, I signed my commits.

Changes :

Fixed the broken link under for mesheryctl command reference under configuration-management
Added App import subcommand in the sidebar of reference.
Updated the link for installation methods under quick start correctly, initially it directed to supported platforms.
Corrected the command for setting up the env variable for Windows
from export KUBECONFIG="$(k3d kubeconfig get 'k3s-default')" to the following
$env:KUBECONFIG = "$(k3d.exe kubeconfig get 'k3s-default')"

🚀 Preview for commit 29e04e0fbecb2a8a33eb5b057dcc2da8c54052f2 at: https://64901f3d44734c3f234f158f--meshery-docs-preview.netlify.app

🚀 Preview for commit d7279bf24058117b0af245dca0358b7136016ee5 at: https://649026d2c41d51480a59bac9--meshery-docs-preview.netlify.app

@thisiskaransgit can you please tell from where i can change the reference page content.

@thisiskaransgit can you please tell from where i can change the reference page content.

@SimardeepSingh1450  Refer to this docs

🚀 Preview for commit 2fcc3fd7aca7803bad72eb15b79d602d67987fb4 at: https://6490439144734c55f74f146a--meshery-docs-preview.netlify.app

🚀 Preview for commit c2609047016c8799ea97a1885d1ca5084821985a at: https://649045500335765c2868d9dd--meshery-docs-preview.netlify.app

🚀 Preview for commit 639f8b026874d2e5fd860f226d9538d81d5563b3 at: https://64904a6e11e1f660d864fadb--meshery-docs-preview.netlify.app

@thisiskaransgit why is it behaving like this ?

@SimardeepSingh1450
Let's discuss this on Meshery Dev call. Please add this as an agenda item in the meeting minutes if you would. :)

@thisiskaransgit can you please tell why is import source-type flag under app behaving like this ?

@SimardeepSingh1450
It does not have to do with the workflow, Increase the rowspan for Application Management Table, and it will work fine

@SimardeepSingh1450, The rest looks good, incorporate the above requests and we're good to go 👍🏼

@SimardeepSingh1450, What's your progress?

@SimardeepSingh1450 ?

merge conflicts resolved

🚀 Preview for commit a75745e2ab68146d082ddb774651d67843e416b4 at: https://64a407fa4016ad008948ab2d--meshery-docs-preview.netlify.app

@SimardeepSingh1450 Let's discuss this on the Meshery Dev call. Please add this as an agenda item in the meeting minutes if you would. :)

@SimardeepSingh1450 Will you be incorporating this feedback?

Closed as completed via #8128


Q: Are object tracking & tracked object information supported?

I think the answer is no after checking the manufacturer API guide and looking through the source a bit (I haven't tried using the library), but I thought I'd ask.  I'm interested in a project where tracked object data would be used to override/implement pan/tilt tracking that keeps the tracked object centered in the overall image and was wondering if the TrackMix would support this.
Or, supposing the object detection & tracking are done separately e.g. in Frigate NVR, are there API commands to make specific (coordinate-based) pan/tilt operations? I saw the up/down left/right but it's unclear whether this moves a fixed amount per command, initiates motion that needs to be stopped, or something else.
What is GetPtzTraceSection used for?
Thanks!

No there is no position information of detected objects.
You only get True/False for detection of people, pet, vehicle etc. (and push notifications).
GetPtzTraceSection is for the limits of the autoTracking.
zoom can be controlled as a absolute number.
pan/tilt can only be controlled with move operations that will start moving the camera indefinetly untill the stop command is supplied.
Doing a specific step involves sending e.g. "pan left" then a delay of eg. 0.5 seconds then "ptz stop"
Hope this gives you enough info.
If you appreciate the reolink integration and want to support its development, please consider sponsering the upstream library or purchase Reolink products through this affiliate link.


Confirm account deletion

Describe the bug
Account deletion doesn't have confirmation, making it super easy to delete account by accident.
Additionally, users don't know that deletion of account won't actually affect other accounts that refer to this account - leaving assets and data owner-less.
To Reproduce
Steps to reproduce the behavior:
1. near delete testmewell.testnet testmewell.testnet
Expected behavior
Confirmation that user indeed wants to delete the account.
Confirm they understand that if this account owns some assets or data - it will not be transferred to beneficiary.
Check if this account owns any tokens or NFTs and provide information that they will not be removed and will be up for grubs by anyone.
User feedback reference
See: https://stackoverflow.com/questions/70313278/a-deleted-testnet-account-gets-fully-restored

We can use the near-contract-helper to identify a list of contracts that the user may own NFTs or FTs in if they implement NEP-171 and NEP-141.
NEP-171/NFTs:
Contract Helper Route: POST { accountId } to /likelyNFTs
Contract method to check ownership: ft_balance_of
NEP141/FTs:
Contract Helper Route : POST { accountId } to /likelyTokens
Contract method to check balance: nft_tokens_for_owner
cc: @volovyk-s

+1 let's do this


Improve release process

Consider to use release-it.
I'm using it in my personal projects, it makes the release much more smoother and deterministic.
It can:

Create a release commit with the same commit message template.
Create a tag, (this will trigger our release action in github)
Update the changelog automatically (based on default template, looks like this, but can be provided with a custom one)
Create a Github release

@vcapretz WDYT?

Closed by #231


Support hash

queryString.parseUrl('https://www.omg.com/page#param1&param2');

Result without hash...
{
    query: Object {  }
    url: "https://www.omg.com/page"
}

That's not currently supported. This is about query strings after all. That being said, I'm willing to merge a high-quality PR adding support for this. (Needs to add tests, docs, index.d.ts update).


Setting scope for scrape to single organization seems to still be making api calls on nested orgs

example:
`bash-4.2$ python3 iq-scrape-conf.py -u  -a : -y "Root Organization"
Proxy server not configured.
Persisted data to ./scrape/System-Config.json
Cannot find SourceControl for organization with id: 4296e6421fae4a02a81d854c69d84298
Cannot find SourceControl for organization with id: 9a1483643be1489eb5d0ecf5dc34b4fd
Cannot find SourceControl for organization with id: b4611c9e468a49cb84ee99cca773f800
Cannot find SourceControl for organization with id: 69aeb65677d8484ba6f325d68d0462c7
Cannot find SourceControl for organization with id: 18bd09dd4b4149a99cf085143e95bc63
Cannot find SourceControl for organization with id: 6997ef09e0d946eaa5379e0140f8cb6a`
the previous run minus the -y parameter resulted in saving of json files. This way takes just as long, but isnt saving the files. I believe the script should be skipping the api calls altogether if outside scope

Might speed it up if you did something like:
    executor = concurrent.futures.ThreadPoolExecutor(10)  # or howver many simultaneous api calls we wanna make     futures = [executor.submit(whatever_function) for org in organizations]     concurrent.futures.wait(futures)


[mgmt] cosmosforpostgresql release

https://github.com/Azure/sdk-release-request/issues/4993

API change check
APIView has identified API level changes in this PR and created following API reviews.
@azure/arm-cosmosdbforpostgresql


add runtime debugging switch

Some debugging is driven by struct local bools, some by define. Replace both with one global that can be toggled by right-click menu, so user can enable debugging in shipping code.

fixed