"Node not found: MultiplayerSynchronizer" when node doesn't exist on all clients

Godot version
4.0.2
System information
Windows 10
Issue description
For nodes with a MultiplayerSynchronizer child:

Error when joining a server but not instancing the node on the client, or when instancing the node only on one client
Error when deleting on only one client (toggle on/off on  MRP's GUI script)
Goes away if you uncheck public_visibility

Originally I had planned to only instance scenes on a client if they were in the same "area" but maybe I am barking up the wrong tree~?  >0<

Steps to reproduce

Start a server
Add a node to the server with a MultiplayerSynchronizer
Connect a client (but don't spawn the node)
Any client spawned after will have an error for every missing node

OR

Start a server
Connect Client
Add a node on the client side only

OR

Start a server + connect a client that share a node with a MultiSync on it
Delete the node on one client

Minimal reproduction project
Note: GUI script has a bool/toggle for deleting a node with a MultiSyncy on it after joining
Multiplayer_Test.zip

I'm experiencing the same issue when attempting to delete a node (which contains a MultiSync with local as authority, spawned in on local via MultiSpawner) on the server side, with one error message per node deleted. The error messages do not occur on clients connected to the server.

Experiencing a similar issue. Any progress on this?

For nodes that gets dynamically added/removed from tree, you want to use a MultiplayerSpawner to ensure proper replication across clients. See this article for example.
You can use the visibility API (public_visibility/set_visibility_for/is_visible_to/add_visibility_filter) to control which player will see and receive updates for a specific node/synchronizer.

Got the same problem here. I think I set up everything correctly:

Got the MultiplayerSpawner in my level, with spawn path to players-container and auto spawn list element 0 to my player.tscn.
In my player.tscn got a MultiplayerSynchronizer that synchronizes all my properties with correct visiblity

Getting the same get_node: Node not found [...]Player/1/MultiplayerSynchronizer error.
After some debugging I found out what happens in my project: The host correctly gets spawned in (Player 1) on his side.
Then when the first client joins (Player 2), on the host-side the client player gets correctly spawned (host now has: Player 1 & Player 2 nodes).
But on the client side, only the client Player get spawned in, but not the host player (so client now only has Player 2 node), which means the client throws an error that Player 1 (the Player 1 node or more precisely it's MultiplayerSynchronizer) is missing...
Am I doing something wrong here? I think I followed the tutorial and setup correctly...

What you are all likely missing is configuring your MultiplayerSpawner's Auto Spawn List to include your player scene, so that when a client joins, the host's player scene is spawned automatically and its MultiplayerSynchronizer node is in your node tree.

I'm having the same problem, but I'm using RPC calls to spawn a node in both client and server. Idk if it's relevant but I'm adding a scene into MultiplayerSpawner and instantiating an inherithed scene

What you are all likely missing is configuring your MultiplayerSpawner's Auto Spawn List to include your player scene, so that when a client joins, the host's player scene is spawned automatically and its MultiplayerSynchronizer node is in your node tree.
Not in my case.  The spawner is populated with the players.
The server:

creates a map (map is also in spawner list)
Creates a player and adds it inside a node in the map
This is the remote window:

The client:

As soon as I connect to the client there's some kind of sync between the server, trying to communicate with the client. Then I get this:

This is the remote window from client:

And this is remote tree from server:

It gets fixed if you hide the "Public Visibility" of the NetworkSynchroniser located inside  each player, which will make the props not to sync. I will try to set the visibility manually only when the scene is completely loaded as this comment recommends and see if that helps

After tinkering a little bit I arrived into the following rule of thumb:

Add every scene that needs sync into multiplayer spawner ( that's the trickiest part, inherited scenes you need to manually/programmatically add all children)
If the server spawns a scene, ok, only spawn scenes in the server
Clients must make RPC calls to spawn the scene in the server, never spawn scenes in client

This apparently works well! But I guess for more performative and non prototype projects you should use low level network
But this looks much more like a Server-Client model than a P2P where every player is responsible for its own scenes... Maybe I'm still missing something
But my rule solves this issue problem
Basically you want something similar to this:
@rpc("any_peer", "call_local", "reliable")
func add_node(nodePath: String):
	if not multiplayer.is_server():
		return
	# Always send the nodePath
	var newNode = load(nodePath).instantiate()
	# Here is the MultiplayerSpawner Spawn Path
	# Don't forget this "true" in add child
	get_node("/root/Game/Level").add_child(newNode, true)

@rpc("any_peer", "call_local", "reliable")
func remove_node(nodePath: String):
	if not multiplayer.is_server():
		return
	# Here you MUST receive a node path!
	var e = get_node(nodePath)
	if e:
		e.queue_free()

When calling from server or client  you can use
remove_node.rpc("res://your_node")
add_node.rpc("res://your_node")

# or

remove_node.rpc(yourNode.get_path())
add_node.rpc(yourNode.get_path())

Just as another note in case someone ends up here, the issue I ran into to cause something like this was accidentally calling add_child on the wrong node. E.g. if you have your players in a node in your scene called 'players' make sure when you add them to the scene you're not adding them somewhere other than what is listed as the spawn path in the MultliplayerSpawner. An obvious issue but easy to miss if you're new to godot's networking

Either if I use RPCs or a predefined spawner, and am not missing the parent node, I can confirm this error. Adding a multiplayer synchronizer to dynamically spawned objects will cause this error on despawn. Everything works as intended, but why this error set is shown in client in each despawn? Something is really of here, and I feel that there is some unexpected thing happening.
I kind of confirm this problem too... but am also new to godot....

I had the same issue while trying to fiddle around the project mentionned in the above comments (https://github.com/godotengine/tps-demo)
Succinctly, I tried to split the UI node from the menu scene in a loading, setting and login scene. Login hold the loading scene, hidden by default. The loading scene is generalist: it holds functions to load any resource and update itself with a loading bar while waiting for the ResourceLoader, and finally emit a signal when done. Its code is as follow:

var path = "res://scenes/world_screen/world_screen.tscn"

signal loaded(path: String)

@onready var loading_progress = %Progress
@onready var loading_done_timer = %DoneTimer

# Called every frame. 'delta' is the elapsed time since the previous frame.
func _process(_delta):
	if visible:
		var progress = []
		var status = ResourceLoader.load_threaded_get_status(path, progress)
		if status == ResourceLoader.THREAD_LOAD_IN_PROGRESS:
			loading_progress.value = progress[0] * 100
		elif status == ResourceLoader.THREAD_LOAD_LOADED:
			loading_progress.value = 100
			set_process(false)
			loading_done_timer.start()
		else:
			print("Error while loading level: " + str(status))
			hide()

func _load(new_path: String, cb: Callable, sub_thread: bool = true):
	show()
	path = new_path
	loaded.connect(cb)
	if ResourceLoader.has_cached(path):
		loaded.emit()
	else:
		ResourceLoader.load_threaded_request(path, "", sub_thread)

func _on_done_timer_timeout():
	loaded.emit(path)

Running it however throws the following set of errors:

This error do not happen if I do not use the loading scene and directly perform the load_threaded_request in the login scene.

The problem is that both MultiplayerSpawner and MultiplayerSynchronizer try and do their thing way to early, Before they themselves have been registered in the scene tree (if being created during runtime by any method). Thus they can't find themselves. Have all of them start with public visibility disabled, and then enabling visibility during _ready() fixes it for me some 80% of the time. Don't know what causes it to still fail sometimes, might be netcode execution weirdness.

Following many of the same tutorials and running into the same issue I've tracked down what I believe to be the issue. When an object is spawned it is replicated across already connected clients correctly. When a client connects and a spawner has already added an object like the World for some reason the object gets added and then removed. I cannot figure out why.
Here are the results of me hosting and then connecting with two clients. The Constants.mprint() function is just a helper function I have to print the multiplayer id along with my message.
I've wired up the spawned and despawned signals to the spawner here.

Then wired up the tree_entered and tree_exited signals on the level object.

Here is my results. The host and the client connected before the level spawn have the level spawn just fine and stay. However the client that connects after the level has been spawned spawns it in just fine but for some reason it exits the tree without the despawned signal being emitted.

This causes the level to not exist in the "late join" client's tree which then doesn't have the player node which doesn't have the player objects but the server's syntonizers expect there to be some. You can prevent the crash by controlling the syntonizer's visibility manually. But I cant get it spawn the level properly.

There is a comprehensive way to do it? I never found a correct way of using godot multiplayer

There is a comprehensive way to do it? I never found a correct way of using godot multiplayer

@olivatooo There is a comprehensive guide on how to spawn level and players here: https://godotengine.org/article/multiplayer-in-godot-4-0-scene-replication/
@banane42 I can't really tell what's happening, make sure you are not removing the level yourself, or try to make an MRP and open a new issue.

@banane42 I can't really tell what's happening, make sure you are not removing the level yourself, or try to make an MRP and open a new issue.

I ran into the original issue, where a spawned player's MultiplayerSynchronizer was not found when other clients connected. My setup, really for experimenting, was to build a dedicated server, have the server auto-spawn a dummy player, so when clients join they can see them.
However, I was adding the player object (to be spawned) during the server's game_manager ready function, which led to this error because the scene was not ready to add a player. At least that's what I think is happening, because once I added a 3 second timer after the ready was hit, the dummy player was added to the scene as expected, and all clients had it spawned properly.
Maybe there is a bug as this is a slightly different use case, but it's likely something that's expected to be there, some node, isn't.

I have the same issue, but the only twist is that it works where I am on the same computer, like running multiple instances of the game on the same machine. But when I open the game on two or more computers, the server computer will run into this issue. I don't know what happen at this point.

While not exactly the same issue as in this thread, the solution in this post shows how to avoid a similar issue by toggling off the public_visibility on the client MultiplayerSynchronizer and then having the server remove the node:
https://forum.godotengine.org/t/online-multiplayer-error-when-queue-free-player/90925/11
It seems there's about a 1 or 2 frame delay with MultiplayerSynchronizers that causes this error when its parent scene is removed. So turning off the public_visibility right before removing the parent scene resolved this for me. However, I still got the error if I queue_free()'d the MultiplayerSynchronizer so it seems that turning off public_visibility is the only way.


Upgrade to Jetty 9.2.9

Hi, I notice that there's a commit upgrading jetty to 9.2.9. I am wondering when is the expected release date for that change?
Thanks!

There is currently no fixed date for Dropwizard 0.8.0 or 0.8.0-rc5 but we try to release it in the first weeks of March.


Update to Camel 4

Camel 4 is coming!
Update YAKS to using Camel 4. This might also imply to update Citrus and its javax.* dependencies

Closed as solved


surround obstacle checker rarely dies in local environment with segmentation fault

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I'm convinced that this is not my fault but a bug.

Description
surround obstacle checker rarely dies (1~2%) in local environment with segmentation fault. It seems that segmentation fault is happened in buffer_core.
https://github.com/ros2/geometry2/blob/aad6f56778e06619da5f58393ff55919b5eac9eb/tf2/src/buffer_core.cpp#L882-L892
(gdb) bt
#0  0x00007fc05a9dba00 in std::_Hash_bytes(void const*, unsigned long, unsigned long) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x00007fc03c1f71b3 in tf2::BufferCore::lookupFrameNumber(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const ()
   from /opt/ros/galactic/lib/libtf2.so
#2  0x00007fc03c1f7230 in tf2::BufferCore::validateFrameId(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) const () from /opt/ros/galactic/lib/libtf2.so
#3  0x00007fc03c1f72ca in tf2::BufferCore::canTransform(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l,<PHONE_NUMBER>l> > > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) const () from /opt/ros/galactic/lib/libtf2.so
#4  0x00007fc054064e8b in tf2_ros::Buffer::canTransform(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l,<PHONE_NUMBER>l> > > const&, std::chrono::duration<long, std::ratio<1l,<PHONE_NUMBER>l> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) const ()
   from /opt/ros/galactic/lib/libtf2_ros.so
#5  0x00007fc054061657 in tf2_ros::Buffer::lookupTransform(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l,<PHONE_NUMBER>l> > > const&, std::chrono::duration<long, std::ratio<1l,<PHONE_NUMBER>l> >) const () from /opt/ros/galactic/lib/libtf2_ros.so
#6  0x00007fc03692a2be in tf2_ros::Buffer::lookupTransform (this=0x7fc014004490, target_frame="", source_frame="base_link", time=..., timeout=...)
    at /opt/ros/galactic/include/tf2_ros/buffer.h:104
#7  0x00007fc036926e0d in surround_obstacle_checker::SurroundObstacleCheckerNode::getTransform (this=0x7fc014004110, source="", target="base_link", stamp=...,
    duration_sec=0.5) at /home/autoware/git/pilot-auto.x2/src/autoware/universe/planning/surround_obstacle_checker/src/node.cpp:409
#8  0x00007fc03692692d in surround_obstacle_checker::SurroundObstacleCheckerNode::getNearestObstacleByDynamicObject (this=0x7fc014004110)
    at /home/autoware/git/pilot-auto.x2/src/autoware/universe/planning/surround_obstacle_checker/src/node.cpp:362
#9  0x00007fc0369261df in surround_obstacle_checker::SurroundObstacleCheckerNode::getNearestObstacle (this=0x7fc014004110)
    at /home/autoware/git/pilot-auto.x2/src/autoware/universe/planning/surround_obstacle_checker/src/node.cpp:307
#10 0x00007fc036925844 in surround_obstacle_checker::SurroundObstacleCheckerNode::onTimer (this=0x7fc014004110)
    at /home/autoware/git/pilot-auto.x2/src/autoware/universe/planning/surround_obstacle_checker/src/node.cpp:212
#11 0x00007fc036a37b2f in std::__invoke_impl<void, void (surround_obstacle_checker::SurroundObstacleCheckerNode::*&)(), surround_obstacle_checker::SurroundObstacleCheckerNode*&> (
    __f=@0x7fc0140a2500: (void (surround_obstacle_checker::SurroundObstacleCheckerNode::*)(class surround_obstacle_checker::SurroundObstacleCheckerNode * const)) 0x7fc036925024 <surround_obstacle_checker::SurroundObstacleCheckerNode::onTimer()>, __t=@0x7fc0140a2510: 0x7fc014004110) at /usr/include/c++/9/bits/invoke.h:73
#12 0x00007fc036a36371 in std::__invoke<void (surround_obstacle_checker::SurroundObstacleCheckerNode::*&)(), surround_obstacle_checker::SurroundObstacleCheckerNode*&> (
    __fn=@0x7fc0140a2500: (void (surround_obstacle_checker::SurroundObstacleCheckerNode::*)(class surround_obstacle_checker::SurroundObstacleCheckerNode * const)) 0x7fc036925024 <surround_obstacle_checker::SurroundObstacleCheckerNode::onTimer()>) at /usr/include/c++/9/bits/invoke.h:95
#13 0x00007fc036a3358f in std::_Bind<void (surround_obstacle_checker::SurroundObstacleCheckerNode::*(surround_obstacle_checker::SurroundObstacleCheckerNode*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) (this=0x7fc0140a2500, __args=...) at /usr/include/c++/9/functional:400
#14 0x00007fc036a2eef3 in std::_Bind<void (surround_obstacle_checker::SurroundObstacleCheckerNode::*(surround_obstacle_checker::SurroundObstacleCheckerNode*))()>::operator()<, void>() (this=0x7fc0140a2500) at /usr/include/c++/9/functional:484
#15 0x00007fc036a2279a in rclcpp::GenericTimer<std::_Bind<void (surround_obstacle_checker::SurroundObstacleCheckerNode::*(surround_obstacle_checker::SurroundObstacleCheckerNode*))()>, (void*)0>::execute_callback_delegate<std::_Bind<void (surround_obstacle_checker::SurroundObstacleCheckerNode::*(surround_obstacle_checker::SurroundObstacleCheckerNode*))()>, (void*)0>() (this=0x7fc0140a24d0) at /opt/ros/galactic/include/rclcpp/timer.hpp:223
#16 0x00007fc036a1b328 in rclcpp::GenericTimer<std::_Bind<void (surround_obstacle_checker::SurroundObstacleCheckerNode::*(surround_obstacle_checker::SurroundObstacleCheckerNode*))()>, (void*)0>::execute_callback() (this=0x7fc0140a24d0) at /opt/ros/galactic/include/rclcpp/timer.hpp:209
#17 0x00007fc05ac30f7d in rclcpp::Executor::execute_any_executable(rclcpp::AnyExecutable&) () from /opt/ros/galactic/lib/librclcpp.so
#18 0x00007fc05ac374bb in rclcpp::executors::MultiThreadedExecutor::run(unsigned long) () from /opt/ros/galactic/lib/librclcpp.so
#19 0x00007fc05aa07de4 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#20 0x00007fc05a680609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#21 0x00007fc05a843133 in clone () from /usr/lib/x86_64-linux-gnu/libc.so.6
#22 0x0000000000000000 in ?? ()
(gdb) frame 7
#7  0x00007fc036926e0d in surround_obstacle_checker::SurroundObstacleCheckerNode::getTransform (this=0x7fc014004110, source="", target="base_link", stamp=...,
    duration_sec=0.5) at /home/autoware/git/pilot-auto.x2/src/autoware/universe/planning/surround_obstacle_checker/src/node.cpp:409
409	      tf_buffer_.lookupTransform(source, target, stamp, tf2::durationFromSec(duration_sec));
(gdb) p source
$1 = ""
(gdb) p target
$2 = "base_link"
(gdb) frame 8
#8  0x00007fc03692692d in surround_obstacle_checker::SurroundObstacleCheckerNode::getNearestObstacleByDynamicObject (this=0x7fc014004110)
    at /home/autoware/git/pilot-auto.x2/src/autoware/universe/planning/surround_obstacle_checker/src/node.cpp:362
362	    getTransform(object_ptr_->header.frame_id, "base_link", object_ptr_->header.stamp, 0.5);
(gdb) p object_ptr_->header.frame_id
$3 = ""
(gdb) p object_ptr_->header.stamp
$4 = {sec = 2, nanosec = 0}

Expected behavior
surround obstacle checker never dies with any situation.
Actual behavior
surround obstacle checker rarely dies (1~2%) in local environment with segmentation fault.
Steps to reproduce

enable surround obstacle checker here
launch planning_simulator.launch.xml
set ego pose and goal
set object arround the ego

Versions
No response
Possible causes
No response
Additional context
No response

@satoshi-ota Are there any suggested placement of the obstacles to reproduce the error? Also, will you be able to do the investigation yourself or would you like someone else to help you.

@satoshi-ota Are there any suggested placement of the obstacles to reproduce the error? Also, will you be able to do the investigation yourself or would you like someone else to help you.

@mitsudome-r I guess that there is no specific condition about the obstacle placement, but this error is caused in following scenario test (the obstacle is always same placement in scenario test).

UC-001-0002_PSim_1_case2_v2

I tried but couldn't fix this error, so I want help... :pleading_face:
cc @0x126 @TomohitoAndo

I have not looked at the code, but what about the target frame being an empty string?
#6  0x00007fc03692a2be in tf2_ros::Buffer::lookupTransform (this=0x7fc014004490, target_frame="", source_frame="base_link", time=..., timeout=...)
    at /opt/ros/galactic/include/tf2_ros/buffer.h:104
#7  0x00007fc036926e0d in surround_obstacle_checker::SurroundObstacleCheckerNode::getTransform (this=0x7fc014004110, source="", target="base_link", stamp=...,
    duration_sec=0.5) at /home/autoware/git/pilot-auto.x2/src/autoware/universe/planning/surround_obstacle_checker/src/node.cpp:409

I have not checked the code but could it be the empty string as input frame being undefined behavior? What if you prevent the lookupTransform when the frame is empty?

Something is strange with the validateFrameId function, no?
https://github.com/ros2/geometry2/blob/1186ee1e1be1dfe3691f31ba7f13662b1f1e1b91/tf2/src/buffer_core.cpp#L162
There is an early return in the case the frame_id is empty. And yet in your case it manages to reach the lookupFrameNumber function. So is there a crash on the lookup of "base_link"? or could it be the function is aggressively optimized because the frame_id is assumed not to be empty?

Thanks for your comments.
Yes, as you said, the function is suspicious. There is an early return in the function, so I thought we don't have to guard empty frame id at outside the lookupTransform, but...

So is there a crash on the lookup of "base_link"? or could it be the function is aggressively optimized because the frame_id is assumed not to be empty?

I can't deny the both possibility.

@satoshi-ota
I could not reproduce the error, so could you share the scenario UC-001-0002_PSim_1_case2_v2 and the lanelet2 map used?

@mehmetdogru cc @mitsudome-r @TakaHoribe
Thanks for your contribute :+1: But unfortunatelly, I can't share the lanelet2_map because of lisence :bow:
I guessed that this strange bug maybe cause no matter whether we use scenario sim or not. So, I trying to reproduce this error in my local environment without scenario simulator. (but I could not reproduce the error, too. :pleading_face:)

Removing @mehmetdogru from assignee until there is a reproducing procedure available in public.

@satoshi-ota  Is anyone from TIER IV working on this? Since the scenario file to reproduce the error is only accessible by TIER IV, we cannot assign anyone from AWF.

I have removed high priority tag since we cannot assign anyone unless we have a way to reproduce the issue.

@satoshi-ota
@satoshi-ota friendly ping.

@satoshi-ota Is anyone from TIER IV working on this? Since the scenario file to reproduce the error is only accessible by TIER IV, we cannot assign anyone from AWF.


[Feature] Incremental strategies for delete+insert and microbatch cause unnecessary cross joins

Is this your first time submitting a feature request?

[X] I have read the expectations for open source contributors
[X] I have searched the existing issues, and I could not find an existing issue for this feature
[X] I am requesting a straightforward extension of existing dbt-snowflake functionality, rather than a Big Idea better suited to a discussion

Describe the feature
Currently, the delete+insert incremental strategy requires a "unique_key", but there's nothing actually enforcing this key be unique. I've done this before, but the issue is that it creates a cartesian join between all the records with matching (non-unique) unique_key values because of the USING. The new microbatch strategy does something similar, where if you don't have an additional predicate, the USING  bit is unnecessary. In both cases, the join explosion can take up a significant amount of resources.
Describe alternatives you've considered
For the delete+insert strategy, I would propose possibly changing the name the "unique_key" config to "incremental_key", and modifying the query to use a correlated EXISTS subquery.
delete from {{ target }}
where exists (
    select 1
    from {{ source }}
    where
    {% if incremental_key is sequence and incremental_key is not string %}
        {% for key in incremental_key %}
            {{ source }}.{{ key }} = {{ target }}.{{ key }}
            {{ "and " if not loop.last}}
        {% endfor %}
    {% else %}
        {{ source }}.{{ incremental_key }} = {{ target}}.{{ incremental_key }}
    {% endif %}
)
{% if incremental_predicates %}
    {% for predicate in incremental_predicates %}
        {{ predicate }} {{ "and" if not loop.last }}
    {% endfor %}
{% endif %};

The using {{ source }} line in the microbatch strategy should simply be deleted.
Who will this benefit?
This would benefit every who uses these incremental strategies by significantly lowering warehouse load (and therefore costs) for these types of queries.
Are you interested in contributing this feature?
I think the code above should work. It may take me a while before I have time to do all the requisite testing before creating a PR.
Anything else?
I ran one query like DELETE FROM target USING source with no additional predicates where the target table had 10 million records and the source table a billion. You can see how relatively expensive that cartesian join is.
Most Expensive Nodes
CartesianJoin 47.3%
Delete 27.0%
TableScan 1.3%

This is related to:

https://github.com/dbt-labs/dbt-core/issues/10863
https://github.com/dbt-labs/dbt-core/issues/10877

Closed by https://github.com/dbt-labs/dbt-snowflake/pull/1192


Review test suppressions for HTML

There are definitely test suppressions for issues that are closed (e.g. 3051) and a lot of test suppressions with no issue associated with them. These should be reviewed.

@alan-knight, is this being worked on in 1.13? If not, we should remove the milestone value.

It might, but not until the Dartium roll is done, so I'll remove it.


Support Universal Windows

Find some way to support UWA libs with reference assemblies.
https://github.com/StephenClearyApps/DotNetApis/blob/f4fddb60c7bcdd89782d5cce56ca5a3a9b0b970b/service/DotNetApis.Nuget/PlatformUtility.cs#L8

Microsoft.NETCore.UniversalWindowsPlatform

netcoreapp1.0 -> Microsoft.NETCore.App 1.0.5
netcoreapp1.1 -> Microsoft.NETCore.App 1.1.2
netcoreapp2.0 -> Microsoft.NETCore.App 2.0.0
netcoreapp2.1 -> Microsoft.NETCore.App 2.1.0
netcoreapp2.2 -> Microsoft.NETCore.App 2.2.0
But at runtime, uses highest patch version: https://docs.microsoft.com/en-us/dotnet/core/versions/selection#framework-dependent-apps-roll-forward
netcoreappX.Y -> Microsoft.NETCore.App X.Y.*


Changing the word subfolder to file

@ThomasKwakman commented on Wed Mar 11 2020
Hi, I guess that there is a small mistake. In the Arguments section, in the paragraph before the diagram:
mydata3.txt because it's a subfolder of a hidden folder.
=> mydata3.txt because it's a file of a hidden folder.

Détails du document
⚠ Ne pas modifier cette section. C’est obligatoire pour docs.microsoft.com ➟ Liaison des problèmes GitHub.

ID: f7ac2fe2-31da-7db6-2d5f-5091022f41d2
Version Independent ID: bbf07c2c-7006-7b94-f12f-800fa7bc67e4
Content: CREATE EXTERNAL TABLE (Transact-SQL) - SQL Server
Content Source: docs/t-sql/statements/create-external-table-transact-sql.md
Product: sql
Technology: t-sql
GitHub Login: @CarlRabeler
Microsoft Alias: carlrab

@srvbpigh commented on Wed Mar 11 2020
Hello, @ThomasKwakman
Thank you for your feedback.
We are actively reviewing your comments and will get back to you soon.
Kind regards,
Microsoft DOCS International Team

@CarlRabeler commented on Wed Mar 11 2020
@ThomasKwakman @CeciAc @srvbpigh Thomas is correct. Please make the change.
Thank you Thomas

@ThomasKwakman @CeciAc @srvbpigh -- I am looking into this document.

Thank you for your feedback.  I have updated the documentation and it should be live by tomorrow.


change static information to the dynamic one from Patient Info drop-down

Change the static information to the dynamic one from Patient Info drop-down http://prntscr.com/gyzb6p
Here should be displayed the actual information about the version

thanks @anna-otrokhova
Again need to consider where this data should come from
Browser side PulseTile info can be provided by this codebase
Server side info (eg The QEWD-Ripple version) needs to come from a server side API
I don't see a related issue linked to anywhere..
Can you take a look please/consider raising a new issue here
https://github.com/RippleOSI/Ripple-Qewd/issues
thanks
Tony

@tony-shannon
There is no update for a long time.

The response from /api/initialise now includes a new property: version.  This is the version number taken from the package.json file for Ripple-QEWD

Can this now be closed?  I believe QEWD is sending what's needed for this issue to be resolved


Fix/sankey support color

[中文版模板 / Chinese template]
🤔 This is a ...

[ ] New feature
[x] Bug fix
[ ] TypeScript definition update
[ ] Bundle size optimization
[ ] Performance optimization
[ ] Enhancement feature
[ ] Refactoring
[ ] Update dependency
[ ] Code style optimization
[ ] Test Case
[ ] Branch merge
[ ] Release
[ ] Site / documentation update
[ ] Demo update
[ ] Workflow
[ ] Other (about what?)

🔗 Related issue link
https://github.com/VisActor/VChart/issues/1027

🔗 Related PR link

🐞 Bugserver case id
650aff38d24b666ad6606b1b

💡 Background and solution

📝 Changelog

Language
Changelog

🇺🇸 English

🇨🇳 Chinese

☑ Self-Check before Merge
⚠ Please check all items below before requesting a reviewing. ⚠

[ ] Doc is updated/provided or not needed
[ ] Demo is updated/provided or not needed
[ ] TypeScript definition is updated/provided or not needed
[ ] Changelog is provided or not needed

🚀 Summary
copilot:summary
🔍 Walkthrough
copilot:walkthrough

bugserver case 报错了


Issue with /docs/tutorials/kubernetes-basics/update-intro/

This is a...

[ ] Feature Request
[x] Bug Report

Problem:
The carousel does not work.
Proposed Solution:
Page to Update:
http://kubernetes.io/docs/tutorials/kubernetes-basics/update-intro/#myCarousel

Mac os sierra 10.12.1
Chrome Version 54.0.2840.98 (64-bit)

@steveperry-53 Can you take a look at this?

I'm working on this in
PR 1773.

Fixed.
http://kubernetes.io/docs/tutorials/kubernetes-basics/scale-intro/
http://kubernetes.io/docs/tutorials/kubernetes-basics/update-intro/


Create Backup Before File Modification

Hi @TomWright,
when I use dasel I like to create a backup if it modifies a file.
Currently I use cp before running dasel to create the backup.
But I would prefer that dasel takes care of it.
What about implementing the following (global) commandline flags ?
 -b, --backup                  create a backup before writing the file
 --backup-suffix string        suffix to use for backup file (default .~)

All of the modifying commands support a -o or --out flag that allows you to change where the modified contents are written.  I expect this largely covers this need, would you agree?

Hmm, I'm unsure. How would you use the --out  flag to modify a config file and create a backup within a single command?

It's more that using the --out flag tells dasel to write the modified file to a new location.

So wouldn't  the --backup flag would be a nice addition then ?

Yeah I think it could help


Help, my backup disappeared

Device information

Neko version: newest (12/31/2021)
Android phone & version: ...

Steps to reproduce
Create backup file;
Delete app
Expected behavior
Backup file actually appears
Actual behavior
Completely disappears
Other details

Fixed it myself. I'm so sorry for bothering


Use browser stack for cross browser testing

Fixed by: https://github.com/Johann-S/bs-stepper/commit/78c3e8b9079a8a6ad64de4200f1c4b4e2042cd2e


Fixed #26369 -- Default formfield callback override

https://code.djangoproject.com/ticket/26369

I didn't see any consensus on the mailing list about adding a new setting.

Started thread here: https://groups.google.com/forum/#!topic/django-developers/TblESVmxGfw

I left my comment the mailing list thread of this already. Basically I don't think this is necessary.
However if it does get added, I think a callback is a very restricted and un-django-ish way of doing this. It would be better to define a signal that gets called when a modelform constructs the form field for a model field. Any listeners to this signal could return a new form field to replace the default. This would allow multiple listeners without adding a new setting.

(Can you remove the '@adamchainz' from your commit title please? GitHub emails me about each time you push a new one after amending/rebasing because it gets a new commit SHA.)

Ooops, absolutely !

Adam reported on the ML that this doesn't work when registering signals in the advised place, apps.py
I don't want to maintain this patch anymore and would instead focus into just making default form fields really usable out of the box by marrying a JS framework.

This has been implemented in a third party extension


Polluted state when navigating back and forth on planning poker

Remember to always provide a description duh... Can't remember where the issue was and can't reproduce anymore. Closing.

Actually I can reproduce this issue. When reloading a session you can still change your votes even after reloading the page. Then pictures like these can occur:

Interesting. Could you provide a step-by-step description in order to have something reproducible? Thanks!


why everything in the readme is inside a code block

why

why

just not used in markdown text :))))


B 20441 int

Agility ticket
Summary
Removing and replacing a bad migration file for updating unpriced MS and CS service items.
this article explains more about the approach used.
Verification Steps for the Author
These are to be checked by the author.

[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] Have the Agility acceptance criteria been met for this change?

Verification Steps for Reviewers
These are to be checked by a reviewer.

[ ] Has the branch been pulled in and checked out?
[ ] Have the BL acceptance criteria been met for this change?
[ ] Was the CircleCI build successful?
[ ] Has the code been reviewed from a standards and best practices point of view?

Setup to Run the Code

Instructions for starting storybook
Instructions for starting the MilMove application
Instructions for running tests

How to test
I set all of the locked_price_cents and pricing_estimate  with MS and CS codes to null in my local with:
"UPDATE mto_service_items AS ms
SET locked_price_cents = null,
pricing_estimate = null
FROM re_task_order_fees AS tf
JOIN re_services AS s
ON tf.service_id = s.id
JOIN re_contract_years AS cy
ON tf.contract_year_id = cy.id
JOIN re_contracts AS c
ON cy.contract_id = c.id
WHERE (s.code = 'MS' OR s.code = 'CS') AND ms.re_service_id = s.id;"
Then I manually ran the migration file in pgAdmin to ensure it ran properly and updated only MS and CS service items that were nulled. I then ran it again to ensure that it updated 0 on concurrent runs.
The following query can be used to pull service items in question:
"SELECT * FROM public.mto_service_items ms
JOIN re_services r ON ms.re_service_id = r.id
WHERE r.code = 'MS' OR r.code = 'CS'"
Be sure to check service items that do not have the MS or CS code.
Frontend

[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, Edge).
[ ] There are no new console errors in the browser devtools.
[ ] There are no new console errors in the test output.
[ ] If this PR adds a new component to Storybook, it ensures the component is fully responsive, OR if it is intentionally not, a wrapping div using the officeApp class or custom min-width styling is used to hide any states the would not be visible to the user.
[ ] This change meets the standards for Section 508 compliance.

Backend

[ ] Code follows the guidelines for Logging.
[ ] The requirements listed in Querying the Database Safely have been satisfied.

Database
Any new migrations/schema changes:

[ ] Follows our guidelines for Zero-Downtime Deploys.
[ ] Have been communicated to #g-database.
[ ] Secure migrations have been tested following the instructions in our docs.

Screenshots

conflicts


Removed flaky assertion

This test was failing because it was dependent on the execution time of the local machine. Removed test to make it more reliable.

Thanks @shazaibn, we'll flesh out that test better at a later point.


rsz: increase rebuffer buffer penalty

Signed-off-by: James Cherry<EMAIL_ADDRESS>

Thanks @jjcherry56 this fixes the issue I was seeing.

@jjcherry56 Ok to merge if PR is done. Secure CI metrics updated with: https://github.com/The-OpenROAD-Project/OpenROAD-flow-scripts/pull/609


python311Packages.denonavr: 0.11.3 -> 0.11.4

Diff: https://github.com/ol-iver/denonavr/compare/refs/tags/0.11.3...0.11.4
Changelog: https://github.com/ol-iver/denonavr/releases/tag/0.11.4
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 258419 run on x86_64-linux 1

5 packages built:

home-assistant-component-tests.denonavr
python310Packages.denonavr
python310Packages.denonavr.dist
python311Packages.denonavr
python311Packages.denonavr.dist


Fix issue #5 and #14

Fix architv/soccer-cli#5
Fix architv/soccer-cli#14
This replaces the .format() with the older % formatting.
The older method makes it easier to work with str and unicode at the
same time.
Since the % formatting is used elsewhere in the script I thought it
shouldn't really matter unless the newer .format() is really desired.

That was an important fix. Thanks @Saturn !


Remove duplicated code display

The same code snippet is displayed twice. I'm certain that I won't be the only one analysing the two code samples to try to find a difference only to realise that they're identical and thus redundant to be shown twice.
"preceding code" in the following sentence is fine to refer to both of the blocks of sample code (view and backing code for the Razor Page) and still makes sense.

@rudiv,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

I think that's why [@]Rick-Anderson has that cshtml file repeated. He had to go back to the markup file to make that point on the URL path and handlers. If that code is cut leaving "preceding code" there, one can't easily, immediately distinguish which "preceding code" it refers to.
Another possibility here would be to state prior to the markup file ...

Referring back to the cshtml file shown above:

... which would signal to the reader that a point is about to be made on the same code shown earlier.

Yeah, I can understand why it was done, but to see such similar snippets with the same lines highlighted isn't intuitive in so far as it makes the reader believe that there would be a difference to the snippet they've just seen.
I'm not sure it matters which piece of code it is referring to so much, it's simply describing how the routing works which applies to both pieces of code, or one of them?

I think he's trying to explicitly ref the asp-page-handler tho in the construction of the request URL.
Could just show those two lines and nothing else ... :hocho: the rest of it ...
<input type="submit" asp-page-handler="JoinList" value="Join" />
<input type="submit" asp-page-handler="JoinListUC" value="JOIN UC" />

That would certainly contain it better, yes!
I'm still not sure if it's needed at all given the context and its referral back to the handlers, but it's certainly better than what it is now! I'll update it now (thanks for the code snippet).

It was just a suggestion. Many of these types of changes go to the author for comment, but [@]Rick-Anderson isn't available at the moment.

@rudiv Thank you for your contribution!


Multi version of node module

I was trying to integrate my custom LLM.
In my code, I have to use<EMAIL_ADDRESS>And in Openai llm code which is already exists, it uses<EMAIL_ADDRESS>So multi version of axios module is needed.
So I fixed langchain/package.json file like this.
"devDependencies": {
...
"axios" : "^0.26.0",
"axios-bittensor"<EMAIL_ADDRESS>...
}

Almost tests, I passed but at Environment tests, I failed.
Cannot find package 'axios-bittensor' imported from /app/node_modules/langchain/dist/llms/bittensor.js
How can I solve this issue?
Here's my PR.
PR for my custom llm integration which has got errors

We'll be removing the Axios dependence from OpenAI soon (watch #2353) which should fix this!


Users cannot login to TeamRadio with their Google account

Platform: Windows
Browser: Chrome
Step to reproduce:

Access to https://www.teamrad.io/
Click button Login
Click button Login with Google

Expected result:
Users will be redirect to the Google login page & can login with their Google account
Actual result:
System show the error 400 redirect_uri_mismatch

We think it is working for now, can you please have a look again, @trinm90 ?


Adding New event Downloading_sensitive_file

Motivation
This event will Detect sensitive files are downloadable.

Feature
Adding new event downloading_sensitive_file

Alternatives

Additional context

@leogr @jasondellaluce @alacuku Please take a look at it. we have event for reading, and writing sensitive file so we should have an event for regarding downloading sensitive files or copying its data to another destination.
/assign

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

This one is not related to issue, hence closing it. Thank you.


components/Footer.vue errors

import is missing PhoneIcon
syntax error:
case "phone";
=>
case "phone":

Thanks @stuta
Fixed, please update to the new version.


feat: Add SimpleAgentComponent and related pytests

Introduce the SimpleAgentComponent, a new agent component that extends ToolCallingAgentComponent. This component allows the agent to utilize various tools and supports both OpenAI and Azure OpenAI language models. It includes configurations for selecting the language model type and manages input types dynamically based on the selected model.
https://github.com/user-attachments/assets/d4e1b8d4-e328-43a5-81ba-2c18e8b867e5
Changes

Added SimpleAgentComponent class in src/backend/base/langflow/components/agents/simple_agent.py
Defined inputs and outputs for the component, including tool handling and language model selection
Implemented methods for building and managing language models (get_llm, _build_llm_model)
Included error handling for model configuration and missing keys in build_config
Updated input types dynamically based on the selected language model

Motivation and Context
This addition enhances the agent framework by providing a straightforward agent capable of leveraging different language models and tools. It offers greater flexibility and extensibility for building complex workflows within the Langflow application.
Future Updates:

Add Graph Run inside the component so we can integrate the input component and output component in the same agent.
Ability to add Agent Name as sender Name

Autogenerated: PR Message
This pull request introduces a new SimpleAgentComponent class, adds unit tests for this component, and modifies existing tests for the ToolCallingAgentComponent. The key changes include the creation of the SimpleAgentComponent class, the addition of unit tests for both the SimpleAgentComponent and ToolCallingAgentComponent, and the integration of various input and output components.
New Components:

src/backend/base/langflow/components/agents/simple_agent.py: Added SimpleAgentComponent class that extends ToolCallingAgentComponent to handle different language models and tools. ([src/backend/base/langflow/components/agents/simple_agent.pyR1-R132](diffhunk://#diff-6bcaea75d1cdcae8ab5798ab147a2956ebbf234384835993ce0a3a3a67abe42cR1-R132))

Unit Tests:

src/backend/tests/unit/components/agents/test_simple_agent.py: Added unit test for SimpleAgentComponent to verify its functionality using a calculator tool. ([src/backend/tests/unit/components/agents/test_simple_agent.pyR1-R31](diffhunk://#diff-f2758044606f96f1378563cfaab80149810adf5f7b8b5b780b63f93d4cb8063eR1-R31))
src/backend/tests/unit/components/agents/test_tool_calling_agent.py: Added unit test for ToolCallingAgentComponent to ensure it can correctly use a calculator tool and produce expected outputs. ([src/backend/tests/unit/components/agents/test_tool_calling_agent.pyR1-R32](diffhunk://#diff-cb1e0fb42766434760104f00ff364c54550e4c9d2df99d66cd58fa3e06c401a4R1-R32))

Thank you @italojohnny

@Cristhianzl  Any suggestion on why the FrontEnd Test Group 4 always fails in this PR?

@Cristhianzl Any suggestion on why the FrontEnd Test Group 4 always fails in this PR?

Found the Error: The simplified Agent is having the name just Agent and the starter template simple Agent Flow is based of the tool calling agent. We might need to update the started template so solve this issue

@Cristhianzl Any suggestion on why the FrontEnd Test Group 4 always fails in this PR?

Found the Error: The simplified Agent is having the name just Agent and the starter template simple Agent Flow is based of the tool calling agent. We might need to update the started template so solve this issue

Updated the Agent Flow.json to use the Agent component instead of the tool calling agent, still the test fails.


Sticky Footer

Please always provide the GitHub issue(s) your PR is for, as well as test URLs where your change can be observed (before and after):
Fix #74
Test URLs:

Before: https://main--icicidirect--aemsites.hlx.live/research/equity

After: https://stickyfooter--icicidirect--aemsites.hlx.live/draft/vivesing/stickyfooter

[ ] New Blocks introduced in this PR
If yes, please provide details below

Block Name
Documentation

Sticky Footer
https://stickyfooter--icicidirect--aemsites.hlx.live/sidekick/blocks/stickyfooter

spacing in mob view

Can we do it without block creation as this doesn't need any specific layout ?
Just open and account link in the section and style say sticky-bottom
and for open trading account link and  sticky-right another section

Can we do it without block creation as this doesn't need any specific layout ?
Just open and account link in the section and style say sticky-bottom
and for open trading account link and sticky-right another section

IIUC you recommend to include this as part of main section and add css classes in the style.css?
I perfer creating a block because there are other styling that is specific to this section along with observers that are fired when scrolled after the block placement in the doc

@vivesing  small mobile view (320) might still need some spacing styling

A quick question, I tried to input a phone number and clicked the button on original site. Looks like it send a POST request with the number and did some validations. Shall we implement the same?

@vivesing small mobile view (320) might still need some spacing styling

@anagarwa it should be fixed now, there were some missing changes from the main

spacing in mob view

@ravverma thanks for pointing it out, fixed it.

A quick question, I tried to input a phone number and clicked the button on original site. Looks like it send a POST request with the number and did some validations. Shall we implement the same?

@JiangLong2019 Yes, I have skipped it intentionally and have opened a question for the customer to recommend the business rules and account creation flow once we click on the submit button. I have added the mobile number validation from their website and the rest can be added once they respond to that.

Can we do it without block creation as this doesn't need any specific layout ?
Just open and account link in the section and style say sticky-bottom
and for open trading account link and sticky-right another section

IIUC you recommend to include this as part of main section and add css classes in the style.css? I perfer creating a block because there are other styling that is specific to this section along with observers that are fired when scrolled after the block placement in the doc

👍

Yes, I have skipped it intentionally and have opened a question for the customer to recommend the business rules and account creation flow once we click on the submit button. I have added the mobile number validation from their website and the rest can be added once they respond to that.

Great! I think maybe the Sign Up block can reuse the logic @sneharora1


Build jolt.circom via Cargo

Stores the compiled artifacts to ./target/<build type>/jolt-core-<post-fix>/out/circom/...
Stores this path to CIRCUIT_DIR
snark.rs reads CIRCUIT_DIR

self-LGTM.


Can't install requirements

I added your requirements to ~/.julia/v0.4/REQUIRE and did a Pkg.resolve() which is probably the wrong way to do it but it seemed to do the right thing - however it fails for me with:
...
INFO: Building HttpParser
ERROR: Build process failed.
 in build! at ./pkg/entry.jl:701

Sounds like an upstream problem?
Afterwards I can't run julia properly, just for completeness:
sobot@solar:~/StackOverflowGitterNotifier.jl$ julia src/stackoverflow2gitter.jl  SO http://www.stackoverflow.com julia-lang https://webhooks.gitter.im/e/...
ERROR: LoadError: write: broken pipe (EPIPE)
while loading /home/sobot/.julia/v0.4/Requests/src/Requests.jl, in expression starting on line 17
ERROR: LoadError: Failed to precompile Requests to /home/sobot/.julia/lib/v0.4/Requests.ji
 in compilecache at ./loading.jl:400
while loading /home/sobot/StackOverflowGitterNotifier.jl/src/stackoverflow2gitter.jl, in expression starting on line 1

@oxinabox let me know if I can provide further info for debugging

Hi sils, I've just got back from holidays, sorry for the short delay in responding.
The normal way to install packages (such as the requirements for this),
is to run from the julia REPL: Pkg.add(PKG_NAME) for registered packages, and Pkg.clone(PKG_GITHUB_URL) for unregistered packages like this one.
rather than editing the ~/.julia/v0.4/REQUIRE and running "Pkg.resolve"
which I am not exactly sure what that would do -- but I guess it should work,
though I am not sure if it would trigger
So this repo, can be installed with all its dependencies by running
from the julia repo: Pkg.clone("https://github.com/oxinabox/StackOverflowGitterNotifier.jl")
which should install it to `~/.julia/v0.4/StackOverflowGitterNotifier.jl'
I have added some install instructions to the readme.
You can just rm -rf ~/.julia/v0.4 and then continue from step 2 of the instructions

julia> Pkg.clone("https://github.com/oxinabox/StackOverflowGitterNotifier.jl")
INFO: Initializing package repository /root/.julia/v0.4
 ...
INFO: Installing URIParser v0.1.6
ERROR: write: broken pipe (EPIPE)
 in uv_write at ./stream.jl:962

hm, I hope I'll find more time to investigate later.

Weird,
I suggest switching to julia 0.5, which is the current stable.
Seems like this is a julia issue, rather than an issue with just this package.
I think it might be an issue which is fixed in 0.5.
So if switching to julia 0.5 is an option, that would be my first place too look.
To help with further debugging could you run at the REPL:
versioninfo()
and paste the results?

julia> versioninfo()
Julia Version 0.4.5
Commit 2ac304d (2016-03-18 00:58 UTC)
Platform Info:
  System: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2650L v3 @ 1.80GHz
  WORD_SIZE: 64
  BLAS: libopenblas (NO_LAPACKE DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.8

I'm going to close this because we have (out of stream) worked around what ever your issues may be.
And because I think it is either an unstream problem or perhaps a hosting problem.
Open an issue later if it reoccurs.


Move code in util package to _internal

There was just a single function cancel_and_await that was only for internal use.

The idea was that this was public as a way to provide SDK users with useful tools, I really don't see why we have to keep this internal. For example await_and_cancel() could be useful for the channels repo. Of course we can't use it there because it would be a circular dependency, but I thought we might want to extract the util package and make it a separate library like we did with channels.
For example, I'm looking into https://github.com/frequenz-floss/frequenz-sdk-python/discussions/197, and that's another clear candidate for this, and can be helpful for channels too, as we also have this kind of objects there (Merge, MergeNamed and Select).
If there is no big opposition, I will move the package back to being public, and add the background service class in util.asyncio too, I think it fits there.


cant start resouirce

i have the problem that i have it in the resource folder and also in the cfg but it doesnt start also it doenst find the script in tx admin if i tyope start the script i get this failure
[      psynapse_chaos] start MugShotBase64
[ citizen-server-impl] Couldn't find resource MugShotBase64.


feat: Refactor and update fixtures

This patch changes how fixtures are stored. Previously, a fixture's folder identified its domain and its filename identified when it was fetched. This has been changed so that the filename indicates the domain and the modified time of the file indicates how recently it was fetched. A fixture's filename can optionally include a modifier to distinguish between two different page types on the same domain, for example.
Also included here are changes to the update-fixture script, both to accomodate the new filename scheme as well as to actually update all fixtures. The functionality for running automatically and opening PRs has been removed but will likely be reintroduced.
Finally, all fixtures have been updated.

Going to let this PR simmer a while while we get all the currently-open changes to extractors merged, to avoid conflicts. Then I'll update this to suit.

Rebased to main and fixed merge conflicts.

Closing because all changes will be captured in three separate PRs: #712, #713, and an upcoming one from this branch.


version 4.1.1. year drop down header arrow not showing up

Describe the bug
() => {
  const [startDate, setStartDate] = useState(new Date());
  return (
    <DatePicker
      selected={startDate}
      onChange={(date) => setStartDate(date)}
      showYearDropdown
      dateFormatCalendar="MMMM"
      yearDropdownItemNumber={15}
      scrollableYearDropdown
    />
  );
};

showYearDropdown = true and select year. Arrows not showing up in menu header
To Reproduce

Go to react-datepicker year drop down
Click on calendar of "year drop down" example and select year. (see attached screen short for more details)
See error

Expected behavior

up arrow in header and down arrow in footer must be there in year drop down menu.
Screenshots

Desktop (please complete the following information):

OS: [Ubuntu 16.04 LTS]
Browser [chrome Version 91.0.4472.77]
Version [4.1.1]

Additional context
It's version specific issue... In version 3.* it's working fine.

Styles for workaround:
.react-datepicker__navigation--years {
  background: none;
  line-height: 1.7rem;
  text-align: center;
  cursor: pointer;
  padding: 0;
  border: 0.45rem solid transparent;
  z-index: 1;
  height: 10px;
  width: 10px;
  text-indent: -999em;
  overflow: hidden;
  position: relative;
  display: block;
  margin-left: auto;
  margin-right: auto;
}

.react-datepicker__navigation--years.react-datepicker__navigation--years-upcoming {
  top: -4px;
  border-bottom-color: #ccc;
}

.react-datepicker__navigation--years.react-datepicker__navigation--years-previous {
  top: 4px;
  border-top-color: #ccc;
}

Hi, here are some styles for a quick workaround :
.react-datepicker__navigation--years::before {
  border-color: #666;
  border-style: solid;
  border-width: 3px 3px 0 0;
  content: '';
  display: block;
  height: 9px;
  left: 11px;
  position: absolute;
  width: 9px;
}

.react-datepicker__navigation--years-upcoming::before {
  top: 17px;
  transform: rotate(315deg);
}

.react-datepicker__navigation--years-previous::before {
  top: 6px;
  transform: rotate(135deg);
}

(tested on v4.8.0)


[BEAM-2596] Split Java and Python precommit jobs

Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Make sure there is a JIRA issue filed for the change (usually before you start working on it).  Trivial changes like typos do not require a JIRA issue.  Your pull request should address just this issue, without pulling in other changes.
[x] Each commit in the pull request should have a meaningful subject line and body.
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Run mvn clean verify to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
[x] If this contribution is large, please file an Apache Individual Contributor License Agreement.

This is to mitigate the issue of timeouts currently happening, until we can completely move to the pipeline build.

R: anyone, since the build remains perma-red/gray
Then also R: @jasonkuster to devise the right next steps

I think we should run the seed job from this PR first to make sure it succeeds, so that we can always run it from master if it does not.

LGTM

Run Seed Job

retest this please

run seed job

retest this please

Coverage increased (+0.05%) to 69.579% when pulling 4ae324fbb92f6bead895718ec4584f01a8a4ff0f on kennknowles:separate-precommits into 9379ca289a00cae169075728e6230a1d4a317659 on apache:master.

run seed job

run seed job

retest this please

OK, I can confirm that the tests are launched as expected. Now either of them may end up red but that is a different breakage. Since the seed job has already adjusted Jenkins, I will merge this so the status in the repo reflects what is running.

Coverage increased (+0.04%) to 69.576% when pulling 25790a4fdbe59ac5b1d5e19cce8ec3620be08183 on kennknowles:separate-precommits into 9379ca289a00cae169075728e6230a1d4a317659 on apache:master.

Coverage increased (+0.05%) to 69.579% when pulling 25790a4fdbe59ac5b1d5e19cce8ec3620be08183 on kennknowles:separate-precommits into 9379ca289a00cae169075728e6230a1d4a317659 on apache:master.


Front page ui

This is a first draft based on the formerly discussed design.
Feel free to give feedback!
Please keep in mind that due to the bombs things in Mauritz tomorrow I am not home, so if you need some very quick fixes best if you do so, otherwise will get to it later.
Also @ubergesundheit  I need your help with implementing the semi date nav to left and right... I need some new components for it and to pull info from some date object. Let me know when you have time to look at it together.

Looks nice! I did some opinionated changes (See this commit)

Changed how the current date ends up in the layout
Refactored extraction of the date into a hook to be reused
Made use of bootstrap variables
Extracted some colors into the theme

What I would change:

I don't like how the EventListItem is being layed out. I would change this to either use bootstrap grid or css grid.

I'll merge this branch into the app-skeleton branch now and will add the buttons so you can continue :)


docs: netlify

What this PR does / why we need it

Jira ID
[XX-XX]

Notes for your reviewers

Coverage decreased (-0.03%) to 41.356% when pulling 7a8ac74596995aa76176e88f13c935981dd1fa57 on jaredallard/feat/netlify into 0e946e8b65f8a1047ffa1e143f56758af9cd44de on main.


Upgrade Substrate

This follow the process that was used upgrading Polkadot 0.9.25 to 0.9.30. It also links to other repos which contain other useful information.
It also references diener, a crate which makes changing Substrate and Polkadot version easier.
Comments welcome.

I don't think this atm correctly describes the order in which dependencies are updated. For example:

create an integration branch that each of the following steps will do a PR into
upgrade all references of the substrate repo at once. This is required so cargo will actually attempt to build
upgrade each pallet one at a time. This should include tests and ensuring the runtime-benchmarks feature builds. Each pallet should be a separate PR to the integration branch to aid review
Once all pallets are upgraded move onto the dscp-runtime. Again ensure tests and runtime-benchmarks build and pass
Upgrade the dscp-node again ensuring tests and runtime-benchmarks builds and work
Finally perform a PR from the integration branch into main


Pages: add a cleartext cache for password and a rate limiter for bcrypt

As suggested by @maxtaco a month ago, this adds a cleartext cache for passwords for the HTTP Basic Auth. Also added a per-site rate limiter for bcrypt.

@jzila Are you OK with merging this?

The more I think about it, we probably shouldn't have this auth stuff at all.

@jzila Agreed. Perhaps we should remove the auth part from the docs as well.


Add context menu in project explorer

could be good, if qbs project explorer would handle context menu, or handle key bindings (for example F2 - rename), just like standard Explorer of vscode

Agreed, but seems, it is hard to do (I can't find any examples for this)... AFAIK, this feature is not implemented even in the CMake Tools extension.

ok, I'm just comparing with qt creator. I thought to switch to vs code, but for C++, Qt, Qml, Qml, qt creator is the best development environment, there are a lot of features in vscode, including in the form of plugins


[WIP] Create tagEditor component and associated service

tagEditor to be used in modals for single and batch editing of tags.
The component will need to:

[ ] Have a binding for common tags of resources that need tags edited
[ ] Manage a tag assignments collection
[ ] Cache/hold available tag categories and options
[ ] Provide a callback after dismissing tags from UI
[ ] Two-way bind tag assignments for both single and multiple value
categories

Add taggingService for higher-level interactions with CollectionsApi.
The service will need to:

[ ] Get all tag categories and associated value options
[ ] Post tag assignments (involves assigning and unassigning tags)

@miq-bot add_label wip, euwe/no

Hi @jjlangholtz ,
I'm not sure if we're doing the same component.  There is a tagging directive at: client/app/components/tagging
It is used to tag blueprints and service items (on the canvas):

It handles adding and removing tags, as well as handles single-value tags.

Closing in favor of #402


feat: add jwtSecret to app.conf & edit application page

Display jwtSecret in "Applications - Edit Application page", to make sure developer could use jwtSecret just as clientSecret when init config before using sdk.
Move jwtSecret string to app.conf to make sure developer could easily find it
Add jwtSecret's length to fit to JJWT

@sh1luo @Kininaru plz review


Can't skip music

Please tick all applicable boxes.

[ x] I am using Python 3.5.3 or higher (run python --version on the command line)
[x] I have followed the official guides to install the bot for my system
[x] I have updated my dependencies to the latest version using the appropriate update script

Which version are you using?

[ ] The latest master version (release-260419)
[x] The latest review version

What type of issue are you creating?

[x] Bug
[ ] Feature request
[ ] Question

Description of issue
When a member or me use the skip command, an error occurs and the command is not taken into account.
Steps to reproduce

use the skip command

Log file
Traceback (most recent call last):
File "/home/valentin/MusicBot/musicbot/bot.py", line 2823, in on_message
response = await handler(**handler_kwargs)
File "/home/valentin/MusicBot/musicbot/bot.py", line 2111, in cmd_skip
num_skips = player.skip_state.add_skipper(author.id, message)
File "/home/valentin/MusicBot/musicbot/constructs.py", line 36, in add_skipper
self.skip_msgs.add(msg)
TypeError: unhashable type: 'Message'

windows: py -m pip install -U discord.py[voice]
Linux: python -m pip install -U discord.py[voice]

Hi @Jcodeerd
I use this command before each bot start, unfortunately it doesn't change anything

Hello,
This is the official problem: https://github.com/Rapptz/discord.py/issues/5866
The code that requires that is at https://github.com/Just-Some-Bots/MusicBot/blob/master/musicbot/constructs.py#L36
self.skip_msgs.add(msg)

Where msg is a discord.Message and self.skip_msgs is a set.
You can workaround that by modifying it to:
self.skip_msgs.add(msg.jump_url)

After this, the skip command starts working again ;)

See #2143
Per discord.py staff, massages were not intended to be hashable, therefore we should not try to hash them.

This error is resolved in discord.py 1.5.1


Error when joining disabled server

Hi, when I try to join to a disabled default server it shows me this error ( https://scr.hu/mBR3M1 )instead of normal message which was set in the messages.properties.

Can someone fix this?

This looks like a client issue with forge or similar.
Please report client version, error (stack trace) in the clients log files (alternatively put launcher to stay open to see the clients console), waterfall version, backend server version.

It's impossible, BungeeCord doesn't habe this error.

Still the error includes net.minecraft.client - the error is just client-side.

Why is it happening?

Still you should provide the information I mentioned as otherwise you can close this because we can't find out "Why is it happening?" otherwise.

I think it is happening on every 1.7 version,
"This server is running Waterfall version git:Waterfall-Bootstrap:1.9-SNAPSHOT:ba75eaa:192 by the Waterfall team"

Please report client version, error (stack trace) in the clients log files (alternatively put launcher to stay open to see the clients console), waterfall version, backend server version.

What is the client log?

There is nothing.

There is always something in the client log.

Why you can't check it?

... need the client log to check it :/

How to check logs?

Google google can help my friend. Check .minecraft folder.

There's nothing in logs. CHECK IT

Here is the content of my latest client log, available in /home/nicholas/.minecraft/logs/latest.log
[16:44:13] [Client thread/INFO]: Setting user: Techcable
[16:44:21] [Client thread/INFO]: LWJGL Version: 2.9.4
[16:44:24] [Client thread/INFO]: Reloading ResourceManager: Default, FMLFileResourcePack:Forge Mod Loader, FMLFileResourcePack:Minecraft Forge
[16:44:25] [Sound Library Loader/INFO]: Starting up SoundSystem...
[16:44:25] [Thread-8/INFO]: Initializing LWJGL OpenAL
[16:44:25] [Thread-8/INFO]: (The LWJGL binding of OpenAL.  For more information, see http://www.lwjgl.org)
[16:44:25] [Thread-8/INFO]: OpenAL initialized.
[16:44:26] [Sound Library Loader/INFO]: Sound engine started
[16:44:28] [Client thread/INFO]: Created: 16x16 textures-atlas
[16:44:30] [Client thread/INFO]: Reloading ResourceManager: Default, FMLFileResourcePack:Forge Mod Loader, FMLFileResourcePack:Minecraft Forge

@Techcable Error on 1.7.10 version: https://scr.hu/7zKyjY
Error on 1.7.6 (or 1.7.2) version: https://scr.hu/ZR1Pq2
In the latest.log file there is just [Client thread/INFO]: Connecting to [ip], [port]

@CzlowiekKuktas
I need the entire log file pastebinned.
Thanks for cooperating though :)

@Techcable As it shows just the connecting information. You can check it by yourself.

@CzlowiekKuktas Multiple instances of the same error usually help when debugging, as it can narrow down the exact cause of the issue.

Plus the stacktrace....... and prior information messages.


Fix order of words in y-axis of dispersion_plot

Input words are reversed to create 'y' ticks but the internal 'words' list is not updated accordingly.
This has been fixed by getting 'words' back from 'word2y', which is sensitive to 'ignore_case'.
If 'words' should not be sensitive to 'ignore_case', then the following should be used instead:
# do not modify input 'words' list (as used to be the case with .reverse())
words = words[::-1]

The two failures are in nltk downloading datasets, maybe some network connection issue?

Looks good! Thanks for this. And I must say that I don't quite know why those two test runs are failing, but I agree that they must be unrelated.


Adding support for analytical storage schema types

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Please ensure to add changelog with this PR by answering the following questions.

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When you are targeting to deploy new service/feature to public regions? Please provide date, or month to public if date is not available yet.
When you expect to publish swagger? Please provide date, or month to public if date is not available yet.
If it's an update to existing version,  please select SDKs of specific language and CLIs that require refresh after swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[ ] PowerShell
[ ] CLI
[ ] Terraform
[ ] No, no need to refresh for updates in this PR

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[x] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
Adding a new API version
[x] Ensure to copy the existing version into new directory structure for first commit (including refactoring) and then push new changes including version updates in separate commits. This is required to review the changes efficiently.
Adding a new service

[ ] Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from Breaking Change Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable or public preview version with Breaking Change Validation errors
[ ] Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

/azp run

/azp run

Hi Ray,
Looks like some of the PR's build is queued but not making progress.  Can this be merged?
Thanks,
Tom

/azp run

@ArcturusZhang , can you have a look at the potential breaking change for GO SDK?

Hi Ray,
Looks like some of the PR's build is queued but not making progress. Can this be merged?
Thanks,
Tom

Checks result look good now.

Hi Ray,
Looks like some of the PR's build is queued but not making progress. Can this be merged?
Thanks,
Tom

Checks result look good now.

Can it be merged now?

It needs @ArcturusZhang to check the potential breaking change for GO SDK.

The breaking changes are accumulated by previous PRs


Conversion back to fast5 is broken in some cases

Using slow5tools v0.3.0
The input file had an unexpected field (end_reason) which triggered a warning on conversion to blow5.
Trying to recreate the fast5 from the blow5 file resulted in:
[list_all_items] Looking for '*low5' files in test_blow5/ [s2f_main] 1 files found - took 0.001s [s2f_iop] 1 proceses will be used [s2f_child_worker] Converting test_blow5//FAL37440_pass_c4fa58d7_179.blow5 to fast5 [slow5_get_aux_enum_labels::ERROR] No enum auxiliary type exists. At src/slow5.c:1181 [slow5_get_aux_enum_labels::ERROR] Exiting on error. At src/slow5.c:1181
If you can provide an email I can send an example file.
Data were generated in May 2020 using GridION - 19.12.6 - Guppy Version was 3.2.10

@mattloose I sent my email to you on Twitter chat.
This end_reason end reason has been an absolute pain. It first appeared as a string data type first, then somewhere it changed to an ENUM. In between how many variants there would have been - even nanopore may not know probably :D
Fun fact: Did you know that the file version in FAST5 could be either a string, or an INT or even a floating-point number? Soon it could be a complex number :D

Looks like this file contains that data as an enum.

This enum type was handled. Could you double-check the slow5tools version you are using as slow5tools --version?
I tried the file you sent on my end, and it seems to convert back and forth:
hasindu@hasindu-xps:/mnt/c/Users/hasindu/Desktop/test_blow5$ ~/slow5tools-v0.3.0/slow5tools f2s orig/ -d blow5
[list_all_items] Looking for '*.fast5' files in orig/
[f2s_main] 1 fast5 files found - took 0.001s
[f2s_iop] 1 proceses will be used.
[search_and_warn::WARNING] slow5tools-v0.3.0: The attribute 'pore_type' is empty and will be stored in the SLOW5 header.
[search_and_warn::WARNING] slow5tools-v0.3.0: The attribute 'pore_type' is empty and will be stored in the SLOW5 header. This warning is suppressed now onwards.
[f2s_child_worker::INFO] Summary - total fast5: 1, bad fast5: 0

[f2s_main] Converting 1 fast5 files took 3.024s

[main] cmd: /home/hasindu/slow5tools-v0.3.0/slow5tools f2s orig/ -d blow5
[main] real time = 3.027 sec | CPU time = 3.047 sec | peak RAM = 0.041 GB
hasindu@hasindu-xps:/mnt/c/Users/hasindu/Desktop/test_blow5$ ~/slow5tools-v0.3.0/slow5tools s2f blow5 -d fast5_back
[list_all_items] Looking for '*low5' files in blow5
[s2f_main] 1 files found - took 0.001s
[s2f_iop] 1 proceses will be used
[s2f_child_worker] Converting blow5/FAL37440_pass_c4fa58d7_179.blow5 to fast5
[s2f_main] Converting 1 s/blow5 files took 3.419s

[main] cmd: /home/hasindu/slow5tools-v0.3.0/slow5tools s2f blow5 -d fast5_back
[main] real time = 3.424 sec | CPU time = 3.438 sec | peak RAM = 0.033 GB

hasindu@hasindu-xps:/mnt/c/Users/hasindu/Desktop/test_blow5$ ls * -lh
blow5:
total 34M
-rwxrwxrwx 1 hasindu hasindu 34M Feb  3 00:13 FAL37440_pass_c4fa58d7_179.blow5

fast5_back:
total 75M
-rwxrwxrwx 1 hasindu hasindu 75M Feb  3 00:13 FAL37440_pass_c4fa58d7_179.fast5

orig:
total 101M
-rwxrwxrwx 1 hasindu hasindu 101M Feb  3 00:11 FAL37440_pass_c4fa58d7_179.fast5

Yes - I've sent you more files...
What I have determined is that the file that breaks has been through ONTs compress_fast5 feature. In that file the end reason is an 8-bit signed int. Not an enum!

But still we will soon update slow5tools to be able to convert such broken FAST5 back and forth. Because compress_fast5 removed the enum labels and just stored as an uint8_t, the end reason field in such files is basically lost. But we will write an update to slow5tools to handle this situation. There are a few ways that come into my mind:

Drop that damaged field when converting to SLOW5
Assuming the int8_t number is the enum index and assuming ONT has not changed the enum order, we can try to reconstruct the enum label when converting to SLOW5. But this recovery could be errenoeous in case our assumption is wrong
Store the damaged field as  it is as a uint8_t and when converting back using s2f propagate the damaged field as it is.

@mattloose Which option do you prefer?
1 is the easiest. I do not like 3 much as I would like to keep the field data types consistent in SLOW5 unlike ONT.
I personally prefer 2.

I would expect slow5 to contain whatever was in the fast5 that was passed to it. I.e I don't think slow5 conversion should be making decisions about what the data are in the file - it should simply be storing what was in the original file.
So I'd vote for 3. In this case the error was caused by the upstream conversion. I'd expect the input error to be carried forward. Especially as if you do 2 and get it wrong it could be really hard to track down as a user as the field would look correct when in fact it was not.

Option 4) do a pull request on the ont conversion tool so it doesn't cause this problem.

@mattloose
The latest slow5tools dev branch has this implemented (option 3 we discussed above)- if the end_reason field is corrupted, f2s will save the corrupted attribute as it is with a warning. Also, s2f works on such created BLOW5 files, propagating the corrupted field as it is. Could you please give a try? To compile from dev branch:
git clone -b dev --recursive https://github.com/hasindu2008/slow5tools
cd slow5tools
autoreconf	# autoreconf --install for macos
./configure
make

@Psy-Fer ONT conversion tool has been potentially fixed.

Hi @hasindu2008 , we only have fast5s with falsely converted "end_reason", but the dev branch seems to dropped support for these "corrupted" files. We carelessly trusted the ont-fast5-api and compressed these files in place, so no original fast5 files can be accessed now. Is there any possibility to rescue these files? Thanks!
I have compiled the dev branch as instructed.
$ slow5tools --version
slow5tools 0.8.0-dirty

But I am still getting the following errors, and only empty slow5 files are generated.
[read_fast5::ERROR] Bad fast5: Could not iterate over the read groups in the fast5 file ./fast5_fail/FAP34515_fail_60478b74_47.fast5.
[f2s_child_worker::ERROR] Bad fast5: Could not read contents of the fast5 file './fast5_fail/FAP34515_fail_60478b74_47.fast5'.
[fast5_attribute_itr::ERROR] Attribute Raw/end_reason in ./fast5_fail/FAP34515_fail_60478b74_17.fast5 is corrupted (datatype H5T_STD_U8LE instead of expected H5T_ENUM). This is a known issue in ont_fast5_api's compress_fast5 (see https://github.com/hasindu2008/slow5tools/issues/59 and https://github.com/nanoporetech/ont_fast5_api/issues/70).
 Please get your FAST5 files fixed before SLOW5 coversion, by bugging ONT through GitHub issues.

$ ll
total 0
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_0.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_10.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_15.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_17.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_27.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_32.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_36.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_37.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_38.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_41.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_44.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_46.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_47.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_4.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_50.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_51.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_57.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_62.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_63.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_7.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_8.blow5
-rw-rw-r-- 1 zhangjy zhangjy 0 Nov 14  2022 FAP34515_fail_60478b74_9.blow5

@Kevinzjy
We ended up erroring out rather than propagating the corrupted attribute as we thought the user must be made aware of this. That way, the users can reach out to ONT [see https://github.com/nanoporetech/ont_fast5_api/issues/70]  and see if there is a way to get those attributes recovered (I suggest opening an issue there and getting them to write a programme that recovers your data).
The reason why I dropped that feature to dump the corrupted attribute to SLOW5 was that such a datatype mismatch (inconsistencies) can cause future headaches, for instance, merging. If ONT does not provide a positive solution for you, we will implement an option to drop such corrupted attributes during fast5 to slow5 conversion, because if ONT is not going to provide a solution, there is no point in having that corrupted attribute anyway.  To add further, that end_reason attribute as pointed out by @mattloose is not to be trusted as MInKNOW at some point was classifying end_reason incorrectly.

@Kevinzjy We ended up erroring out rather than propagating the corrupted attribute as we thought the user must be made aware of this. That way, the users can reach out to ONT [see https://github.com/nanoporetech/ont_fast5_api/issues/70] and see if there is a way to get those attributes recovered (I suggest opening an issue there and getting them to write a programme that recovers your data).
The reason why I dropped that feature to dump the corrupted attribute to SLOW5 was that such a datatype mismatch (inconsistencies) can cause future headaches, for instance, merging. If ONT does not provide a positive solution for you, we will implement an option to drop such corrupted attributes during fast5 to slow5 conversion, because if ONT is not going to provide a solution, there is no point in having that corrupted attribute anyway. To add further, that end_reason attribute as pointed out by @mattloose is not to be trusted as MInKNOW at some point was classifying end_reason incorrectly.

Thanks for the reply @hasindu2008 . I totally agree that ONT should do something to fix this issue.  I'll open a new issue there to see if they can provide a resolution.


Add additional ng-package.json for nested sub-repos

Is your feature request related to a problem? Please describe.
When module is compiled from a sub-repo, like it is in the senzing app, ng-packagr incorrectly generates the package with a nested namespace. ie, if I have a project with the following structure:

myApp
 |________dist
 |________ rest-api-client-ng (subrepo)
 |________ sdk-components-ng (subrepo)
 |
 |________ local project module
 |________ src
 |           |___app
 |
 |_ package.json

when building the sdk-components-ng module it will generate the module with a name of "myApp/sdk-components-ng/projects/sdk-components-ng", which is invalid.
Describe the solution you'd like
When I compile sdk-components-ng, the name should be "@senzing/sdk-components-ng" just like it is when compiling the module from a standalone repo.
Describe alternatives you've considered
Personally I feel like this is an issue with ng-packagr(see https://github.com/ng-packagr/ng-packagr/blob/master/docs/secondary-entrypoints.md), and I will open an issue with that project as well, but we need a workaround on our schedule for now.
Additional context
The easiest way to fix this is to add an additional ng-package.json, with the output directories one level higher that can be referenced from a parent repo build script.

Pull Requests:

https://github.com/Senzing/sdk-components-ng/pull/37


Feature needed

can page display is always maintained on one screen?
now we can set row height with pix,in that case ,the scroll will show anyway,
so if i can set a row display prop on grid:
rowtype:"scroll"|"flex"
and set on item
item.flexable:true|false
when the screen height changes ,the page can alway on one screen,

I also need this.


couldn't use Markdown todo list feature

when I use hexo, and the next theme to write my blog , but the Markdown- [ ] TODO  feature dosen't work.
expect like this:

[ ] tode

but the result is:
 [ ] todo
Could you tell me what causes it, Thank you!

The Markdown parser that Hexo used does not support this feature.

Thank you! I use HTML tags instead of the Markdown to sovle this problem.


Added information for AADDS

@AlexFilipin : Thanks for your contribution! The author, @billmath, has been notified to review your proposed change.

Also see https://github.com/MicrosoftDocs/azure-docs/pull/31965


meta img not appear on link pasting to social media

feature img not appearing on social media link pasting. Could you assign me this issue?

yes for sure


Fixed some typos in README.

Fixed spelling errors.

@udit-001 Thanks for the PR. Hope to see more contributions from you


RuntimeError: failed to find a rendering device

when i ran python scripts/universal_training_script.py --cfg configs/parameters/peg_insertion.yaml, there are some errors.
The first:
CUDA Toolkit: 11.8, Driver: 12.0
Devices:
"cpu"    | x86_64
"cuda:0" | NVIDIA RTX A6000 (sm_86)
"cuda:1" | NVIDIA RTX A6000 (sm_86)
"cuda:2" | NVIDIA RTX A6000 (sm_86)
"cuda:3" | NVIDIA RTX A6000 (sm_86)
Kernel cache: /home/ziwu/.cache/warp/1.0.0-beta.2
Process ForkServerProcess-1:
Traceback (most recent call last):
File "/home/ziwu/anaconda3/envs/mani_vitac/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/home/ziwu/anaconda3/envs/mani_vitac/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/ziwu/anaconda3/envs/mani_vitac/lib/python3.10/site-packages/stable_baselines3/common/vec_env/subproc_vec_env.py", line 29, in _worker
env = _patch_env(env_fn_wrapper.var())
File "/home/ziwu/ManiSkill-ViTac2024/scripts/universal_training_script.py", line 39, in _init
env = gym.make(env_name, **env_args)
File "/home/ziwu/anaconda3/envs/mani_vitac/lib/python3.10/site-packages/gymnasium/envs/registration.py", line 802, in make
env = env_creator(**env_spec_kwargs)
File "/home/ziwu/ManiSkill-ViTac2024/scripts/../envs/peg_insertion.py", line 750, in init
super(ContinuousInsertionSimGymRandomizedPointFLowEnv, self).init(**kwargs)
File "/home/ziwu/ManiSkill-ViTac2024/scripts/../envs/peg_insertion.py", line 132, in init
self.scene = sapien.Scene()
File "/home/ziwu/anaconda3/envs/mani_vitac/lib/python3.10/site-packages/sapien/wrapper/scene.py", line 24, in init
[sapien.physx.PhysxCpuSystem(), sapien.render.RenderSystem()]
RuntimeError: failed to find a rendering device
About this error, i wonder whether the execution of these scripts needs the support of a screen. or special display devices?
The other:
Traceback (most recent call last):
File "/home/ziwu/ManiSkill-ViTac2024/scripts/universal_training_script.py", line 91, in
env = SubprocVecEnv(
File "/home/ziwu/anaconda3/envs/mani_vitac/lib/python3.10/site-packages/stable_baselines3/common/vec_env/subproc_vec_env.py", line 120, in init
observation_space, action_space = self.remotes[0].recv()
File "/home/ziwu/anaconda3/envs/mani_vitac/lib/python3.10/multiprocessing/connection.py", line 255, in recv
buf = self._recv_bytes()
File "/home/ziwu/anaconda3/envs/mani_vitac/lib/python3.10/multiprocessing/connection.py", line 419, in _recv_bytes
buf = self._recv(4)
File "/home/ziwu/anaconda3/envs/mani_vitac/lib/python3.10/multiprocessing/connection.py", line 384, in _recv
chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer
It looks like i need to connect some hardware to execute that script?

Thank you for your question. We have added --no_render option for training on servers in the lastest commit.

Hi, I also met a similar issue, and I have set no_render to true. The output is as following:
Warp 1.0.0-beta.2 initialized:
CUDA Toolkit: 11.8, Driver: 12.1
Devices:
"cpu"    | x86_64
"cuda:0" | NVIDIA GeForce RTX 3050 Ti Laptop GPU (sm_86)
Kernel cache: /home/ViTac2024/sapienipc-exp/build
other arg: cfg is set to configs/parameters/peg_insertion.yaml
other arg: no_render is set to True
Warp 1.0.0-beta.2 initialized:
CUDA Toolkit: 11.8, Driver: 12.1
Devices:
"cpu"    | x86_64
"cuda:0" | NVIDIA GeForce RTX 3050 Ti Laptop GPU (sm_86)
Kernel cache: /home/ViTac2024/sapienipc-exp/build
Warp 1.0.0-beta.2 initialized:
CUDA Toolkit: 11.8, Driver: 12.1
Devices:
"cpu"    | x86_64
"cuda:0" | NVIDIA GeForce RTX 3050 Ti Laptop GPU (sm_86)
Kernel cache: /home/ViTac2024/sapienipc-exp/build
Process ForkServerProcess-1:
Process ForkServerProcess-2:
Traceback (most recent call last):
File "/root/miniconda3/envs/mani_vitac/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/root/miniconda3/envs/mani_vitac/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/root/miniconda3/envs/mani_vitac/lib/python3.10/site-packages/stable_baselines3/common/vec_env/subproc_vec_env.py", line 29, in _worker
env = _patch_env(env_fn_wrapper.var())
File "/home/ViTac2024/ManiSkill-ViTac2024/scripts/universal_training_script.py", line 42, in _init
env = gym.make(env_name, device=wp_device, **env_args)
Traceback (most recent call last):
File "/root/miniconda3/envs/mani_vitac/lib/python3.10/site-packages/gymnasium/envs/registration.py", line 802, in make
env = env_creator(**env_spec_kwargs)
File "/home/ViTac2024/ManiSkill-ViTac2024/scripts/../envs/peg_insertion.py", line 764, in init
super(ContinuousInsertionSimGymRandomizedPointFLowEnv, self).init(**kwargs)
File "/home/ViTac2024/ManiSkill-ViTac2024/scripts/../envs/peg_insertion.py", line 136, in init
self.scene = sapien.Scene()
File "/root/miniconda3/envs/mani_vitac/lib/python3.10/site-packages/sapien/wrapper/scene.py", line 24, in init
[sapien.physx.PhysxCpuSystem(), sapien.render.RenderSystem()]
File "/root/miniconda3/envs/mani_vitac/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/root/miniconda3/envs/mani_vitac/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
RuntimeError: failed to find a rendering device
File "/root/miniconda3/envs/mani_vitac/lib/python3.10/site-packages/stable_baselines3/common/vec_env/subproc_vec_env.py", line 29, in _worker
env = _patch_env(env_fn_wrapper.var())
File "/home/ViTac2024/ManiSkill-ViTac2024/scripts/universal_training_script.py", line 42, in _init
env = gym.make(env_name, device=wp_device, **env_args)
File "/root/miniconda3/envs/mani_vitac/lib/python3.10/site-packages/gymnasium/envs/registration.py", line 802, in make
env = env_creator(**env_spec_kwargs)
File "/home/ViTac2024/ManiSkill-ViTac2024/scripts/../envs/peg_insertion.py", line 764, in init
super(ContinuousInsertionSimGymRandomizedPointFLowEnv, self).init(**kwargs)
File "/home/ViTac2024/ManiSkill-ViTac2024/scripts/../envs/peg_insertion.py", line 136, in init
self.scene = sapien.Scene()
File "/root/miniconda3/envs/mani_vitac/lib/python3.10/site-packages/sapien/wrapper/scene.py", line 24, in init
[sapien.physx.PhysxCpuSystem(), sapien.render.RenderSystem()]
RuntimeError: failed to find a rendering device

we just fixed the error. Please try again


obtain list of (modules + build files) from a list of module files - gradle

we would like the user to supply a list of build files, we will get list of modules, and run on each module separately.

Implementation step for https://github.com/konveyor/tackle-test-generator-cli/issues/122

Implemented and merged via PR: https://github.com/konveyor/tackle-test-generator-cli/pull/178


Rename loggers/base.py to loggers/logger.py and changed LightningLoggerBase to Logger

What does this PR do?

Fixes #11971

Changed loggers.logger.LightningLoggerBase to loggers.logger.Logger
Then renamed loggers/base.py to loggers/logger.py.
Created a new loggers/base.py, which contains LightningLoggerBase with a deprecation warning and subclasses from Logger to support backward compatibility.
Changed all internal imports to support the refactoring.
Changed loggers/test_base.py to loggers/test_logger.py
Created a new loggers/test_base.py to test the deprecation warning from loggers.base.LightningLoggerBase

I still haven't updated the documentation, I first wanted to make sure that I am on the right track and if there are any necessary changes needed.
Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

[x] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[ ] Did you make sure to update the documentation with your changes? (if necessary)
[x] Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
[ ] Did you list all the breaking changes introduced by this pull request?
[ ] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[ ] Is this pull request ready for review? (if not, please submit in draft mode)
[ ] Check that all items from Before submitting are resolved
[ ] Make sure the title is self-explanatory and the description concisely explains the PR
[ ] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

[x] Had fun? 😄

@MohammedAlkhrashi Thanks so much for your PR! Mind fixing the merge conflicts and failing tests so we can get this merged?

@MohammedAlkhrashi Thanks so much for your PR! Mind fixing the merge conflicts and failing tests so we can get this merged?

Alright 👍 , I will try to fix the conflicts and the failing tests by next week if that's okay.

Awesome, thanks!

Hi everyone, I've fixed the merge conflicts, failed tests, and updated the docs accordingly.  However, I was wondering if we should also implement wrappers for other functionality in current loggers/base.py for backward compatibility? Such as DummyLogger, or LoggerCollection. Currently, only LightiningLoggerBase is backward compatible if imported directly from 'pl.loggers.base'.
Additionally, I am not sure about the depreciation test procedure exactly, should I move my deprecation tests to tests/deprecated_api?

Great work!

I was wondering if we should also implement wrappers for other functionality in current loggers/base.py for backward compatibility? Such as DummyLogger, or LoggerCollection.

Yes exactly, we should. The exception is LoggerCollection where you would need to just keep the existing deprecation message (because it gets removed in 1.8).

Additionally, I am not sure about the deprecation test procedure exactly, should I move my deprecation tests to tests/deprecated_api? If so, I am assuming it's moved to the 1.8 files, is that correct?

Yes, see my latest commit and the addition in test_remove_1-9.py. The same would be done for the other classes that had to move.

Yes exactly, we should. The exception is LoggerCollection where you would need to just keep the existing deprecation message (because it gets removed in 1.8).

Great, just to make sure, this would include creating wrappers for all classes and functions including zero_rank_experiment and merge_dicts, right?
As for the LoggerCollection, that means I would still create a wrapper for it as well, however, I shouldn't create a new deprecation test as one already exists, did I get this correctly?

Yes, see my latest commit and the addition in test_remove_1-9.py. The same would be done for the other classes that had to move.
Got it, thanks @awaelchli 👍

Yes, I think so. I see no other way :)