Serialisation Question

First up, I love the work you've done. I'm working on implementing it in my RPG project built using CRYENGINE. That leads to me having a few more constraints than some projects, which have led to me asking this question.
I am trying to serialise the entities and components out to disk. Sadly, XML is the only format I've got a decent chance of using due to the state of the libraries I am working with. Even then, it's a mess. The code to write an XML file is done, so I only need to read it in now, and that's 80% done.
I have some test output XML:
<entities>
 <entity>
  <components>
   <position typeGUID="d22ce389-15df-4ef3-b6cf-fd7490900765">
    <properties CryXmlVersion="2" X="20.799999" Y="20.799999"/>
   </position>
   <velocity typeGUID="812ddc29-d565-4294-8a1d-ce5af770151b">
    <properties CryXmlVersion="2" DX="0.80000001" DY="0.80000001"/>
   </velocity>
  </components>
 </entity>
 <entity>
  <components>
   <position typeGUID="d22ce389-15df-4ef3-b6cf-fd7490900765">
    <properties CryXmlVersion="2" X="15.6" Y="15.6"/>
   </position>
   <velocity typeGUID="812ddc29-d565-4294-8a1d-ce5af770151b">
    <properties CryXmlVersion="2" DX="0.60000002" DY="0.60000002"/>
   </velocity>
  </components>
 </entity>
</entities>

The idea is simple enough, read in the entities, make a new entity with EnTT, read each component - and use the GUID to figure out which class type I need to instantiate. That's where I am stuck.
Is there already a method to register types somewhere with a key and then have one created by passing in the key? Is this even possible? I don't mind having to register the components with something, as they already have to do that to work with CRYENGINE.
Is there another way to handle this? I did look at the snapshot code, and ran into a few issues as I tried to test it. My archive types didn't seem to have the right functions or something along those lines. I probably need to put more time into that.
I tried using Cereal as a quick and dirty way to get started with the testing but it doesn't compile within my project with the restrictions of CRYENGINE...which for some reason has decided to #define malloc and a few other crimes against good code. It relies on RapidJSON and RapidXML too, which are both looking a little abandoned - plus, I would be adding a third serialisation library to the code >.<
Ideally, I'd just be able to use a factory to create the components, one that is key based (templated). I'm just not sure if you already support that, or how it would work if not supported.

Hello!
You could try using EnTT's meta for deserialisation.  Here's an implementation for JSON. I think it should be more or less straight forward to implement the same for XML. One day I may try to make it more generic, but right now I don't have time for it.

First up, I love the work you've done.

Thank you, I'm glad you like it and feel free to give your feedback or suggestions and to join the gitter channel if you need help. :+1:

which have led to me asking this question. [...] just not sure if you already support that, or how it would work if not supported.

I don't know if the snapshot part fits well in this case. I've never used it with xml actually. You may make it work of course but I've not an example for you, I'm sorry.
As @Innokentiy-Alaytsev pointed out, an option is to use the meta system though.
You can register your types with an identifier, then read the name of the component from your xml, hash it (as in entt::hashed_string) and retrieve the meta type. Using it and a good amount of aliasing (as in entt::as_alias), you can create your components directly in the registry and modify their data members by means of an entt::meta_any, with no copies at all. Data members are identified by name as well, so again you can read the name of the data member you want to set, retrieve it and pass the value you read from the xml.
Does it make sense to you?

That makes a decent amount of sense. I'll have a crack at it in a day or two - see how it goes.

Ok. Feel free to leave the issue open, so you can ask more questions here if you need. Otherwise, join the gitter channel if you want faster answers, there are several people there who can help you.
I'll close this issue within a week if there are no updates or if you don't close it first. :wink:
Let us know how it goes. :+1:

Can you keep it open just a bit longer. I've had family around all week and not been able to get any work done on it yet. Thanks.

Sure, it's not a problem. I know what you mean. :wink:
Ping me if you need help or want a feedback. :+1:

I made a little progress. I've used hash strings to register a class into the meta. I have a little test code to construct an instance of the class based on the string e.g.
		auto postionType = entt::resolve("position"_hs);
		auto any = postionType.construct(20.0f, 30.0f);

I can cast that to the actual type to check it works, and it does. So good, so far.
Now, how do I take that entt::meta_any variable and assign it as a component to an entity? I have a registry created and an entity ready to take a value - I just haven't figured out what function to call to assign it to that registry.

Hello!
The way I do it is by adding a meta-function accepting a registry and an entity and assigning the actual component from the meta-any instance.
template < class TComponent >
void AssignComponentFromMetaAny (
    TComponent const& i_component_instance,
    ::entt::entity    i_entity,
    ::entt::registry* io_registry)
{
	ASCLEPIA_ECS_DEBUG_ASSERT (io_registry, assert::Level::kLogicCheck);
	ASCLEPIA_ECS_DEBUG_ASSERT (
	    io_registry->valid (i_entity), assert::Level::kLogicCheck);

	io_registry->assign< TComponent > (i_entity, i_component_instance);
}

::entt::meta< ObjectComponent > ()
    .type (HS ("ObjectComponent"))
    .func< &::asclepia::ecs::AssignComponentFromMetaAny< ObjectComponent > > (
	    ::asclepia::ecs::MetaAnnotation::kAssignToEntity)
    .ctor<> ()
    .ASCLEPIA_ECS_NAMED_MEMBER_VARIABLE (ObjectComponent, i, "i")
    .prop (::asclepia::MetaTrait::kIsRequired)
    .ASCLEPIA_ECS_NAMED_MEMBER_VARIABLE (ObjectComponent, f, "f")
    .ASCLEPIA_ECS_NAMED_MEMBER_VARIABLE (ObjectComponent, s, "s");

if (auto const assign_component_func =
        attachment_type.func (MetaAnnotation::kAssignToEntity))
{
	auto const invoke_res = assign_component_func.invoke (
	    *attachment_instance,
	    *attachment_instance,
	    i_entity,
	    &io_registry);
		ASCLEPIA_ECS_DEBUG_ASSERT (invoke_res, assert::Level::kLogicCheck);
}

Wow, that looks like it will work. It took a few guesses, but reading the example code for meta helped me figure out the calling conventions and stuff. I was having a spot of bother with getting it working and so simplified it by skipping the template function and putting a couple of test functions on the component. Once I had the calling convention right it just worked. Lovely.
I need to switch it back to using the template since littering my components with a helper function each isn't a great plan. Once I do that and clean up my test code I'll post here with the minimal code I needed to make it work (skipping over the serialisation parts, since that is cryengine dependant).
Thanks for the help! I'm off to have a beer for now.

Hopefully this is the last piece of the puzzle.
I can't seem to make the .func call the template version of the function. To get around it I just added a function with that signature to each of my test components and called it from them. That's not a great idea going forward, since I'll be duplicating and specialising a lot of code - which is exactly what templates are meant to do.
I've added a template at the same namespace scope as my test functions i.e.
template <typename TComponent>
void AssignComponentFromMetaAny(
	TComponent const& i_component_instance,
	::entt::entity    i_entity,
	::entt::registry* io_registry)
{
	io_registry->assign< TComponent >(i_entity, i_component_instance);
}

There's a class in the same namespace as the template with a test function I am calling. At one point it registers the meta for one of the components:
		entt::meta<ECS::Position>()
			.type("position"_hs)
//			.func<&ECS::Position::AssignComponentFromMetaAny>("assign"_hs)
			.func<&AssignComponentFromMetaAny <ECS::Position> >("assign"_hs)
			.base<ECS::IComponent>()
			.ctor<>()
			.ctor<float, float>();

To my eye, it looks like it should work, but the compile spews the usual soup of garbage when working with templates:
1>------ Build started: Project: Chrysalis, Configuration: Debug x64 ------
1>Item_uber.cpp
1>Plugin_uber.cpp
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(201,18): error C2672: 'std::invoke': no matching overloaded function found (compiling source file D:\Chrysalis\solutions\win64\Plugin_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(196): message : see reference to function template instantiation 'auto entt::internal::invoke::<lambda_11e77117be53435ad121b47c2e0009b6>::operator ()<Type,Type,entt::entity,entt::basic_registry<entt::entity>*>(Type *,Type *,entt::entity *,entt::basic_registry<entt::entity> **) const' being compiled
1>        with
1>        [
1>            Type=Chrysalis::ECS::Position
1>        ] (compiling source file D:\Chrysalis\solutions\win64\Plugin_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(659): message : see reference to function template instantiation 'entt::meta_any entt::internal::invoke<Type,void Chrysalis::AssignComponentFromMetaAny<Chrysalis::ECS::Position>(const TComponent &,entt::entity,entt::registry *),Policy,0,1,2>(entt::meta_handle,entt::meta_any *,std::integer_sequence<unsigned __int64,0,1,2>)' being compiled
1>        with
1>        [
1>            Type=Chrysalis::ECS::Position,
1>            TComponent=Chrysalis::ECS::Position,
1>            Policy=entt::as_is_t
1>        ] (compiling source file D:\Chrysalis\solutions\win64\Plugin_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(653): message : see reference to class template instantiation 'entt::internal::meta_function_helper<void (const TComponent &,entt::entity,entt::registry *)>' being compiled
1>        with
1>        [
1>            TComponent=Chrysalis::ECS::Position
1>        ] (compiling source file D:\Chrysalis\solutions\win64\Plugin_uber.cpp)
1>D:\Chrysalis\code\ChrysalisCore\Item\ItemSystem.h(222): message : see reference to function template instantiation 'entt::extended_meta_factory<Type,std::integral_constant<void (__cdecl &)(const TComponent &,entt::entity,entt::registry *),void Chrysalis::AssignComponentFromMetaAny<Chrysalis::ECS::Position>(const TComponent &,entt::entity,entt::registry *)>> entt::meta_factory<Type>::func<void Chrysalis::AssignComponentFromMetaAny<Chrysalis::ECS::Position>(const TComponent &,entt::entity,entt::registry *),entt::as_is_t>(const uint32_t) noexcept' being compiled
1>        with
1>        [
1>            Type=Chrysalis::ECS::Position,
1>            TComponent=Chrysalis::ECS::Position
1>        ] (compiling source file D:\Chrysalis\solutions\win64\Plugin_uber.cpp)
1>D:\Chrysalis\code\ChrysalisCore\Item\ItemSystem.h(219): message : see reference to class template instantiation 'entt::meta_factory<Chrysalis::ECS::Position>' being compiled (compiling source file D:\Chrysalis\solutions\win64\Plugin_uber.cpp)
1>D:\Chrysalis\code\ChrysalisCore\Item\ItemSystem.h(216): message : see reference to class template instantiation 'entt::meta_factory<Chrysalis::ECS::IComponent>' being compiled (compiling source file D:\Chrysalis\solutions\win64\Plugin_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(196,103): error C2893: Failed to specialize function template 'unknown-type std::invoke(_Callable &&,_Types &&...) noexcept(<expr>)' (compiling source file D:\Chrysalis\solutions\win64\Plugin_uber.cpp)
1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.24.28314\include\type_traits(1579): message : see declaration of 'std::invoke' (compiling source file D:\Chrysalis\solutions\win64\Plugin_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(196,103): message : With the following template arguments: (compiling source file D:\Chrysalis\solutions\win64\Plugin_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(196,103): message : '_Callable=void (__cdecl &)(const TComponent &,entt::entity,entt::registry *)' (compiling source file D:\Chrysalis\solutions\win64\Plugin_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(196,103): message : '_Types={Type &, Type &, Type &, Type &}' (compiling source file D:\Chrysalis\solutions\win64\Plugin_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(201,18): error C2672: 'std::invoke': no matching overloaded function found (compiling source file D:\Chrysalis\solutions\win64\Item_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(196): message : see reference to function template instantiation 'auto entt::internal::invoke::<lambda_11e77117be53435ad121b47c2e0009b6>::operator ()<Type,Type,entt::entity,entt::basic_registry<entt::entity>*>(Type *,Type *,entt::entity *,entt::basic_registry<entt::entity> **) const' being compiled
1>        with
1>        [
1>            Type=Chrysalis::ECS::Position
1>        ] (compiling source file D:\Chrysalis\solutions\win64\Item_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(659): message : see reference to function template instantiation 'entt::meta_any entt::internal::invoke<Type,void Chrysalis::AssignComponentFromMetaAny<Chrysalis::ECS::Position>(const TComponent &,entt::entity,entt::registry *),Policy,0,1,2>(entt::meta_handle,entt::meta_any *,std::integer_sequence<unsigned __int64,0,1,2>)' being compiled
1>        with
1>        [
1>            Type=Chrysalis::ECS::Position,
1>            TComponent=Chrysalis::ECS::Position,
1>            Policy=entt::as_is_t
1>        ] (compiling source file D:\Chrysalis\solutions\win64\Item_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(653): message : see reference to class template instantiation 'entt::internal::meta_function_helper<void (const TComponent &,entt::entity,entt::registry *)>' being compiled
1>        with
1>        [
1>            TComponent=Chrysalis::ECS::Position
1>        ] (compiling source file D:\Chrysalis\solutions\win64\Item_uber.cpp)
1>D:\Chrysalis\code\ChrysalisCore\Item\ItemSystem.h(222): message : see reference to function template instantiation 'entt::extended_meta_factory<Type,std::integral_constant<void (__cdecl &)(const TComponent &,entt::entity,entt::registry *),void Chrysalis::AssignComponentFromMetaAny<Chrysalis::ECS::Position>(const TComponent &,entt::entity,entt::registry *)>> entt::meta_factory<Type>::func<void Chrysalis::AssignComponentFromMetaAny<Chrysalis::ECS::Position>(const TComponent &,entt::entity,entt::registry *),entt::as_is_t>(const uint32_t) noexcept' being compiled
1>        with
1>        [
1>            Type=Chrysalis::ECS::Position,
1>            TComponent=Chrysalis::ECS::Position
1>        ] (compiling source file D:\Chrysalis\solutions\win64\Item_uber.cpp)
1>D:\Chrysalis\code\ChrysalisCore\Item\ItemSystem.h(219): message : see reference to class template instantiation 'entt::meta_factory<Chrysalis::ECS::Position>' being compiled (compiling source file D:\Chrysalis\solutions\win64\Item_uber.cpp)
1>D:\Chrysalis\code\ChrysalisCore\Item\ItemSystem.h(216): message : see reference to class template instantiation 'entt::meta_factory<Chrysalis::ECS::IComponent>' being compiled (compiling source file D:\Chrysalis\solutions\win64\Item_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(196,103): error C2893: Failed to specialize function template 'unknown-type std::invoke(_Callable &&,_Types &&...) noexcept(<expr>)' (compiling source file D:\Chrysalis\solutions\win64\Item_uber.cpp)
1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.24.28314\include\type_traits(1579): message : see declaration of 'std::invoke' (compiling source file D:\Chrysalis\solutions\win64\Item_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(196,103): message : With the following template arguments: (compiling source file D:\Chrysalis\solutions\win64\Item_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(196,103): message : '_Callable=void (__cdecl &)(const TComponent &,entt::entity,entt::registry *)' (compiling source file D:\Chrysalis\solutions\win64\Item_uber.cpp)
1>D:\Chrysalis\code\SDK\EnTT\src\entt\meta\factory.hpp(196,103): message : '_Types={Type &, Type &, Type &, Type &}' (compiling source file D:\Chrysalis\solutions\win64\Item_uber.cpp)
1>Done building project "Chrysalis.vcxproj" -- FAILED.
2>------ Skipped Build: Project: INSTALL, Configuration: Debug x64 ------
2>Project not selected to build for this solution configuration
========== Build: 0 succeeded, 1 failed, 5 up-to-date, 1 skipped ==========

What could be causing this and how do I fix it?

It seems like at some point you get a pointer to the registry and then take its address again and pass the latter to the function. It's the best guess I have.

Sorry guys if I'm late to the party.
@ivanhawkes what version of EnTT are you using? master or a tag?
I'm from mobile now so I'm not 100% sure but I want to reassure you. At a first glance, the error isn't your fault nor of the library. It's due to a new fresh bug of MSVC instead.
There is a PR open for it and I've copy-merged the changes on branch experimental but it hasn't landed on master yet. Probably it should, even though this will cause some conflicts.

I opted for the master tag, thinking stability would be a good introduction to the project.
It's good to hear it's neither my nor the libraries fault - not so surprising to hear it's come from MSVC. I've been running into trouble a little with that lately. A version or two of cryengine are failing to build thanks to one issue or the other with the compilers. I recently had to delete and re-install MSVC to downgrade my compilers to get other code to build again.
Do you have an ETA on when the experimental stuff might get merged into master? If it's not too far into the future I might just push on with the error and the workaround I am using until a fix comes along.

The issue is on the latest version of msvc afaik. See #373 for more details.
I try to not merge unstable things on master for stability actually but I can do nothing on compilers updates that break backwards compatibility with the language.
experimental will be a huge change from the point of view of working with EnTT across boundaries.
I'm almost finished but the upcoming vacancies could introduce a delay.
Because this issue affects master, I could try to cut a new patch release from master instead.
This should be easier and will make the patch available soon.
I cannot really merge the PR as is because of the conflicts it would have with experimental. However, I can take the relevant parts and reduce this problem to a minimum.

This is the error reproduced also on master.
It confirms that at least MSVC 19.24.28314.0 suffers from this problem. :+1:
May I ask you what version of MSVC are you running currently?

Mine is MSVC 19.24.28314.0 - so, same version it seems.
I'll do a pull on master and see what happens.

I can confirm that this fix works for my issue! The .func is happily accepting the templated code now.
Thanks for all the time you've spent helping me get this test running. With the ability to read and write components to XML I can now start prototyping some components and testing systems. I've been looking forward to this for quite a while now.
I'll clean up the test code a bit and post the interesting parts here for anyone who is interested.

With everything I need for the test working, here's the breakdown on the issue with code to show the solution.
Issue - I needed to be able to read and write the entity and component data to XML / JSON files. Initial tests using the snapshot code didn't go well, mainly because I didn't get the functions all defined for the archive manager. It needed one more which I didn't find out about till late in this process. That led me to try and do it using views. I quickly realised that it would be very  unwieldy doing it like that as each component needed it's own block of code. The views also didn't return entities unless they had all the components in it's list. I had to switch back to snapshots.
I started with two component classes, Position and Velocity. While EnnT would be happy with POD I needed a little abstraction to make cryengine happy. Each component is therefore derived from a base class called IComponent. This gives me a little leeway on handling things.
struct IComponent
{
	IComponent() = default;
	virtual ~IComponent() = default;

	// This should be pure virtual but the ECS needs to be able to instantiate the struct, so...here's nothing.
	virtual bool Serialize(Serialization::IArchive& archive) { return true; };

	virtual const CryGUID& GetGuid() const
	{
		static CryGUID guid = "{DEADDEAD-DEAD-DEAD-DEAD-DEADDEADDEAD}"_cry_guid;

		return guid;
	}

	virtual const string& GetName() const
	{
		static string name = "ILLEGALVALUE";

		return name;
	}
};

Only the Serialize function is interesting here. It's responsible for serialising data to and from archives. Cryengine provides support code for that, and I was happy to re-use it for this.
The first step in the process is to register the components with the meta system so we can look them up later, construct them, and then assign them to registries.
		entt::registry saveRegistry;

		// Register meta.
		entt::meta<ECS::IComponent>()
			.type("icomponent"_hs);

		entt::meta<ECS::Position>()
			.type("position"_hs)
			.func<&AssignComponentFromMetaAny <ECS::Position> >("assign"_hs)
			.base<ECS::IComponent>()
			.ctor<>()
			.ctor<float, float>();

		entt::meta<ECS::Velocity>()
			.type("velocity"_hs)
			.func<&AssignComponentFromMetaAny <ECS::Velocity> >("assign"_hs)
			.base<ECS::IComponent>()
			.ctor<>()
			.ctor<float, float>();

I made sure to register their base class, just in case. The main thing here though is calling the .func member to register a templated function that will add the instantiated components correctly to a registry. Doing this allows us to avoid having a massive switch statement with a case for each component type. The classes aren't known till runtime, so they need a runtime solution to deal with creating them and assigning them.
		entt::registry loadRegistry;
		TestLoad(loadRegistry);

I need a registry to load the test data into and a function to perform the load.
	void TestLoad(entt::registry& registry)
	{
		// Load the file into an in-memory structure.
		if (XmlNodeRef entitiesNode = GetISystem()->LoadXmlFromFile("chrysalis/parameters/items/test.xml"))
		{
			// Iterate through all the children which should be individual entities.
			for (int i = 0, n = entitiesNode->getChildCount(); i < n; ++i)
			{
				// Grab the entity, which is basically an empty wrapper.
				XmlNodeRef entityNode = entitiesNode->getChild(i);

				// We should create a fresh entity for the components to attach onto.
				auto entity = registry.create();

				// Get the properties node - this holds all the properties for this component.
				if (XmlNodeRef componentsNode = entityNode->findChild("components"))
				{
					// Iterate through all the components.
					for (int i = 0, n = componentsNode->getChildCount(); i < n; ++i)
					{
						// Grab the entity, which is basically an empty wrapper.
						XmlNodeRef componentNode = componentsNode->getChild(i);

						// Using the tag as a unique ID for the class for now.
						auto hash = entt::hashed_string{ componentNode->getTag() };

						// Ask the system for the class and default construct a component of that type.
						auto component = entt::resolve(hash);
						auto any = component.construct();

						// Get the properties node - this holds all the properties for this component.
						XmlNodeRef propertiesNode = componentNode->findChild("properties");

						// Serialise the properties across to the component.
						auto& iComponent = any.cast<ECS::IComponent>();
						Serialization::LoadXmlNode(iComponent, propertiesNode);

						// Call the component specific function that assigns it to the registry.
						if (auto const component_func = component.func("assign"_hs))
						{
							auto const invoke_res = component_func.invoke(
								any,
								any,
								entity,
								&registry);
						}
					}
				}
			}
		}
	}

You can glance over the XML stuff, it's basically just working through a DOM, getting each entity and reading the component data. It creates a new entity for each node under the main one. It then iterates the components. The XML tag has the hashed string name we used to register in the meta. A call to resolve followed by construct will give us a shiny new component. It exists now, it's just not in a registry attached to an entity yet.
Because I already have code that can serialise from the XML DOM to a component, I get that sorted with a call to the LoadXMLNode routine (not shown). This relies on the iComponent having a Serialize method and polymorphism.
Next, we check the component to see if it has a function (.func) attached to it that matched the name for the template that assigns components to entities. If it exists we invoke that function. The meta is handling the job of figuring out what the component is, and which function and parameters it is called using. That gets our new component added into the registry and attached to it's entity.
To make this a round trip we need to be able to save those entities and components back out to an XML file.  I'm using the snapshot system for this, since it can save out correctly even when an entity doesn't include all of the listed component types.
struct Serial
{
	// Input.
	// NOTE: Missing one function, but we're not doing input yet, so haven't chased it up yet.
	template<typename Type>
	void operator()(entt::entity& entity, Type& component)
	{
	}

	// Output.
	void operator()(unsigned int entity)
	{
		// Not really sure why this is getting called. Need it defined to prevent compile errors.
	}

	void operator()(entt::entity entity)
	{
		if (m_entitiesNode)
		{
			// Each entity get's a couple of nodes made for storing it's components.
			XmlNodeRef entityNode = m_entitiesNode->newChild("entity");
			XmlNodeRef componentsNode = entityNode->newChild("components");

			// Store a copy in a map to enable fast lookup when iterating the components.
			m_nodeMap[entity] = componentsNode;
		}
	}

	template<typename Type>
	void operator()(entt::entity entity, const Type& component)
	{
		// Grab the components node from the map and then inject the component into the XML DOM.
		if (XmlNodeRef componentsNode = m_nodeMap[entity])
		{
			SaveComponent(componentsNode, component);
		}
	}

	void StartSave()
	{
		m_entitiesNode = GetISystem()->CreateXmlNode("entities");
	}

	void EndSave()
	{
		m_entitiesNode->saveToFile("chrysalis/parameters/items/test-out-snapshot.xml");
	}

private:
	XmlNodeRef m_entitiesNode;
	std::map<entt::entity, XmlNodeRef> m_nodeMap;
};

		Serial serial;
		serial.StartSave();
		loadRegistry.snapshot()
			.entities(serial)
			.component<ECS::Position, ECS::Velocity>(serial);
		serial.EndSave();

The serial struct is just the minimum code I needed to handling calls from the snapshot. It's not complete and it's definitely not well written...test code.
The snapshot makes calls onto the serial object. I interpret those into a few lines of code to modify an XML DOM. When it's finished, I write that DOM to a file. Not elegant, but it works with the systems I have on hand.
Overall, it's not a lot of code. Knowing what code to write was tough but the real trial was all the times something wasn't exactly right for the templates and MSVC would spew tons of spurious error messages that are eye wateringly hard to read.
Hopefully this helps any people who run into this issue in future.

Just a side note that probably will make you happy. :)
You can avoid the call to assign the updated component to the entity.
It's possible to define a constructor to which you pass a registry and an entity. The constructor can then default-assign the component of the given type to the entity and returns an aliased meta any to work with.
This way, you can read from the XML file, instantiate the right component and modify the data members of the object you get from the received meta any. It will edit directly the instance contained in the registry.
Moreover, you get rid completely of any copy of your components with a nice boost in terms of performance.

Also @roig this issue can be of interest for you know what. :wink:

Oh, that sounds interesting, especially since it would allow me to cut out a bunch of code.

Well, that didn't take long to test, and it works! For the record I have added a new .ctor to the meta which passes in an entity and a registry pointer. That takes care of the call to assign the newly constructed component.
Code changes:
// Construct and immediately assign to an entity in a registry.
Position(const entt::entity entity, entt::registry* registry)
{
	registry->assign<Position>(entity, *this);
}

Add that to the position component.
.ctor<const entt::entity, entt::registry*>()

Add that to the line which creates the meta entry.
auto any = component.construct(entity, &registry);

Change the line which constructs the component so it takes the two parameters.
I must say, I'm thoroughly impressed with this body of work. I wouldn't want to be writing all that template code, let alone the funky methods needed for type erasure and all the other cool things the code is doing. Thanks once again!

I spoke too soon. While it does run, it loses the data in the components and saves out garbage values for them. I think the issue is that the component is both constructed and assigned to the registry before any values have been filled in. It seems like it is copying by value rather than moving the actual reference. My call to:
auto& iComponent = any.cast<ECS::IComponent>();

Returns what I presume is just a stale copy of the data rather than a reference to it.
Have I missed a step or some nuance?

Uhm, no, it's not like this. I'm trying to sketch it out of my mind since I've not the code at hand right now.
The idea is that of creating a free function to construct your types directly in the registry and return them by reference:
template<typename Type>
Type & assign(const entt::entity entity, entt::registry* registry) {
	return registry->assign<Type>(entity);
}

Then register it as a constructor that returns an aliased meta_any:
.ctor<&assign<Position>, entt::as_alias_t>()

Now, when you do this:
auto any = entt::resolve("Position"_hs).construct(entity, &registry);

What you obtain is an instance of meta_any that doesn't contain a copy of the component. It holds instead a reference to the objects within the registry, that is the one already assigned to your entity. So far, we have no copies at all.
Because you used a base class, you can do this now:
auto& iComponent = any.cast<ECS::IComponent>();

Editing iComponent will modify the instance in the registry.

Side note.
Actually, I did something slightly different though. I've the constructor that returns an alias. When I read the xml or json file, I find the name of the component and construct it as:
auto any = entt::resolve(entt::hashed_string::value(name_from_xml_as_string)).construct(entity, &registry);

The meta_any comes with a type member that returns a meta_type. Moreover, it can be used as an argument to set the parameters.
So, imagine I read from the xml that the parameter x must be set to 20. You can do something like this:
auto type = any.type();
type.data(entt::hashed_string::value(member_to_set_from_xml_as_string)).set(any, value);

Note that:

value is the value you read from the xml, so a variable of integer type with value 20, nothing special here. If you want to pass it by reference and avoid copies you can still use std::ref(value) to exploit aliasing.

*any it the component on which you want to set the value, that is the one within the registry. Since it's already an alias, you can pass it directly to the setter. A meta_handle is created from it and it still behaves as an alias by copying the reference from the meta_any.

Again, you've zero copies in this process (especially if you use std::ref for the arguments) and you set (as in .set) a data member of a component directly in the registry. No need to replace it later.

I hope I gave you a hint on how this works. Let me know if something is unclear. :+1:

The base class is there to make other parts of cryengine happy. I expect I'll want to expose these components to the editor in some places. There's some functions they all need to expose e.g. Serialize. Since I want to use the existing serialise code to handle marshalling data over for these components it makes sense to just allow them to be derived from it. That gives me the cheap and easy option of casting to an IComponent and calling the Serialize on it. It's probably not technically needed, but it's likely to be useful at some point.
I think I understand the code you posted, and I've popped it into my project and it does work as expected - so, happy days! I'm reading the wiki page on runtime reflection now, and it has the salient information I think I didn't know first time around.

I'm looking into how I can make inventory systems, item systems, and perhaps spell / RPG systems of other sorts. Should be a ton of fun :D

It's probably not technically needed, but it's likely to be useful at some point.

Sure, it makes sense. I was just curious because it hasn't an actual role in the example you posted so it seemed superfluous to me at a first glance. Thanks for answering. :+1:

I've popped it into my project and it does work as expected

:tada:
Aliasing is probably the less known and more powerful feature of the runtime reflection system.
It makes possible to do almost everything with literally no copies, that is a huge boost in terms of performance other than reducing quite a lot the number of headaches.
As a side note, I'll soon allow aliasing also for non-copyable types. It will make possible to pass the registry by reference to our functions from the external context.

Should be a ton of fun

You know where to find me/us. :smile:
Since your work is definitely interesting, feel free to drop you updates here, in the gitter channel or wherever you find it's suitable for the purpose.
I'm looking forward to know your progresses and thanks for using EnTT! :heart:

Hello!
You could try using EnTT's meta for deserialisation.  Here's an implementation for JSON. I think it should be more or less straight forward to implement the same for XML. One day I may try to make it more generic, but right now I don't have time for it.

Pastebin code is expired, do you mind pasting it in Github ?

Uhm... This question is from 2019. It's likely that many things changed in the meantime. Maybe it's worth asking about what you're looking for to provide a more up-to-date example?


Set-DbaInstance, Clear-DbaInstance

name: Feature request/idea 💣
about: having a set of functions that permit the human interactive user to not repeat parameters if the target doesn't change. Think Connect-ViServer of PowerCLI

Summary of new feature
do not type -sqlinstance 'foo' for each function if an instance is set. Fortunately, PS has intrinsec support for it, but probably is not well known.
with those function, you could
Set-DbaInstance -SqlInstance 'foo'
Get-DbaDatabase .....
Backup-DbaDatabase .....
....

instead of what usually happens now
Get-DbaDatabase -SqlInstance 'foo'
Backup-DbaDatabase -SqlInstance 'foo'
....

Proposed technical details (if applicable)
Thanks @SQLDBAWithABeard ,
$PSDefaultParameterValues = @{"*-Dba*:SqlInstance" = 'foo'}
suffice for the set. Clear will just remove the previously set value (if any)
Not sure if the same should support, e.g., -Database, '-ExcludeDatabase`, etc etc etc
Latest version of dbatools as of writing
0.9.338

and also SQLCredential

Just curious but how is this different than doing:
Set-DbaInstance -SqlInstance 'foo'
Get-DbaDatabase .....
Backup-DbaDatabase .....
....```

As

```powershell
$server = Connect-DbaInstance 'foo'
Get-DbaDatabase -SqlInstance $server...
Backup-DbaDatabase -SqlInstance $server.....
....

saves on the keystrokes no need to keep specifying SqlInstance and sqlcredential
Set-DbaInstance -SqlInstance 'foo'
Get-DbaDatabase
Backup-DbaDatabase

So if you are working on foo (or a group of foos you specify once at the beginning of session and then all future commands use it

That seems redundant the way it is being explained but I'll wait to see the commands. I see that as basically a CMS at the command line.

I love this and have wanted something similar. I'll work on it.

Right now, this is in my Core profile
$password = ConvertTo-SecureString "dbatools.IO" -AsPlainText -Force
$cred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList "sqladmin", $password
$PSDefaultParameterValues["*:SqlInstance"] = "localhost:14333"
$PSDefaultParameterValues["*:Source"] = "localhost:14333"
$PSDefaultParameterValues["*:Destination"] = "localhost:14334"
$PSDefaultParameterValues["*:Primary"] = "localhost:14333"
$PSDefaultParameterValues["*:Mirror"] = "localhost:14334"
$PSDefaultParameterValues["*:SqlCredential"] = $cred
$PSDefaultParameterValues["*:SourceSqlCredential"] = $cred
$PSDefaultParameterValues["*:DestinationSqlCredential"] = $cred
$PSDefaultParameterValues["*:PrimarySqlCredential"] = $cred
$PSDefaultParameterValues["*:MirrorSqlCredential"] = $cred
$PSDefaultParameterValues["*:WitnessSqlCredential"] = $cred
$PSDefaultParameterValues["*:Confirm"] = $false
$PSDefaultParameterValues["*:SharedPath"] = "/shared"

Perhaps I can mix and match that with our config tool.

yup, found out about psdefaultparameters on 2019.

lol, wait we can create two commands to hand hold

If anyone is interested in picking this up after 2.0 release please create a new feature issue.
I would say at this point it benefits users more to be made aware of how to set param values for any command in a given module.


better approach for a cascading changes in a baobab tree

Hi @Yomguithereal !...I've a little question because I don't know which could be the better approach in this case, suppose than my tree has a 'login' field, when I set this field in my authentication component I need retrieve a list of results using ajax and fill other field (articles,friends or whatever)
I know than cascading changes are considered bad in flux and using the dispatcher and waitfor I can change the two stores,  but in baobab I can't find the correct solution:

authenticationComponent must not change other fields except those related to authentication
I could write a function outside my react components watching for login field changes and then retrieve the articles and fill my tree
inside my List or Articles components watch for changes in login and then retrieve the articles...
create a function which set the login details and then retrieve the articles  (this seems to be the best approach)

also in the readme I can see this

// I am not saying this is what you should do but
    // anyway, if you need to access cursors:
    this.cursors.name.set('Jack');

is better idea create auxiliar functions which changes the tree instead of access directly to the cursor?
setName('Jack') instead of  this.cursors.name.set('Jack');
thanks!!...

at the end I created a function which set the login details and load the details, I think than the resulting code is pretty clear and simple, thanks!...
I'm still curious about the text in the readme about change the cursors directly
thanks!

Hello @cocodrino. As a general rule, you should indeed avoid to update the tree from the component themselves. Even if I let the user access the cursors from the component just in case, you would rather use an action layer à la Flux that will update the tree.

                         ┌────────────────────┐
           ┌──────────── │       State        │ ◀───────────┐
           │             │    (Baobab tree)   │             │
           │             └────────────────────┘             │
        Renders                                          Updates
           │                                                │
           │                                                │
           ▼                                                │
┌────────────────────┐                           ┌────────────────────┐
│        View        │                           │       Actions      │
│ (React Components) │  ────────Triggers───────▶ │ (Functions/Events) │
└────────────────────┘                           └────────────────────┘

thanks so much @Yomguithereal I've found than the code is pretty clear and simple in that way...a last question: if I need change a cursors based in the data stored in other baobab field, is it a good approach?

var currentTweets2 = data.tree.select('tweets', function(ob){
    var _c = data.tree.get(["currentTweetViewOption"]);
    return ob.id == _c
});

var currentTweets = function () {
    var _c = data.tree.get(["currentTweetViewOption"]);
    return data.tree.select('tweets', {id: _c})
};

I know than facets is the correct solution when you need get some view, but it's read only, what is the best approach in this case?
thanks again

thank so much @Yomguithereal, right now I'm migrating a small flux dummy project and I notice than my baobab code is much more clear and simple now, thank you

Don't hesitate to link to your code if it's Open Source so we can exchange about the strategies used.


Deploy finesse for kubernetes

Would like to see a checklist from an issue template for deployments. We should never have empty descriptions.
https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/about-issue-and-pull-request-templates


run cps ir browser tests in the bots

@whesse - we would like to run dart2js with --use-cps-ir in the bots, what would be the best way to proceed? Do we reuse the existing dart2js bots for this, or add more machines to keep the turnaround times more reasonable?

I would add more bots, to keep them separate.  I thought we were already
running this.
On Jan 14, 2016 22:26, "sigmundch"<EMAIL_ADDRESS>wrote:

@whesse https://github.com/whesse - we would like to run dart2js with
--use-cps-ir in the bots, what would be the best way to proceed? Do we
reuse the existing dart2js bots for this, or add more machines to keep the
turnaround times more reasonable?
—
Reply to this email directly or view it on GitHub
https://github.com/dart-lang/sdk/issues/25479.

We are running CPS on the dart2js unit tests, but I don't think we are running them on the html tests. If we were running CPS on the html tests, then that bot would be red since CPS fails many of the html tests.

@whesse - how is this coming along?
To clarify further from what Harry said - we currently have one bot running dart2js in CPSIR + d8, but no bot running on actual browsers. The cpsir+d8 config covers some tests in the html and co19 suites, but none of the browser tests (which are large part of those suites).
We need to add coverage for chrome, ff, ie11, ie10 & safari.
Is there something we can do to help? or something we can do short term before we get all the new bots set up?

Adding new bots isn't that hard - I can do it, but it will slow down
whatever slaves I pick to also do these ones.  Maybe I can use our cloud
slaves, that don't have IPv6.
On Tue, Feb 9, 2016 at 11:41 PM, sigmundch<EMAIL_ADDRESS>wrote:

@whesse https://github.com/whesse - how is this coming along?
To clarify further from what Harry said - we currently have one bot
running dart2js in CPSIR + d8, but no bot running on actual browsers. The
cpsir+d8 config covers some tests in the html and co19 suites, but none of
the browser tests (which are large part of those suites).
We need to add coverage for chrome, ff, ie11, ie10 & safari.
Is there something we can do to help? or something we can do short term
before we get all the new bots set up?
—
Reply to this email directly or view it on GitHub
https://github.com/dart-lang/sdk/issues/25479#issuecomment-182112402.

--
William Hesse

cool, alternatively, any chance we can get more slaves from infra?
If not, I wonder if it's worth splitting the suites to reduce the load on the slaves. For example, I expect the bulk of the co19/language suites are unlikely to have browser-specific behavior, so we could run those only in a subset of the runtimes.

regardless - I'd be ok starting with either of the three options and improving things a bit a a time. Are any of the cloud slaves windows/mac?

No, they are linux, but we have a couple extra windows/mac ones, I think.

:+1: thanks Bill!

We will not add testing of cps-ir on browsers, while development of it is paused.


Support tintColor?

Added on 2.4.0 coming out shortly https://github.com/georstat/react-native-image-cache/blob/f75ce870aa340ba31434d0d9986ca17af7c0439f/src/types.d.ts#L66

Great! it works well now.
thanks @efstathiosntonas


Memory leak using nested name_scope in eager mode

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Both Windows 10 and Ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: Using desktop only
TensorFlow installed from (source or binary): tf-nightly
TensorFlow version (use command below): 2.5.0-dev20201201
Python version: Both 3.7 and 3.8
Bazel version (if compiling from source): Not compiling from source
GCC/Compiler version (if compiling from source): Not compiling from source
CUDA/cuDNN version: 11
GPU model and memory: RTX 2070

Describe the current behavior
It seems using nested name_scope can lead to memory leak. For example, in my setup, the memory grows indefinitely using this code.
import tensorflow as tf
for i in range(int(1e7)):
    with tf.name_scope("first"):
        with tf.name_scope("second"):
            pass

I'm actually not sure this is valid usage of name_scope. However, it seems this king of nested usage of name_scope is used in Tensorflow Probability (here for example). Another minimal example leading to memory leak is:
import tensorflow_probability as tfp
distr = tfp.distributions.Normal(loc=1.0, scale=2)
for i in range(int(1e7)):
    distr.sample()

I don't know why, but appending "/" to the name of the scope seems to fix the leak. Here are samples without memory leaks:
import tensorflow as tf
for i in range(int(1e7)):
    with tf.name_scope("first/"):
        with tf.name_scope("second"):
            pass

or
import tensorflow_probability as tfp
distr = tfp.distributions.Normal(loc=1.0, scale=2, name="foo/")
distr._name += "/"  # Force "/" because it's removed in the constructor
for i in range(int(1e7)):
    distr.sample(name="sample/")

It seems like ctx.name_scope is now using a C++ Safe_PyObjectPtr. I'm not sure why but maybe the reference counter is incorrect.

Hi ,can I work on this issue?

@matthieucoquet,

It seems using nested name_scope can lead to memory leak. For example, in my setup, the memory grows indefinitely using this code.

I did not observe any memory leaks while running this given code snippet. Although second example runs indefinitely, the memory consumption is constant. Please find the gist of it here. Thanks!

@amahendrakar
I see the RAM usage going up in the colab notebook you linked (first sample, using only name_scope).
Using the tracemalloc module, I also see increasing memory usage in the colab notebook.
import tensorflow as tf
import tracemalloc
tracemalloc.start()

snapshot1 = tracemalloc.take_snapshot()
for i in range(int(1e7)):
    with tf.name_scope("first"):
        with tf.name_scope("second"):
            pass
snapshot2 = tracemalloc.take_snapshot()
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
print("[ Top 10 differences ]")
for stat in top_stats[:10]:
    print(stat)

Was able to reproduce the issue with the latest TF-nightly, i.e. v2.5.0-dev20201208.
However, I did not observe much increase in the memory usage with TF v2.2 and TF v2.3. Please check the linked gist for reference. Thanks!

I also observed the following API aliases or similar APIs can cause the same issue in older versions of tensorflow.
Users should be cautious when using them on both CPU and GPU from tensorflow 2.4.0 (v2.4.0-rc4-71-g582c8d236cb) to 2.5.1 (v2.5.0-160-g8222c1cfc86).

(tf.name_scope), tf.compat.v1.name_scope

Code to reproduce the issue in tf.compat.v1.name_scope in older versions
import tensorflow as tf
print(tf.version.GIT_VERSION, tf.version.VERSION, flush=True)
print(tf.config.list_physical_devices(), flush=True)

import tracemalloc
tracemalloc.start()

snapshot1 = tracemalloc.take_snapshot()
for i in range(int(1e7)):
    with tf.compat.v1.name_scope("first"):
        with tf.compat.v1.name_scope("second"):
            pass
snapshot2 = tracemalloc.take_snapshot()
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
print("[ Top 10 differences ]")
for stat in top_stats[:10]:
    print(stat)

On my GPU machine, xxx.
v2.5.0-160-g8222c1cfc86 2.5.1
[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'), PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
[ Top 10 differences ]
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6676: size=525 MiB (+525 MiB), count=10000000 (+10000000), average=55 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6602: size=488 B (+488 B), count=1 (+1), average=488 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6598: size=480 B (+480 B), count=1 (+1), average=480 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6605: size=472 B (+472 B), count=1 (+1), average=472 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6690: size=448 B (+448 B), count=1 (+1), average=448 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6479: size=448 B (+448 B), count=1 (+1), average=448 B
45322-5-p/tf.compat.v1.name_scope.py:37: size=440 B (+440 B), count=1 (+1), average=440 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6677: size=440 B (+440 B), count=1 (+1), average=440 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6462: size=432 B (+432 B), count=1 (+1), average=432 B
45322-5-p/tf.compat.v1.name_scope.py:35: size=424 B (+424 B), count=1 (+1), average=424 B

This behavior is also reproducible on my CPU machine:
v2.5.0-160-g8222c1cfc86 2.5.1
[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]
[ Top 10 differences ]
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6676: size=525 MiB (+525 MiB), count=10000000 (+10000000), average=55 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6602: size=488 B (+488 B), count=1 (+1), average=488 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6598: size=480 B (+480 B), count=1 (+1), average=480 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6605: size=472 B (+472 B), count=1 (+1), average=472 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6690: size=448 B (+448 B), count=1 (+1), average=448 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6479: size=448 B (+448 B), count=1 (+1), average=448 B
45322-5-p/tf.compat.v1.name_scope.py:37: size=440 B (+440 B), count=1 (+1), average=440 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6677: size=440 B (+440 B), count=1 (+1), average=440 B
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:6462: size=432 B (+432 B), count=1 (+1), average=432 B
45322-5-p/tf.compat.v1.name_scope.py:35: size=424 B (+424 B), count=1 (+1), average=424 B

It seems to be fixed in tensorflow 2.6.0 (v2.6.0-rc2-32-g919f693420e) and later versions.

Hi,
Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.
The Tensorflow team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.
Please follow the release notes to stay up to date with the latest developments which are happening in the Tensorflow space.


Dmitri/3770 dnsconfig

This PR updates the DNS configuration in the cluster record copy when a cluster is installed via an Ops Center and the Ops Center is older than the version of gravity used to create the cluster.
I removed all default DNS configuration settings besides those in CreateSite (and now one in RequestClusterCopy).
Fixes https://github.com/gravitational/gravity.e/issues/3770.

Yes, I teste on Azure/AWS going from 1->2


chore(RELEASE-1269): default to RHBA for advisory

new default of RHBA for advisories when a type is not provided.

/retest


[Python/CloudEvents] JobSink Integration / Long Running Functions

As described in https://github.com/knative/func/issues/2586, we need to integrate the Python CloudEvents functions with JobSink

when the env variable KNATIVE_EXECUTION_MODE is equal to batch the python runtime will read the event from the file and pass it to the handler
If the function returns a new event it would be produced to an endpoint that is pointed by K_SINK env variable (if present) (so that you can use JobSink in combination with SinkBinding to have "job callbacks")

For this one, we will need to update the parliament library: https://github.com/boson-project/parliament


Qelectron config isn't editable through UI

When trying to edit qelectron related config options through the UI, it doesn't seem to save the modified settings even though editing other types of config works as expected.

@Prasy12 can you add this in?

@santoshkumarradha @kessler-frost - The current UI supports only the SDK and the executor edition. The others were kept as readonly as per the earlier requirements. Can you let us know what exactly needs to be editable from the Qelectron front?

I tried editing the QiskitExecutor , IBMQExecutor settings and they both weren't working. I get a pop up saying the settings have been saved but when I reload the page it's back to the original default settings.

The classical executor settings like DaskExecutor were working fine.


The base path for a Resource should be declared within the resource module

For syntactic clarity and reduced coupling, it makes more sense to bind a Resource to a base path within the Resource module file. That's where the path parameters will be used, so the path itself should be defined there.

Fixed by #28


Support fail on duplicate key option

https://github.com/circe/circe/issues/1130 was resolved for the Jawn parser (https://github.com/circe/circe/pull/1149), but I don't believe this is the case for circe-yaml
Happy to be told I'm wrong, but would be very happy to see/use this feature with circe-yaml as well.

Implementation would probably involve adding a way to pass options to create a separate parser object, then having that parser object pass the relevant option(s) to SnakeYAML LoaderOptions
https://www.javadoc.io/doc/org.yaml/snakeyaml/latest/org/yaml/snakeyaml/LoaderOptions.html


Spawning inside other users is strange

It looks weird from the inside and from outside

This is related (and would be mitigated by) #310.


Not working anymore?

I have tried this sample and it seems to not work anymore :(

Were you able to get it to work @MrHerickS? I am facing the same issues and the documentation isn't the greatest.

Were you able to get it to work @MrHerickS? I am facing the same issues and the documentation isn't the greatest.

It seems like they have deprecated the nodejs drivers, but I was able to convert it to spotware api v2 and use in my projects.

@MrHerickS May I ask that you share your wisdom? Here's what I did:

I made a fork of this repo and updated the protos. See here: https://github.com/tpalts/connect-nodejs-samples
I subscribed to authentication response, as well as spots

Here's the problem:

I cannot catch any events
When I enter faulty credentials, the connection fails, which gives me reason to believe that the connection itself works, but the code is unable to catch events

How did you overcome this? If needed, check out my fork. The readme describes how to run it.

@MrHerickS May I ask that you share your wisdom? Here's what I did:

I made a fork of this repo and updated the protos. See here: https://github.com/tpalts/connect-nodejs-samples
I subscribed to authentication response, as well as spots

Here's the problem:

I cannot catch any events
When I enter faulty credentials, the connection fails, which gives me reason to believe that the connection itself works, but the code is unable to catch events

How did you overcome this? If needed, check out my fork. The readme describes how to run it.

I had plans to push my code to a repo, do you have skype or anything I can contact you in the meantime?


Uncaught TypeError: Cannot read property 'on' of undefined

When you close a file tab, the above JavaScript error occurs. I'm seeing this on Mac OS X 10.10.5, with Chrome 50.0.2661.102 (64-bit).

It appears that editor.getSession() and editor.on() are both not defined, even though it's been used it earlier code.

Is this fixed?

Yes, should be fixed now.


 Provide a means to add global plugin configurations

Summary
Provide a means to configure plugins globally.
Not implemented as of Kong Ingress controller version 0.1.0, the current latest version.
Not specific to K8S version or platform.
Background
Kong can add plugins globally that then apply to any request that does not match a more specific plugin configuration.
At present, the KongPlugin object does not provide any means to configure global plugins, and it is not possible to configure them via Ingress metadata, as all Ingress objects must apply to specific routes.
Implementation considerations

KongPlugin will need an annotation to specify that the plugin should be applied globally, similar to consumerRef for applying on a per-consumer basis. global: true or similar would work.
Conflicting or ambiguous configuration (e.g. setting both global: true and consumerRef: yoda on the same KongPlugin) must be handled in some way. The IC could either reject this configuration and return an error when attempting to create the object, or it could attempt to infer the user's intent (creating both a global and consumer-specific plugin configuration with the same settings).

Hello @rainest,
#112 just got merged into master. This will be available in our next release. Documentation will be updated before the release process. Thanks!


README

README
As a bootcamp student,
I want the prework notes to be structured on a webpage,
So that I can easily find and read the information.
Given the prework project repository,
When I view the README for the repository,
Then I can see a professional README that provides information on the project and how to use it

successfully created our README file


Add tagged releases

Hi! I have a transitive dependency on go-genproto by way of gRPC. I'm bringing my project up-to-date to use Go modules, and I'm finding that it's hard to reason about what version of go-genproto I'm using because there are no tagged releases.
Go has officially adopted semantic versioning, with tags like v1.2.3. Would you consider adding such tags for go-genproto?

Thanks for filing this. We only plan to use modules for dependency management in this repo, and don't plan to tag.
This is due to the fact that this repo is only used for auto-generated code and that the HEAD commit is always the most relevant (since this is the definition that's used by APIs deployed to prod). As such cloud.google.com/go always depends on the latest version of genproto. (you're likely getting an older version because grpc-go depends on an older version of cloud.google.com/go)
Therefore, the overhead of maintaining versions, release notes, and multi-module repositories down the line is not worth the little gain that the readability of versions provide.

it's hard to reason about what version of go-genproto I'm using because there are no tagged releases.

Ideally you should not have to: always use the latest version by way of go get -u. Let us know if for some reason you do need to reason about versions, though.

Here's an example of where I need to reason about versions: My package's current go.mod contains:

google.golang.org/genproto v0.0.0-20180518175338-11a468237815

When I run go mod verify, this line changes to:

google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8

Did the version go up or down? I have to check out the repository and run git show twice to find out. If that was a tagged version, it would be very easy and quick to see.

Therefore, the overhead of maintaining versions, release notes, and multi-module repositories down the line is not worth the little gain that the readability of versions provide.

I don't think there's any need to provide release notes, and certainly not multi-module repositories. My use case would be met with a simple git tag v1.0.0 and an occasional re-tag like get tag v1.0.1, git tag v1.0.123, etc.
For what it's worth, the current Go documentation treats version tagging as a requirement:

As a result of Semantic Import Versioning, code opting in to Go modules must comply with these rules:
Follow semver (with tags such as v1.2.3).

Version tagging is a requirement for semver. We expressly do not support semver, or versions in general. Note that this approach is followed by a large number of repositories including all x/* repos.
And, thanks for providing an example. However, the answer to having to think about which version of genproto to use is always go get -u. As mentioned, HEAD is what's actually deployed, and therefore always the best choice of what to use. :)

Version tagging is a requirement for semver. We expressly do not support semver, or versions in general.

Interesting. My reading of that documentation suggests that version tagging and semver are considered requirements for "code opting in to Go modules." Should I file a ticket to change the docs?

Note that this approach is followed by a large number of repositories including all x/* repos.

I also recently filed a ticket for that: https://github.com/golang/go/issues/31371. :-)

Oh, sorry, I didn't read your link properly. Yes, you're totally right, that wiki expressly states (stated) that modules must follow semver. I've updated the wiki to be a bit less strict and note the fact that semver is not required.
Note: that wiki is community run. It is close, but not exactly canonical documentation.

Quick update: @thepudds wisely points out that cmd/go doc also heavily implies semver is required: https://golang.org/cmd/go/#hdr-Module_compatibility_and_semantic_versioning

The go command requires that modules use semantic versions

We're discussing how best to tackle this quirk. :)

Welp, looks like the semver requirement is correct, but pseudo-versions are semver. :laughing: My fault! Pulling back the doc change with a small amendment.
All that said, unfortunately the point remains that this repo will continue using pseudo-versions.

Here is the way I would look at it personally:

Modules are required to follow semver in order for the go command to behave as documented.
Modules that do not have semver tags are recorded by consumers using a semver version in the form of a pseudo-version. (Typically this will be a v0 major version, unless the module author constructed a v2+ module following the "Major Subdirectory" approach).
Hence, modules that do not apply semver tags and have not created a "Major Subdirectory" are effectively declaring themselves to be in the semver v0 major version series.

Sidenote:  as far as I am aware, semver draws a distinction between semver, a semver version, and a semver VCS tag (the last of which is not covered by the spec itself but is mentioned in an unpublished FAQ on master.   There’s more that could be said on that, but it gets progressively less interesting…).


CSS Compressor should remove rules with empty declaration blocks

Originally reported on Google Code with ID 203
Minify version:
PHP version: 2.1.4

What steps will reproduce the problem?
1. insert an empty selector in the source css file like p{}

a {color:#aaa}
p {}
q {color:#bbb}

Expected output:
a{color:#aaa}q{color:bbb}

Actual output:
a{color:#aaa}p{}q{color:bbb}

Did any unit tests FAIL?
No

Please provide any additional information below.
Especially for long selector paths this can save many bytes, example: .cellpaddingtopbottom>tbody>tr>td
{}

In which cases can this happen?
Where css developers read PageSpeed help pages 'Writing Efficient CSS' <http://developer.mozilla.org/en/Writing_Efficient_CSS>.
This could lead them to group the selectors by type of rule:
1. id rules
2. class rules
3. tag rules
4. universal rules
and within each group sort by key selector, and further follow more specific overrides:
a {color:blue}
   .class a {color:grey}
   .class a {color:green}

In such cases you can easily have selectors where you have nothing to set for the "parent
seletor name" but for readability you would like to keep its name there, like:
a {}
   .class a {color:grey}
   .class a {color:green}

Reported by cerieljacobs on 2010-11-03 01:33:13

Now on CSSmin


testcase.yaml - twister filtering of tests based on DTS address sizing / DTS node prop values

Is your enhancement proposal related to a problem? Please describe.
Currently we have to special case testing various platforms based on dts address sizing. https://github.com/zephyrproject-rtos/zephyr/pull/55103#issuecomment-1449050118 shows the clear example. This is specifically needed when we attempt to validate a test against all native_posix/qemu platforms. Another example https://github.com/zephyrproject-rtos/zephyr/pull/56666#discussion_r1179505773.
Describe the solution you'd like
In a testcase.yaml we should be able to filter based on device-tree properties value. This would allow us to directly case by dts bus address sizing.
Describe alternatives you've considered
Status Quo:
Pro: No work to be done
Con: We keep having to special-case tests causing a lot of boiler-plate which may have to be updated with new qemu.

I'll take a quick look at it myself, but I'm not as familiar with twisters code.

Can you give an example of what the filter would kinda look like in the testcase.yaml?

Can you give an example of what the filter would kinda look like in the testcase.yaml?

I'd like to make a filter like so:
tests:
  # section.subsection
  drivers.sbs_gauge_new_api.emulated:
    tags: drivers fuel_gauge
    filter: >
      dt_node_path_prop("/soc", "#address-cells") > and
      dt_compat_enabled("sbs,sbs-gauge-new-api") and
      (CONFIG_QEMU_TARGET or CONFIG_BOARD_NATIVE_POSIX)

I'm willing to iterate on it a bit myself but wanted to motivate the actual work/PR before doing so. Or if someone who could do it in 5 minutes wanted to do it instead it probably taking me closer to an hour or two as I've never made a change to kconfigfunctions.py
Let me know if you want me to give it a try first or if you think what I'm asking for is mistaken.


Feat/replace md from static to fetched dynamically

Description
This PR builds up on the previously merged test PRs and adds the chipstatus shortcode that fetches markdown content dynamically for chip support status pages.
The files are fetched from respective links in the shortcodes. For more information on how to upload, edit, or remove these files, see this repo's GitLab mirror > Wiki >  Upload files bypassing git.
This PR also replaces the static content on the ESP32-C61 chip status page with the chipstatus shortcode.
Emoji shortcodes
Prior to adding the chip support markdown content statically to this repo, the checkmark ✅ and hourglass ⏳ emojis were replaced with their respective shortcodes :white_check_mark: and :hourglass_flowing_sand:. These shortcodes are not text-based and could potentially lead to different issues.
Since the chip support markdown content is going to be stored outside of git, the emoji shortcodes were replaced back with their icons. This helps render the fetched markdown content considerably faster on a page. In case these icons cause any issues, we can still use the emoji shortcodes if we revert the commit feat(chip-status): remove emoji shortcode rendering for simplification added in this PR.
Related

#186
#330

Testing

Extensive testing has been done in related PRs and locally.

Checklist
Before submitting a Pull Request, please ensure the following:

[x] 🚨 This PR does not introduce breaking changes.
[ ] All CI checks (GH Actions) pass.
[x] Code is well-commented, especially in complex areas.
[x] Git history is clean — commits are squashed to the minimum necessary.

@ginkgm PTAL

I don't see the preview, but code LGTM!


Add handler to return all files in a TUF repository in a single HTTP request

To optimize network requests, we've been asked to implement client and server logic to get a whole repository in a single network request.

Closing. No longer relevant.


Forbids auto-approval to MSS endpoint

Don't accept assignment if user provides a MSS endpoint in the "AutoApproveSubscriptionSites" list.
Will take effect only when we deploy a new ReqMgr...
@ticoann , please take a look.

Patch looks good. created reqmgr2 side of validation #5796


⚠ Bofrost.es has degraded performance

In 9315849, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1635 ms

Resolved: Bofrost.es performance has improved in 700bbd9 after 6 minutes.


Bump DataFusion version

Which issue does this PR close?

N/A
Rationale for this change

I would like to test some of the bug fixes in DF and see if more of the benchmark queries now work.
What changes are included in this PR?

Use DF just before https://github.com/apache/arrow-datafusion/pull/3907 because we are talking about rolling that back.
Are there any user-facing changes?

I don't know yet

Looks great - I went ahead and fixed the cargo.toml bug - this uncovered some necessary changes needed in the python crate.
I believe the other test is https://github.com/apache/arrow-datafusion/pull/3953

This brings also some improvements to q7 and q19 :partying_face:

Looks great. One more thing about the format of the datafusion.proto, is there any standard format so that future PRs will not be influenced by the format change?

I also pushed changes to compile python and updated revs.
Let's see if everything is green now.

The earlier failing test passes now too

Thanks @Dandandan !

Looks great. One more thing about the format of the datafusion.proto, is there any standard format so that future PRs will not be influenced by the format change?

Filed https://github.com/apache/arrow-ballista/issues/462 for this

LGTM. I cannot approve because I created the PR.


[UIMA-6225] Unable to install a pear package into a directory containing special character like <,> or &

Escape XML element values in XMLUtil (which is used to serialize PEAR installation descriptors
Added unit test trying to install PEAR into a path with an XML special character
Cleaned up and modernized TestPearInstallationVerification
Formatted XMLUtil (basically adding missing curlies)

When reviewing, please switch the diff view to ignore whitespace to reduce visual clutter introduced by the formatting/curlies:


Update grid style to match material design specs

This is still a work in progress. Do NOT merge.
Change-Id: I19e87d40cf87f02104a54db849b9cf711a5d6616

Review status: 0 of 1 files reviewed at latest revision, 5 unresolved discussions, all commit checks successful.

vaadin-components-gwt/src/main/webapp/vaadin-grid/vaadin-grid.html, line 184 [r1] (raw file):
Is there a reason for dropping the line-height definition? Now it reverts to the default inherited line-height (which is by default about 1.4 in browsers).

vaadin-components-gwt/src/main/webapp/vaadin-grid/vaadin-grid.html, line 194 [r1] (raw file):
Could use the shorthand color #eee;

vaadin-components-gwt/src/main/webapp/vaadin-grid/vaadin-grid.html, line 203 [r1] (raw file):
Leave the shorthand color, less bytes to transfer.

vaadin-components-gwt/src/main/webapp/vaadin-grid/vaadin-grid.html, line 204 [r1] (raw file):
I would rather not re-specify the font-family, so that font inherits from the host. So just specify font-weight and font-size.

vaadin-components-gwt/src/main/webapp/vaadin-grid/vaadin-grid.html, line 288 [r1] (raw file):
Does this selector actually work correctly in all browsers?

Comments from the review on Reviewable.io

vaadin-components-gwt/src/main/webapp/vaadin-grid/vaadin-grid.html, line 313 [r1] (raw file):
Which selector? I'm not aware of any issues with '>' please explain. And should we update all usage here?

Comments from the review on Reviewable.io


Impressive!  A "pin" checkbox in your titlebar?

This right here:

is why I love to look at the things users are making.
I never know what I'm going to see.  It's always something interesting.
Nice job of not just making something interesting, but posting a screenshot of it too.  I'm impressed!

TIP - The User Settings API may work for your settings file since you're using a JSON file.  It may save a lot of code while providing the same functionality you have.
It's documented here:
https://pysimplegui.readthedocs.io/en/latest/#user-settings-api

yes, I use checkbox in titlebar because that when I set no_titlebar=True the program icon in task bar will disappear. So I use the 'Pin' checkbox to keep the window on the top.

Yea, at the moment I don't have an option to create a taskbar icon for no-titlebar windows or windows that have a custom titlebar.   It's great for "Rainmenter" type of windows where you don't want a titlebar.  But, will be nice to add the option late to include one.
I noticed that there is no call to change the keep_on_top setting so I just added 2 calls, one to set and one to clear.  Not supposed to be doing features right now (so don't tell anyone) but this is a good one to quickly add.  They're only on GitHub at the moment.
I made this little demo to show how to use them and your clever feature:
import PySimpleGUI as sg

"""
    Demo - Pin a window on top

    Note that the PIN used requires Python 3.7+ due to a tkinter problem
    This demo uses a Window call only recently added to GitHub in Aug 2021

    Copyright 2021 PySimpleGUI
"""

sg.theme('dark green 7')

PIN = '📌'
# This custom titlebar inveses the normal text/background colors. Uses a little bigger font
my_titlebar = [[sg.Text('Window title', expand_x=True, grab=True, text_color=sg.theme_background_color(), background_color=sg.theme_text_color(), font='_ 12', pad=(0,0)),
                sg.T(PIN,  text_color=sg.theme_background_color(), background_color=sg.theme_text_color(), enable_events=True, k='-PIN-',  font='_ 12', pad=(0,0),  metadata=False)]]

layout = my_titlebar + \
         [  [sg.Text('This is my window layout')],
            [sg.Input(key='-IN-')],
            [sg.Button('Go'), sg.Button('Exit')]  ]

window = sg.Window('Window Title', layout, no_titlebar=True, resizable=True, margins=(0,0))

while True:
    event, values = window.read()
    print(event, values)
    if event == sg.WIN_CLOSED or event == 'Exit':
        break
    if event == '-PIN-':
        window['-PIN-'].metadata = not window['-PIN-'].metadata
        if window['-PIN-'].metadata:
            window['-PIN-'].update(text_color='red')
            window.keep_on_top_set()
        else:
            window['-PIN-'].update(text_color=sg.theme_background_color())
            window.keep_on_top_clear()

window.close()

Unfortunately, early versions of tkinter can't handle some UNICODE chars so you'll need Python 3.7+

Thanks for your nice demo!
I will update my codes on next version to use this nice feature!

Thank you for the idea.
I not only added a new capability to PySimpleGUI, but I also figured out a nice way to "patch" when an older version of PySimpleGUI is being used.  I searched through all of the demos and this is the first one that I've used this technique.


Added Image tag import on navbar preview code.

Hi @siamislam07,
Thank you for your great contribution and we are waiting for your next contribution.