Publish SDK 1.4.0

1.4.0 version is coming soon!
MSI Link: TBD will be posted soon
Nuget Link: 1.4.0-beta.0 1.4.0-alpha.4
Deb package link: TBD will be posted soon
Summary of changes are here
Updates - 3/5/2020:

1.4.0-alpha.4 Nuget Package is available. It contains new alpha firmware 1.6.108079014 see #530 for the latest on firmware.
Linking issue https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/1118 Neon instructions needed (3/9/2020).

Updates - 3/3/2020:

1.4.0-alpha.3 Nuget package is available and it contains ARM64 binaries for Ubuntu 18.04 and has been tested on Jetson Nano.
Alpha Firmware 1.6.107078014 is provided in this package, however it is not recommended for Linux platforms #1098.

Where can we find the depth library for ARM?

@tkircher all the information is coming in the next few days. I know everyone is very excited about this release :)

Definitely excited. Got a Jetson Nano specifically for this.

Following for arm information

Where would I find a Windows Installer for this release?

This issue is still a TODO, title updated to reflect that.

Updated description and added Nuget Link

How is ARM64 support on Azure Kinect Body Tracking SDK going?

@UnaNancyOwen , ARM support for Body Tracking SDK is in progress and coming soon. I will update with more details.

Incidentally, after upgrading to AzureKinectDK_Fw_1.6.107078014.bin, from the nuget 1.4.0 release, the Kinect stopped responding. Factory reset did not work, and the LED is blinking white.

Incidentally, confirmed that k4aviewer works on the Jetson Nano (arm64).

Description updated, 1.4.0-alpha.3 replaced with
1.4.0-alpha.4

Please consider merge these pull requests (#822, #1097) before to release v1.4.0.
It provide all SDK APIs in C# wrapper by adding missing functions.

Where can we find the depth library for ARM?

same question

The ARM depth library is in the NuGet package, under linux/lib/native/arm64/
The link is here:
https://www.nuget.org/api/v2/package/Microsoft.Azure.Kinect.Sensor/1.4.0-alpha.4
and the file is actually just a zip file. So rename it from .nuget to .zip and unzip it.

The ARM depth library is in the NuGet package, under linux/lib/native/arm64/
The link is here:
https://www.nuget.org/api/v2/package/Microsoft.Azure.Kinect.Sensor/1.4.0-alpha.4
and the file is actually just a zip file. So rename it from .nuget to .zip and unzip it.

Thanks, I know where to find the libdepthengine2.0 file now, but still don't know where to place it under arm64 architecture to run k4aviewer sucessfully.

You almost certainly want to put it in /lib/aarch64-linux-gnu/

How did you get 1.4.0-alpha.4 to install on jetson nano.
I have nuget installed put it doesn't want to pull the package?
Any ETA on the .deb package
nano@nano-desktop:~$ nuget
NuGet Version: <IP_ADDRESS>

You almost certainly want to put it in /lib/aarch64-linux-gnu/

THANKS! My jetson nano works well!

got it had to do this:
alias nuget="mono /usr/local/bin/nuget.exe"

Got the Kinect for Azure running on the Jetson Nano. :)
Anybody have any ideas on making the Kinect for Azure run with Gstreamer.
I want to use the Kinect for Azure with Deepstream.
I have already done this with the Kinect V2
https://youtu.be/MiVizFsyMRU
I know the Kinect V2 & the Kinect for Azure are different but it shouldnt be impossible.

If anybody is interested here is a video of the Kinect for Azure running on the Jetson Nano
https://youtu.be/NpavmEy9zJU

@valdivj what is the (full) command line argument you used.   I'm new to nuget as well.  the syntax isn't intuitive.  also did you have to update the firmware on the K4A?

Had to install mono first
Then ran this:$ alias nuget="mono /usr/local/bin/nuget.exe"
Then this:$ nuget install Microsoft.Azure.Kinect.Sensor -Version 1.4.0-alpha.4
You might want to try this :https://github.com/microsoft/Azure-Kinect-Sensor-SDK
They just added arm support 3 days ago.

Is it possible to use it on RaspberryPi 4? Any tips on how can I perform the installation?

Is it possible to use it on RaspberryPi 4? Any tips on how can I perform the installation?

The short answer is no.

BTW，does anyone try applying Azure SDK ROS package on JETSON Nano? I failed during compling the ROS azure package. Maybe need to modify cmakelist.txt to adapt the arm64 version sdk?

Beta Released today

@UnaNancyOwen we will include #1097 but not #822 since we need more time to review.

@tesych Thanks! Why is #822 stagnant? It seems that reviews have been paused for several months.

@UnaNancyOwen we run into few technical issues, that we are working through and will release as soon as possible.

@tesych I see. I'm looking forward to provide it in v1.4.x (or v1.5.0). Thanks, :)

Here is a link to github repo with instructions to get Azure for Kinect to run on nano
https://github.com/valdivj/Azure-for-Kinect-Jetson-nano

As @valdivj mentions, this is the biggest event since the release of the camera.  many thanks!

Good news, But I just have Nvidia Xavier, Could the SDK with ARM support can be applicable to Nvidia Xavier? if not, could you give the Nvidia Xavier arm support.

Good news, But I just have Nvidia Xavier, Could the SDK with ARM support can be applicable to Nvidia Xavier? if not, could you give the Nvidia Xavier arm support?

Sorry @nonlinear, we don't have any Xavier's to test against. I am thinking it should work though.

@UnaNancyOwen , ARM support for Body Tracking SDK is in progress and coming soon. I will update with more details.

Any updates on this? We'd like to test on an NVidia Jetson TX2.

@nonlinear1 it absolutely works on the jetson nano, so you really shouldn't any issue on the xavier

This a little off topic but if  you are going to use the Jetson Nano and the Kinect for Azure here is a link to a video that shows you how you can easily make a portable Battery power supply to run the pair.
https://youtu.be/WuC0PN5PuYA

Does someone build this new SDK and successfully complie AZURE_ROS_PKG on arm64(jetson nano). Wondering some instructions using this ARM64 SDK on AZURE_ROS_PKG.  Seems the manully compling file structure cannot fit the AZURE_ROS_PKG required provided by microsoft.  Or it will have some updates on AZURE_ROS_PKG ?

Just now, I try to compile the Azure Kinect SDK for ARM in Xavier. But I failed. I first download  1.4.0-alpha.4 from https://www.nuget.org/api/v2/package/Microsoft.Azure.Kinect.Sensor/1.4.0-alpha.4. Then I copy the /linux/lib/native/arm64/release/libdepthengine.so.2.0 from 1.4.0-alpha.4 to "usr/lib" in Xavier. After that, I download the Azure Kinect SDK from https://github.com/microsoft/Azure-Kinect-Sensor-SDK. Is something wrong? Could someone know? Thanks a lot. the following is  a snapshot of error.

@nonlinear1, you are close. This docker file script manages the dependencies that ARM64 builds need. You should be able to use this to get your dependencies installed. This script is for cross compiling ARM64 on AMD64, so don't run the script directly on ARM64 or you will end up with AMD64 binaries.

Thank you webs-b I have succeed in compiling the Azure Kinect SDK in Xavier. OS: ubuntu 18.04. This is the snap shot:

I tried to compling the SDK in Xavier too, but I failed.Is there are something wrong?
FAILED: tests/Transformation/CMakeFiles/transformation_ut.dir/transformation.cpp.o
/usr/bin/c++   -I../extern/azure_c_shared/src/inc -I../extern/azure_c_shared/src/pal/linux -isystem ../extern/googletest/src/googletest/include -isystem ../extern/googletest/src/googletest -I../src/../include -Isrc/sdk/include -I../extern/spdlog/src/include -I../tests/UnitTests/utcommon/inc -isystem ../extern/googletest/src/googlemock/include -isystem ../extern/googletest/src/googlemock -I../include -O2 -g -DNDEBUG -fPIE -fvisibility=hidden -fvisibility-inlines-hidden   -Wall -Wextra -Wno-missing-field-initializers -Werror -pthread -std=gnu++11 -MD -MT tests/Transformation/CMakeFiles/transformation_ut.dir/transformation.cpp.o -MF tests/Transformation/CMakeFiles/transformation_ut.dir/transformation.cpp.o.d -o tests/Transformation/CMakeFiles/transformation_ut.dir/transformation.cpp.o -c ../tests/Transformation/transformation.cpp
../tests/Transformation/transformation.cpp: In member function ‘virtual void transformation_ut_transformation_depth_image_to_point_cloud_Test::TestBody()’:
../tests/Transformation/transformation.cpp:403:43: error: call of overloaded ‘abs(double)’ is ambiguous
if (std::abs(check_sum - reference_val) > 0.001)
^
In file included from /usr/include/c++/5/cstdlib:72:0,
from /usr/include/c++/5/ext/string_conversions.h:41,
from /usr/include/c++/5/bits/basic_string.h:5352,
from /usr/include/c++/5/string:52,
from /usr/include/c++/5/bits/locale_classes.h:40,
from /usr/include/c++/5/bits/ios_base.h:41,
from /usr/include/c++/5/ios:42,
from /usr/include/c++/5/ostream:38,
from ../extern/googletest/src/googletest/include/gtest/gtest.h:56,
from ../tests/UnitTests/utcommon/inc/utcommon.h:7,
from ../tests/Transformation/transformation.cpp:4:
/usr/include/stdlib.h:837:12: note: candidate: int abs(int)
extern int abs (int __x) __THROW attribute ((const)) __wur;
^
In file included from /usr/include/c++/5/ext/string_conversions.h:41:0,
from /usr/include/c++/5/bits/basic_string.h:5352,
from /usr/include/c++/5/string:52,
from /usr/include/c++/5/bits/locale_classes.h:40,
from /usr/include/c++/5/bits/ios_base.h:41,
from /usr/include/c++/5/ios:42,
from /usr/include/c++/5/ostream:38,
from ../extern/googletest/src/googletest/include/gtest/gtest.h:56,
from ../tests/UnitTests/utcommon/inc/utcommon.h:7,
from ../tests/Transformation/transformation.cpp:4:
/usr/include/c++/5/cstdlib:166:3: note: candidate: long int std::abs(long int)
abs(long __i) { return __builtin_labs(__i); }
^
/usr/include/c++/5/cstdlib:174:3: note: candidate: long long int std::abs(long long int)
abs(long long __x) { return __builtin_llabs (__x); }
^
/usr/include/c++/5/cstdlib:179:3: note: candidate: __int128 std::abs(__int128)
abs(__GLIBCXX_TYPE_INT_N_0 __x) { return __x >= 0 ? __x : -__x; }
^

Azure Kinect worked on Jetson Nano! 👍
I'm look forward to more optimization in the future.

Has the Azure Kinect Body Tracking SDK with ARM64 supported been released?
Do we still have to wait a while for that? Thanks,

I have another error now!
FAILED: tools/k4aviewer/CMakeFiles/k4aviewer.dir/k4apointcloudwindow.cpp.o
/usr/bin/g++   -I../tools/k4aviewer -Isrc/sdk/include -I../include -Isrc/record/sdk/include -I../extern/imgui/src -I../extern/imgui/src/examples -I../extern/imgui/src/examples/libs/gl3w -I../extern/imgui/src/examples/libs/glfw/include -I../extern/libjpeg-turbo/src -I../extern/libusb/src/libusb -I../extern/libyuv/src -I../extern/libyuv/src/include -I../extern/glfw/src/include -O2 -g -DNDEBUG -fPIE -fvisibility=hidden -fvisibility-inlines-hidden   -Wall -Wextra -Wno-missing-field-initializers -Werror -idirafter/usr/include/soundio -std=gnu++11 -MD -MT tools/k4aviewer/CMakeFiles/k4aviewer.dir/k4apointcloudwindow.cpp.o -MF tools/k4aviewer/CMakeFiles/k4aviewer.dir/k4apointcloudwindow.cpp.o.d -o tools/k4aviewer/CMakeFiles/k4aviewer.dir/k4apointcloudwindow.cpp.o -c ../tools/k4aviewer/k4apointcloudwindow.cpp
../tools/k4aviewer/k4apointcloudwindow.cpp: In member function ‘virtual void k4aviewer::K4APointCloudWindow::Show(k4aviewer::K4AWindowPlacementInfo)’:
../tools/k4aviewer/k4apointcloudwindow.cpp:95:12: error: ‘SeparatorEx’ is not a member of ‘ImGui’
ImGui::SeparatorEx(ImGuiSeparatorFlags_Vertical);
^~~~~~~~~~~
../tools/k4aviewer/k4apointcloudwindow.cpp:95:12: note: suggested alternative: ‘Separator’
ImGui::SeparatorEx(ImGuiSeparatorFlags_Vertical);
^~~~~~~~~~~
Separator
[13/21] Building CXX object tools/k4aviewer/CMakeFiles/k4aviewer.dir/k4awindowset.cpp.o

I have another error now!
FAILED: tools/k4aviewer/CMakeFiles/k4aviewer.dir/k4apointcloudwindow.cpp.o
/usr/bin/g++ -I../tools/k4aviewer -Isrc/sdk/include -I../include -Isrc/record/sdk/include -I../extern/imgui/src -I../extern/imgui/src/examples -I../extern/imgui/src/examples/libs/gl3w -I../extern/imgui/src/examples/libs/glfw/include -I../extern/libjpeg-turbo/src -I../extern/libusb/src/libusb -I../extern/libyuv/src -I../extern/libyuv/src/include -I../extern/glfw/src/include -O2 -g -DNDEBUG -fPIE -fvisibility=hidden -fvisibility-inlines-hidden -Wall -Wextra -Wno-missing-field-initializers -Werror -idirafter/usr/include/soundio -std=gnu++11 -MD -MT tools/k4aviewer/CMakeFiles/k4aviewer.dir/k4apointcloudwindow.cpp.o -MF tools/k4aviewer/CMakeFiles/k4aviewer.dir/k4apointcloudwindow.cpp.o.d -o tools/k4aviewer/CMakeFiles/k4aviewer.dir/k4apointcloudwindow.cpp.o -c ../tools/k4aviewer/k4apointcloudwindow.cpp
../tools/k4aviewer/k4apointcloudwindow.cpp: In member function ‘virtual void k4aviewer::K4APointCloudWindow::Show(k4aviewer::K4AWindowPlacementInfo)’:
../tools/k4aviewer/k4apointcloudwindow.cpp:95:12: error: ‘SeparatorEx’ is not a member of ‘ImGui’
ImGui::SeparatorEx(ImGuiSeparatorFlags_Vertical);
^~~~~~~~~~~
../tools/k4aviewer/k4apointcloudwindow.cpp:95:12: note: suggested alternative: ‘Separator’
ImGui::SeparatorEx(ImGuiSeparatorFlags_Vertical);
^~~~~~~~~~~
Separator
[13/21] Building CXX object tools/k4aviewer/CMakeFiles/k4aviewer.dir/k4awindowset.cpp.o

Please open a new bug and share steps to reproduce how you got to this build error. My guess is that you are working from the ZIP file download, which we do not support. You need to clode the repo so that submodules get populated and built.

@tesych @wes-b Please update download link in Microsoft Docs. It's still v1.3.0.
https://docs.microsoft.com/en-us/azure/Kinect-dk/sensor-sdk-download#windows-download-link

@tesych @wes-b Please update download link in Microsoft Docs. It's still v1.3.0.
https://docs.microsoft.com/en-us/azure/Kinect-dk/sensor-sdk-download#windows-download-link

Thank you! I have submitted a PR.

If anybody is interested here is a video of the Kinect for Azure  running Nvida Deepstream using the Jetson Nano.
https://youtu.be/Pew5plgRKps

@valdivj using the drill body and battery is a great idea. This is mine:

@tkircher  Looks Clean!
All it needs is a label with your Brand on it. : )

This version (1.4.0) does not support body tracking sdk on Ubuntu 18.04.
Is there any way to deal with this?

This version (1.4.0) does not support body tracking sdk on Ubuntu 18.04.
Is there any way to deal with this?

Body Tracking team is also adding ARM support. A new version of BT is comming that will work with 1.4.0.

Thank you webs-b. I have succeeded in compiling the Azure Kinect SDK in Xavier. OS: ubuntu 18.04. This is the snap shot:

Hello, can you send some link to some instruction where I can install this successfully for Nvidia Xavier!! It would be really helpfully :) Thank you.

Hello, can you send some link to some instruction where I can install this successfully for Nvidia Xavier!! It would be really helpfully :) Thank you.

https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/usage.md#debian-package

Locking thread to encourage the creation of new issues


feat: add ts-rs mapping to our jrpc

Description
The ts-rs crate is very helpful but it has some downsides (bugs).
If you want to generate new mappings. You have to run test with feature ts without integration test : cargo test --workspace --exclude integration_tests export_bindings --features ts. This will run only the ts-rs exporting. But there is a catch, some of the structs are export to wrong directory (dan_layer\bindigs). You have to manually move them to the /bindings folder. Don't forget to run prettier afterwards.
Probably not all settings are exported to typescript correctly. We will fix once we start replacing the anys in the webui projects.
Motivation and Context
It seems like you can change the export_to path to get it into the right directory. But then the ts-rs will panic (because the are expecting that each struct is on the same level.... Maybe in the future version it will work.
Breaking Changes

[x] None
[ ] Requires data directory to be deleted
[ ] Other - Please specify

Nice - perhaps we can add a node package for these bindings and add a command to build them?
Then we can include that package in the web UIs


Exam: Repository of an Programming Exercise can not be opened

Describe the bug
When taking part in an exam as a student and doing a Programming Exercise the repository cannot be opened. When clicking on "Open repository" the exam page is loaded and not the repository in the online editor
To Reproduce

Start exam with student (having a Programming Exercise)
Select Programming Exercise
Click "Open repository"
you will be directed to Exam page and not the repository in the online editor

Expected behavior
The click should lead to the repository
Screenshots
No response
Which version of Artemis are you seeing the problem on?
7.3.1
What browsers are you seeing the problem on?
Chrome
Additional context
No response
Relevant log output
No response

Same problem on exam summary as well.

I was working on moving this button anyway. I'll discuss this with Stephan


Varience shadow map breaks up fragment shader.

Activating VSM shadows breaks up in default lit shader (without clustered lighting)
Error log
ERROR: 0:855: 'getShadowCoordPersp' : no matching overloaded function found

850: getLightDirPoint(light4_position);
851: dAtten = getFalloffLinear(light4_radius, dLightDirW);
852: if (dAtten > 0.00001) {
853: dAtten *= getSpotEffect(light4_direction, light4_innerConeAngle, light4_outerConeAngle, dLightDirNormW);
854: dAtten *= getLightDiffuse(litShaderArgs.worldNormal, dViewDirW, dLightDirW, dLightDirNormW);
855: getShadowCoordPersp(light4_shadowMatrix, light4_shadowParams, dLightDirW);
856: float shadow4 = getShadowSpotVSM32(SHADOWMAP_PASS(light4_shadowMap), dShadowCoord, light4_shadowParams, 15.0, dLightDirW);
857: dAtten *= mix(1.0, shadow4, light4_shadowIntensity);
858: dDiffuseLight += dAtten * light4_color;
859: dHalfDirW = normalize(-dLightDirNormW + dViewDirW);
860: dSpecularLight += getLightSpecular(dHalfDirW, dReflDirW, litShaderArgs.worldNormal, dViewDirW, dLightDirNormW, litShaderArgs.gloss, dTBN) * dAtten * light4_color* litShaderArgs.specularity;

Steps to Reproduce

Add any light to scene.
Change shadow type to any Varience Shadow Map type.
getShadowCoordPersp is broken and objects with shadows are invisible.

Screenshot

Thanks for the report @defektu , we'll fix it in the next few days.

Second report: https://forum.playcanvas.com/t/getlightdefuse-error-on-1-62-1-works-alright-on-1-61-3/30802/2

This second report seems related to chunk changes, not shadow issue - the error message is different.

My bad, I will go back to support

another report https://forum.playcanvas.com/t/suddenly-and-without-changing-anything-i-received-this-error-help/30803

As this affects Editor rendering, should be consider this fix being a patch release?


Add User-Agent to fetch requests

I noticed that certain URLs were throwing 403s when trying to fetch URLs. Link to Sentry.
The reason is that the page was intercepting the User-Agent for urllib Python-urllib/2.7 and then blocking it. I think it makes sense to identify our preview bot with it's own User Agent. This shouldn't break any existing functionality, but should allow us to preview URLS for Cricket. Google and other use similar custom User-Agents for their site crawling robots.
Before:

After:

This resolves PREVIEWS-2.

review me

reviewme


Grappelli Support

When Grappelli override admin's tempaltes the UI breaks.

I thin it is obvious that I don't have the time to implement this. However, if anyone is interested, I can reopen the issue, no prob.


✨ Making NegativeDouble comparable

📝 Description
We would like to make NegativeDouble comparable by implementing the Comparable interface.
interface NegativeDouble : Comparable<NegativeDouble> {
    override fun compareTo(other: NegativeDouble): Int
}

🔗 Dependencies
This issue is blocked by the following ones:

[x] #557

✅ Checklist

[ ] Make the type comparable, test its behavior with Kotlin and its compatibility with Java, update the public API binaries and the unreleased changelog.
[ ] Close this issue as completed and update tracking ones if present.

See #628 for more details on why we are not going to implement this issue for now.


Menge DLL expose ability to add and remove agent

Use Case:
to spawn new agents and to remove agents during simulation.
In XMLSimulatorBase.h
/*!
 *	@brief		Add an agent with specified position to the simulator whose properties
 *				are defined by the given agent initializer.
 *
 *	It uses the agent initializer to define the values of the remaining agent
 *	parameters.
 *
 *	@param		pos			The 2d vector representing the agent's position
 *	@param		agentInit	The AgentInitializer necessary to parse AgentSet properties
 *	@returns	A pointer to the agent (if initialization was succesful) or NULL if
 *				failed.
 */
virtual BaseAgent * addAgent( const Math::Vector2 & pos,  AgentInitializer * agentInit ) = 0;

The only way to add new agent is in SimXMLLoader.cpp
// Second pass, parse Generators
for( child = node->FirstChildElement(); child; child = child->NextSiblingElement()) {
	if ( child->ValueStr() == "Generator" ) {
		AgentGenerator * generator = AgentGeneratorDB::getInstance( child,
																	_sceneFldr );

		if ( generator == 0x0 ) {
			logger << Logger::ERR_MSG << "Unable to instantiate agent generator "
				"specifcation on line " << child->Row() << ".";
			return false;
		}
		// Now instantiate the agents
		const size_t AGT_COUNT = generator->agentCount();
		Vector2 zero;
		for (size_t i = 0; i < AGT_COUNT; ++i) {
			BaseAgent * agent = _sim->addAgent(zero, profileSel->getProfile());
			generator->setAgentPosition(i, agent);
			_sim->getInitialState()->setAgentState(agent->_id, stateSel->getState());
		}
		_agtCount += (unsigned int)AGT_COUNT;

		generator->destroy();
	}
}

if we could expose SimulatorInterface in menge_c_api.h
class MENGE_API SimulatorInterface: public XMLSimulatorBase
we can get into the getInitialState() of XMLSimulatorBase to:
/*!
 *	@brief		Sets the state for the given agent.
 *
 *	@param		id			The identifier of the agent.
 *	@param		stateName	The name of the state to which the agent belongs.
 */
_sim->getInitialState()->setAgentState(agent->_id, stateSel->getState());

OR
/*!
 *	@brief		Reports the state name for the given agent.
 *
 *	@param		id			The identifier of the agent.
 *	@returns	The name of this agent's start state.
 */
const std::string getAgentState( size_t id ) const;

Menge's architecture was changed with the idea of eventually being able to dynamically change the population (i.e., most of the structures use map in place of vectors so that it can adapt to whatever data is available. Part of that design will be the eventual inclusion of agent Sources and Sinks as proper elements.  This work has not been done yet.
I fully encourage you to try what you're suggesting and let us know how it goes. My fear is that there is some finalization work and dependencies that might make this a bit murky.
One of the things I've done in the past to simulation dynamic populations is to have a "sink" state that teleports the agents far away from the main simulation domain, giving them a zero-velocity velocity component. You would then need to have an appropriate transition out of that state that would cause them to teleport back into the meaningful domain. The nature of the transition depends on what effect you're trying to achieve. Let us know if you want to dabble with this hacky approach and we can provide some tips.

@MengeCrowdSim  part of the challenges is lack of sufficient documentation.
For the example 4Squares, which part of the XML files  4SquareB.xml or 4SquareS.xml define the beheviour of agent moving from ONE square to the opposite Square?
For the example Soccer: what is soccer.nav   ( it is a nav_mesh; can I assume it is a path which the agent must follow?)
For the example Botteneck: same with botteneckMap.txt
Some explanation of these files will help
What is VelComponent?
How many types are VelComponent? e.g. road_map; nav_mesh;  More?

You are right.  The documentation always looms over my head.
The first thing I can say is that you should build the doxygen locally. The doxygen on the website (http://gamma.cs.unc.edu/Menge/docs/code/menge/html/) is not up to date. The local doxygen is more up to date. (Looking through it, I see that it's, at best, partially documented. With the focus on the ___S.xml files. I obviously need to reserve more cycles for documenting...)
Another source for enlightenment is to read this paper:
https://collective-dynamics.eu/index.php/cod/article/view/A1
It discusses the architectural overview explaining things like "what is a VelComponent"? Take a look at Section 3. The appendix creates a simple example and dissects the ___S.xml and ___B.xml file.
Some of the detail questions:

The navigation files are undocumented. The roadmap is simple, the nav mesh is complex. I've got cobbled tools that I use to manufacture those data structures.  I've got a background project, a gui-based authoring/execution tool.  Editing XML is a pain. If you're interested, I can send you my python-based authoring tools (limited though they may be) to create roadmaps and nav meshes.

The road map tool is a gui.
The nav mesh generation takes an obj file and converts it to my proprietary nav mesh file format. It assumes that you can generate the obj file (which was the quickest, dirtiest way for me to go from nothing to nav mesh.)

In 4squareB.xml, each agent is assigned a goal position that is its initial position mirrored over the origin (https://github.com/MengeCrowdSim/Menge/blob/master/examples/core/4square/4squareB.xml#L5). It then attempts to move there by walking directly towards that goal point (https://github.com/MengeCrowdSim/Menge/blob/master/examples/core/4square/4squareB.xml#L6)

VelComponent: This is the module that is responsible for defining the instantaneous preferred velocity for an agent (given the agent's state -- including goal, position, etc.) The simplest is the zero velocity component (which is, your preferred velocity is to always go nowhere). Then it gets more complex: take a constant velocity, a constant direction at preferred speed,  a velocity directly towards a point at preferred speed, and, for handling complex environements: follow a vector field, road map, or a navigation mesh versions you've alluded to.
You can see the various declarations here: https://github.com/MengeCrowdSim/Menge/tree/master/src/Menge/MengeCore/BFSM/VelocityComponents

@MengeCrowdSim  Thanks for patiently providing this explanation. VERY USEFUL. It makes more sense NOW.
I will take your offer to look into the GUI later once I understand further the data structure of the road map and Nav mesh.
Q1: One of the goals using menge is to be able to dynamically for each agent, control the goal position during simulation. e.g. Agent x should move from pos A to pos B along a rough calculation of road_Map (perhaps by A-start etc. ). When the Agent x reach pos B. This trigger a callback. This callback request the next goal position e.g. pos C. So another A-start path is calculated to populate the road_map for Agent x to move from pos B to pos C.
I think many of these components are inside menge.
However, I do not have the overview know where these information. Can this be done in plugin? perhaps there is already example using this? I read there are trigger, event, associated with agent, as well as transition (Finite State way)..
=> In short, with the C# binding, I think Menge is now an important Crowd .NET library. Keep it up. WE NEED THIS!!!!!!!

The big question is: How do you pick B initially? And then, upon reaching B, how do you pick C? If you're doing something really esoteric, then a plug-in may be necessary. But, generally, the behavior you've described is very much in line with the behavioral finite state machine that is the heart of Menge.
The agent starts in state 1. State 1 has a goal associated with it and a means for planning for that goal (e.g., a road map). While the agent is in State 1, it will attempt to move toward the goal.
We create a transition from State 1 to state 2. The transition is "goal reached". State 2 then selects a goal and a means to move towards it, and you repeat.
For example, if I scattered a bunch of random goals in a domain, and I wanted the agent to simply randomly move from point to point, I could do that with a single state and a transition pointing back to itself (the tradeshow demo does this -- a set of goal points from which a goal is selected every time an agent enters the state).
If, however, you need to pick a goal on some more complex state, or based on history, or prediction, then you'd need to implement a new GoalSelector that implements that logic.
(Thanks for the encouragement. Always nice to get.)

Thanks for keep answering my questions 👍

Q: Which examples demonstrate how to use Event triggers, Event effects, Event Targets?? Thanks

A: https://github.com/MengeCrowdSim/Menge/blob/master/examples/core/event.xml with the details in https://github.com/MengeCrowdSim/Menge/blob/master/examples/core/event/eventB.xml
In this scenario, every time an agent reaches a goal, it makes all of the other agents (who have not yet reached the goal) get fatter and faster.
Elaboration: the event system was the newest feature to Menge's architecture. The underlying mechanism works, but there aren't many implementation of Event components. I had the idea of using events to create a Pac-man game with Menge under the hood; I thought it would be a cool demo. Short version: there is little out-of-the-box event infrastructure ready to go.
A thought to keep in the back of your mind, what an agent is up to changes over time. If the change is a function of the agent state, then it belongs in the FSM, if it is due to something that has nothing to do with the agent, it belongs in an event.

@MengeCrowdSim  Great job!  Now VS2015 works for both win32, x64 and velocities are extracted.
Q1: How could we expose MORE for .NET binding e.g.
	MENGE_API std::string GetAgentStringID(size_t i);
´
	MENGE_API const Menge::Agents::BaseAgent* GetAgentID(size_t i);

	MENGE_API const Menge::Agents::BaseAgent* GetAgentNeighbor(size_t i);

	MENGE_API const Menge::Agents::Obstacle* GetAgentObstacle(size_t i);

=> Possibilities
(a) Follow the #define MENGE_API __declspec(dllimport)
Implement the corresponding .NET classes for Menge::Agents::BaseAgent and Menge::Agents::Obstacle
(b) Alternative1, create a Manage .NET C++ DLL (as you have discussed, it is not ideal for Unity).
For user who only interested in e.g. WinForm
I have tried the followings:
=> Compile the existing MengeCore with Common Language Runtime Support (/clr) support
=> The resulting MengeCore.DLL can NOW be referenced using WinForm program
(c) Alternative2, create a new Managed C++/CLR class library e.g. MengeNET.dll
Include all header and source files from the existing MengeCore.
Please feedback which approach has the team tried? Pros and Cons?
Prefer (c) approach, less challenging to expose classes e.g.  GoalSelector etc.

@MengeCrowdSim  The (c) approach works - although there are issues which I have not completely figure  out .
Create a new CLR class library with Common Language Runtime Support (/clr) (default)
Change Character Set to : Use Multi-Byte Character Set
Copy call header and source files (except menge_c_api.h and menge_c_api.cpp).
Modified the CoreConfig.h slightly
#if defined(_MSC_VER)
  #if defined( MENGE_STATICLIB )
    #define MENGE_API
	#define MATHEXTERN
  #else
    #if defined( MENGECORE_EXPORTS )
      // We are building the DLL, export the symbols tagged like this
      #define MENGE_API __declspec(dllexport)
	  #define MATHEXTERN
	#elif defined( MENGECORE_NET )
	  #define MENGE_API
	  #define MATHEXTERN
    #else
      // If we are consuming the DLL, import the symbols tagged like this
      #define MENGE_API __declspec(dllimport)
	  #define MATHEXTERN extern
    #endif
  #endif

To create CLR e.g. MengeNET.dll (turn on MENGECORE_NET ),
#pragma unmanaged
#include <string.h>
#include "MengeCore/CoreConfig.h"

#include "MengeCore/Agents/BaseAgent.h"
#include "MengeCore/Agents/SimulatorInterface.h"
#include "MengeCore/BFSM/FSM.h"
#include "MengeCore/PluginEngine/CorePluginEngine.h"
#include "MengeCore/Runtime/SimulatorDB.h"

#pragma managed
using namespace System;
using namespace System::Runtime::InteropServices;

namespace MengeNET {

        public ref class MengeWrapper
	{
	   private:
		Student *_stu;
		Menge::Agents::SimulatorInterface * _simulator = 0x0;

		const char* StringToChar(String^ managedString)
		{
			char* str2 = (char*)(void*)Marshal::StringToHGlobalAnsi(managedString);
			return str2;
		}
	  public:
		MengeWrapper()
		{
		}
		~MengeWrapper()
		{
			delete _stu;
			_stu = 0;

			delete _simulator;
			_simulator = 0;
		}
		bool InitStimulator(String ^ behaveFileName, String ^ sceneFileName, String ^ modelName,
                String ^ pluginPathName)
		{
				const char * behaveFile = StringToChar(behaveFileName);
				const char * sceneFile = StringToChar(sceneFileName);
				const char * model = StringToChar(modelName);
				const char * pluginPath = StringToChar(pluginPathName);

                                 const bool VERBOSE = false;
				if (_simulator != 0x0) delete _simulator;
				Menge::SimulatorDB simDB;

				// TODO: Plugin engine is *not* public.  I can't get plugins.
				Menge::PluginEngine::CorePluginEngine engine(&simDB);
				if (pluginPath != 0x0) {
					//engine.loadPlugins(pluginPath); // <== Can not be compiled [not sure why?)
				}

				Menge::SimulatorDBEntry * simDBEntry = simDB.getDBEntry(std::string(model));
				if (simDBEntry == 0x0) return false;

				size_t agentCount;
				float timeStep = 0.1f;			// Default to 10Hz
				int subSteps = 0;				// take no sub steps
				float duration = 1e6;			// effectively no simulation duration.
				std::string outFile = "";		// Don't write an scb file.
				std::string scbVersion = "";	// No scb version
				bool verbose = false;
				_simulator = simDBEntry->getSimulator(agentCount, timeStep, subSteps, duration,
					behaveFile, sceneFile, outFile, scbVersion,
					verbose);

				return _simulator != 0x0;
		}
                 int AgentCount()
		{
			assert(_simulator != 0x0);
			return (int)_simulator->getNumAgents();
		}

		void SetTimeStep(float timeStep)
		{
			assert(_simulator != 0x0);
			_simulator->setTimeStep(timeStep);
		}

		bool DoStep()
		{
			assert(_simulator != 0x0);
			return _simulator->step();
		}

		bool GetAgentPosition(int i, float %x, float %y, float %z)
		{
			assert(_simulator != 0x0);
			Menge::Agents::BaseAgent * agt = _simulator->getAgent(i);
			if (agt != 0x0) {
				x = agt->_pos._x;
				y = _simulator->getElevation(agt);
				z = agt->_pos._y;
				return true;
			}
			return false;
		}

		bool GetAgentVelocity(int i, float %x, float %y, float %z)
		{
			assert(_simulator != 0x0);
			Menge::Agents::BaseAgent * agt = _simulator->getAgent(i);
			if (agt != 0x0) {
				x = agt->_vel._x;
				y = 0; // get elevation
				z = agt->_vel._y;
				return true;
			}
			return false;
		}

		bool GetAgentOrient(int i, float  %x, float  %y)
		{
			assert(_simulator != 0x0);
			Menge::Agents::BaseAgent * agt = _simulator->getAgent(i);
			if (agt != 0x0) {
				x = agt->_orient._x;
				y = agt->_orient._y;
				return true;
			}
			return false;
		}

		int GetAgentClass(int i)
		{
			assert(_simulator != 0x0);
			Menge::Agents::BaseAgent * agt = _simulator->getAgent(i);
			if (agt != 0x0) {
				return static_cast<int>(agt->_class);
			}
			return -1;
		}

		float GetAgentRadius(int i)
		{
			assert(_simulator != 0x0);
			Menge::Agents::BaseAgent * agt = _simulator->getAgent(i);
			if (agt != 0x0) {
				return agt->_radius;
			}
			return -1;
		}
	};
}

=> There is no more need for MengeCS, Only MengeCSExe to call the MengeWrapper from MengeNET.dll (Manage C++/CLI)
          MengeWrapper menge = new MengeWrapper();
           String scene = "4square";
           String mengePath = @"Directory to Menge\";
           String behaveXml =<EMAIL_ADDRESS>mengePath, scene);
           String sceneXml =<EMAIL_ADDRESS>mengePath, scene);
           if (menge.InitStimulator(behaveXml, sceneXml, "orca", ""))
           {
               this.listBox1.Items.Add(String.Format("New simulator created."));
               this.listBox1.Items.Add(String.Format("\t{0} agents", menge.AgentCount()));
               for (int i = 0; i < 20; ++i)
               {
                   this.listBox1.Items.Add(String.Format("Step {0}", i + 1));
                   if (!menge.DoStep())
                   {
                       this.listBox1.Items.Add(String.Format("Simulation done...quitting"));
                       break;
                   }
                   for (int a = 0; a < menge.AgentCount(); ++a)
                   {
                       float x = 0;
                       float y = 0;
                       float z = 0;

                       float x1 = 0;
                       float y1 = 0;

                       float x2 = 0;
                       float y2 = 0;
                       float z2 = 0;
                       menge.GetAgentPosition(a, ref x, ref y, ref z);
                       Vector3 p = new Vector3(x, y, z);

                       menge.GetAgentOrient(a, ref x1, ref y1);
                       Vector2 d = new Vector2(x1, y1);

                       menge.GetAgentVelocity(a, ref x2, ref y2, ref z2);
                       Vector3 v = new Vector3(x2, y2, z2);

                       this.listBox1.Items.Add(String.Format("\tAgent {0} at ({1}, {2})  Moving Direction: ({3}, {4} with Velocity {5} m/s Speed: (Vx:{6}, V:y{7}, V:z{8} ))", a, p.X, p.Z, d.X, d.Y, v.Length(), v.X, v.Y, v.Z));
                   }
               }

           }
           else
           {
               System.Console.WriteLine("Error initializing simulation");
           }

If you check out this branch (https://github.com/curds01/MengeCS/tree/advanced_wrapper_cppclr) you can see my initial CLR pass on this from back in...January or so.)
I can see that what you have is sufficient, but in my solution, I'd opted for a more class-centric approach.

@JimSEOW I'd recommend that you fork Menge and MengeCS and create a branch with this stuff. It'll greatly facilitate sharing code and being able to contribute your efforts back.

@MengeCrowdSim thx to the ground work you have done, this makes it possible for me to proceed with the approach (c).
Now, thanks the support, I am able to work with VS2015 x64, to gradually learn the deeper parts of the codes. I am trying to figure out how to "CLR Re-construct" the different classes e.g. FSM, SimulatorInterface, Goal, State, etc. created after passing the two XML files (Scene and Behaviour)
I need advice, how to
(1) extract the individual agent's goal; how to define new goal; new guide path from present destination to the next destination, how to raise event, how to convert this event to CLR call back etc. This is a slow process as I learning the code little by little.
(2) This is like creating custom Behaviour FSM XML file on the fly and individualized to each agent. Can this be done?
(3) How to create the BFSM attributes in C#, through the CLR wrapper to create the BSFM without the XML file.
Need to slowly figure out this.
=> Thanks. Great work! This may take a few months, at least now I see it is more feasible than before.

Before you start down this path, I think it's worth level setting.
Everything in Menge's architecture and design is biased towards consuming specifications, instantiating simulators, and running them. It has not been streamlined to programmatically construct simulators. And, more particularly, hasn't been designed to reconfigure the simulator during simulation.
  What do you hope to do that can't be done by constructing the appropriate XML files? (I can even imagine constructing such an XML file dynamically/programmatically in memory and instantiating a simulator on that.)
So, perhaps we want to take a step back and discuss what you want to do and how best to go about achieving that. It may well be that what you want to do is 100% captured in XML and you don't know it yet.
I'd suggest we put this github issue on hold and have this discussion via email until we have a clear objective. Email at<EMAIL_ADDRESS>

@MengeCrowdSim Definitely not to re-invent the wheel. My impression is that the input is ONE direction. XML files => Simulation for t time => Display. What I hope to achieve, while Simulation for t time, click randomly on one of the agents, the agent re-evaluate the new information provided through the clicking, based on simple rules, decide one out of a list of available goals, define the (re-calculated) new guide path, move to the new destination NOT PRE-DEFINED by the XML files.  Can this be done through the XML files?

@MengeCrowdSim there are still issues of not being able to compile:
     if (pluginPath != 0x0) {
	//engine.loadPlugins(pluginPath); // <== Can not be compiled [not sure why?)
 }

To AddAgent, one needs AgentInitializer????
newly created AgentInitializer  or AgentInitializer  that has been populated with information parsed from xml?

In the code snippet, I need you to give me a more specific reference of where the bad code snippet is. If I have to go searching, it'll take me a lot longer to provide insight.

re: AgentInitializer. The AgentInitializer is responsible for generating agents from XML. It's not strictly required so long as the work done by the AgentInitializer is performed otherwise.

Use case: I picked up the following things from what you've written:
a. You want user interaction (a user clicks on some visualization of an agent to initiate an event).
b. The result of the clicking event causes the agent to select a new goal from an already known set of goals -- this is the "list of available goals" -- and then the agent proceeds toward the goal.

So, this is very doable in the current architecture.
We are definitively missing the event trigger you would need to invoke from your UI program. The only thing we're missing is an event trigger. (It's what your UI program would invoke when you've clicked on an agent.) We might also have to implement some event "targets" and "effects". But that's not a problem.
The goal changing is easy. Depending on whether the candidate goals are known before the simulation starts or not (it's not clear to me from your description), and what the "simple rules" are for picking a new goal, you can use out-of-the box functionality. Worst case, we embed your "simple rules" in a custom GoalSelector implementation in a plug-in.
So, based on what you've described, I believe the correct way to solve this is via XML (with the possibility of some plug-ins).

As per my previous comment, I've just pushed some additions to Menge (#57).  This handles the event side of what I discussed above.

@MengeCrowdSim Would you please share the GUI tools so we can take a look at them, specially the one that converts obj file to nav mesh.
Thank you.

The GUI tools aren't official. As such, rather than living under @MengeCrowdSim they live in my personal github.
https://github.com/curds01/MengeUtils
The tool you want, specifically, is objToNavMesh.py. It'll produce a .nav file you can use in Menge. It's slightly brittle; if your obj has non-manifold surfaces (e.g., more than two faces sharing an edge), or other strange artifacts, it may choke and the error messages are lacking. If you encounter these issues, let me know and I can fix it. It's really only documented on the command-line (but it's straightforward: obj mesh in, nav mesh out.) :)


AWS Fleet Provisioning support with ESP-IDF (IDFGH-6559)

Hi,
I am trying to setup ESP-IDF to do AWS Fleet Provisioning using CSR, I don't find any source or support for this on ESP32. Can you help me in finding it for me? What is the procedure to set this up and the complete process.
Even if you have a beta version also should be fine for me.
Thanks,
GR.Venkatesh

@venkateshgr-ayla Thanks for reporting, this is a duplicate of https://github.com/espressif/esp-aws-iot/issues/85, will close this one and let us track in https://github.com/espressif/esp-aws-iot/issues/85.


Improved formatting & imports for Python's oso and sqlalchemy-oso

I messed up the previous pull request and decided to try again a new one - sorry about that. This one should be better and hopefully pass the checks.
Original PR:
I've decided to try and improve the formatting and its consistency somewhat, as well as clean up the namespaces a bit. It's still not perfect, and I've probably missed a couple issues, but I think it's way better like this than it was before. I've also added __all__ variables to a couple __init__.py files.
The decision to not indent multi-line strings may be a bit controversial, as well as some other things. I can change those if necessary.
Please let me know what you think.

Oh, are we using SQLAlchemy 1.3 for oso/tests/polar_roles_sqlalchemy_helpers.py? The requirements.txt say it's 1.4, though.
declared_attr should be in orm there.

Hey @tomashozman!
Thanks for opening this PR!
Did you use any tools to generate these formatting changes? It could be tough for us to review a ~1k line diff line-by-line. It would be a lot easier if we could first commit any auto-formatting changes first.

Hello!
The docstring / comment formatting was manual, as well as the slight namespace changes. The rest was isort and auto-formatting it with Black, just in case I've missed anything.
Also, speaking of missing things...

I'm not sure what's wrong with the SQLAlchemy import near the end of Python's check. It's probably a rookie mistake, perhaps somebody else can figure it out.

Hey @tomashozman.
We really appreciate the effort you put into this, but at this time we aren't able to review large PRs which have mostly formatting changing. We rely on automatic formatting tools to handle those for us. We would consider a PR which added some additional tooling like isort to clean up imports.
Alternatively, you could look at some of the good first issues we have listed.
Again, many thanks for the effort in this contribution, and sorry we couldn't help you get this merged.


Syntax for documenting credentials with a specific value inside the porter.yaml

credentials:

name: CDP_CRED
value: ""

Credentials file:
{
"name": "CFS",
"created": "2020-07-15T09:08:45.72513-05:00",
"modified": "2020-07-15T09:08:45.72513-05:00",
"credentials": [
{
"name": "CFS_LICENSE",
"source": {
"value": "asdfasdf"
}
]
}
Is it possible to make this work? When I do porter install I get the following: Error: invalid bundle: validation failed for credential "CFS_LICENSE": credential env or path must be supplied
It doesn't seem to make sense that "env or path must be supplied" when obviously you can generate your credentials file with source/value.
Any help very appreciated!

Hi @coreyperkins,
The error returned is around the credential definition in the Porter manifest itself, which represents the credential destination inside of the Docker image that executes the bundle, rather than the credential source which is provided by the CFS credential set shown above.
Can you provide the porter.yaml manifest for this bundle (or at least the relevant credentials: section)?  The CFS_LICENSE credential needs either env: TBD_ENV_VAR or path: /path/to/CFS_LICENSE_cred.  As an example, check out the credentials section in the Azure Wordpress example bundle.

@vdice I will send that as soon as I am back at my work station. I don't want to use an environment variable. I'd prefer to simply supply the credentials via the json file on Porter install. From what you are saying that doesn't sounds possible. If I must use a path to a file what would the contents of the file look like if this was only a single value, like a password?

Indeed, a destination of raw type 'value' could perhaps be handy.  It would be interesting to think about how that could work.
Porter at its core being an implementation of the CNAB Spec has the need to adhere to what the spec requires for credentials.  Hence, in this case, the requirement for either an env or a path value.  However, Porter also can drive updates to the spec, so with further thought, perhaps this is something we can look into.
In the meantime, indeed, all credentials, regardless of size, etc., need to be placed in the image that executes the bundle either in the form of an env var or a filepath.  Hope this helps!

@vdice I agree. A raw type 'value' would be quite handy and relatively straight-forward.
Here's the credentials section:
{"credentials":{"CFS_LICENSE":{"env":"CFS_LICENSE","required":true},"OIDC_AUTHORITY":{"env":"OIDC_AUTHORITY","required":true},"OIDC_CLIENTID":{"env":"OIDC_CLIENTID","required":true},"OIDC_CLIENTSECRET":{"env":"OIDC_CLIENTSECRET","required":true},"kubeconfig":{"path":"/root/.kube/config","required":true}}
I am having some minor questions to get passed to get my bundle functional.

Should the environment variables exist on the machine where the porter bundle is made? They do atm.
Should the environment variables exist on the machine where the porter bundle is installed? They do atm.

I have the credentials sections in the porter.yaml and in the bundle.json (above) when it is created.
I am also passing the following to the -c parameter on porter install
{     "name": "kubeconfig",     "created": "2020-07-14T11:47:00.452976144-05:00",     "modified": "2020-07-14T11:47:00.452976144-05:00",     "credentials": [         {             "name": "kubeconfig",             "source": {                 "path": "/home/$AdminUsername/.kube/config"             }         },                     {             "name": "OIDC_AUTHORITY",             "source": {                 "env": "OIDC_AUTHORITY"             }         },                     {             "name": "OIDC_CLIENTID",             "source": {                 "env": "OIDC_CLIENTID"             }         },                     {             "name": "OIDC_CLIENTSECRET",             "source": {                 "env": "OIDC_CLIENTSECRET"             }         },                     {             "name": "CFS_LICENSE",             "source": {                 "env": "CFS_LICENSE"             }         }     ] }
When you say the credentials must be placed in the image that executes the bundle (the invocation image) that makes me think the environment variables need to exist on both the machine where I do the porter build and on the machine where I do the porter install. Is that correct?
Thank you for your help!

I am having some minor questions to get passed to get my bundle functional.

Should the environment variables exist on the machine where the porter bundle is made? They do atm.
Should the environment variables exist on the machine where the porter bundle is installed? They do atm.

Good questions.  It again comes down to whether we're talking about the Credential Set which is supplied to Porter at install/action runtime and designates the sources of the credentials (either via the local user's env/filesystem or hosted in a cloud store, such as Azure Key Vault by way of the Azure plugin) or whether we're talking about the credential destination definitions in the Porter manifest (and resulting bundle.json file).
For the former Credential Set sources, they must exist on the machine where the porter bundle is installed.  For the latter credential destinations, Porter handles creation of the destination env var (or file path) after parsing the Credential Set given to it on install.  And so, for the question:

When you say the credentials must be placed in the image that executes the bundle (the invocation image) that makes me think the environment variables need to exist on both the machine where I do the porter build and on the machine where I do the porter install. Is that correct?

Porter handles placing the credential values in the image to the locations spec'd in the porter.yaml/bundle.json (e.g. env or path) after resolving each credential value from the supplied Credential Set.
The credential sources needn't exist on the user's machine where the bundle is being built, as they aren't injected at buildtime; rather, they are only injected at runtime (during porter install/upgrade/uninstall/etc.).

Great, I hope this helps you move forward!
Giving the raw value type for a credential destination some thought and I encountered a couple considerations.  Maybe you can help bounce some ideas around.

It's unclear how to supply the raw value to the runtime execution environment (classically this is a Docker container via Porter's (and CNAB's) default driver of Docker, but could be VM, Kubernetes, etc.).  How would the scripts/commands inside of the container access this value if not in the form of an env var or file?  (If you're curious, here's the go code where we setup the env and tar up the files prior to container start/run.)

Consider what the porter.yaml/bundle.json ends up looking like with credential destinations of value -- the value itself would be printed and leaked, right?  Presumably all credential values are sensitive, so this wouldn't be ideal.

@vdice ... those are good considerations indeed. I would certainly not want to create a security concern. What if an encryption step was implemented on both ends to ensure the values could only be retrieved by the machine running the install?

Another angle to simplify/improve UX would be a Porter-specific piece of functionality wherein it would be okay for credential(s) in the Porter manifest to not specify either env or path.  The value would then only be useful in the context of the manifest itself, via {{ bundle.credentials.mycred }} in install/upgrade/uninstall/etc. steps, and would either not be injected into the container at all (maybe behind the scenes Porter can just say path: /dev/null to appease the spec) or choose some default to inject it in the container (maybe env: CREDNAME).  Like I say, for creds that are only really intended to be used in the manifest via templating and not accessed via scripts/programs copied the container itself, perhaps this makes sense.
Users could would still supply env or path if they need to also access cred(s) via the application(s) running in the container itself, of course.

@coreyperkins I was hoping to follow up with you on this question

What if an encryption step was implemented on both ends to ensure the values could only be retrieved by the machine running the install?

With plugins you can do that today, for example we have plugins to let you retrieve encrypted credentials from Azure Key Vault and Hashicorp Vault. Here's what that looks like:

You configure porter to use the plugin in ~/.porter/config.toml. For example:

default-secrets = "carolynvs"

[[secrets]]
  name = "carolynvs"
  plugin = "azure.keyvault"

  [secrets.config]
    vault = "carolynvs"

When you define a credential set, you can chose to lookup a credential from "secret", as opposed to "env" or some other location on the local host.
Now when you execute a bundle, porter will use the plugin to just-in-time to retrieve the encrypted secret from Key Vault or Vault, etc.
The decrypted value is injected into the bundle's executing docker container as either an environment variable or a file. The bundle is able to use the secret (either through porter's templating or by accessing the env var/file).
Porter removes the container when the bundle finishes executing.

Would that fit your requirements?

@carolynvs Thank you so much for the response, sorry for my delay. I wasn't aware that the secrets plugin could also be used with hashicorp vault in addition to Azure, that is interesting. I do believe this would meeting my requirement and still keep the secret "stuff" secret. Thank you!

Yup! @dev-drprasad wrote a plugin to integrate Vault. You can find more info at https://github.com/dev-drprasad/porter-hashicorp-plugins


[review] Klocwork fix PFW 3.1.0

[ ]   File: test/test-subsystem/TESTSubsystemString.cpp:L49-55
 You do not realy nead those std:: but it is more a matter of style.
[ ]   General Comment
 plz review @krocard

This pull request is now managed with CodeReviewHub. Refresh.

plz review @krocard

:shipit:


Warning when symbols are included indirectly

As below, b.h references class A in a.h, but b.h doesn't include a.h directly but indirectly.
clangd will show a warning for class A in b.h, for it doesn't know class A.
a.h
class A
{
    ...
}

b.h
void test(const A & a);

b.cpp
#include "a.h"
#include "b.h"

void test(const A & a)
{
   ...
}

I think direct includes as clangd does is more robust, but it incurrs more needless compilation when header files change.
If using extra declares as below, there might be too much declares like:
class A;
class B;
class C;

Still not good enough.
It is not bad if clangd knows indirect inlcudes.
Is there an option for this?

This is https://github.com/clangd/clangd/issues/45. Please see the discussion there for why it's not easy to support.


Guide about custom 404 pages

Communicate to the users that adding a 404.html in their root directory of the docs, will make Read the Docs to find it and serve it when a 404 happens
Also, suggest the usage of the sphinx-notfound-page extension.

I accepted all the suggestions. Feel free to merge if you consider.

@stsewd @ericholscher are we happy merging this at this point?


fix: fix query failure on simultaneous execution of multiple queries

Description

The default connection that Redshift driver provides is not thread safe. This leads to query failure on simultaneous execution of multiple queries. This PR replaces the connection object by a connection pool.
Update the Redshift driver version.

Fixes #15333
Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Manual
JUnit TC
Not adding a cypress test because AFAIK, we don't have a Redshift DB instance in our CI test setup.

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

/ok-to-test sha=9068685

/ok-to-test sha=0861854

/ok-to-test sha=cc96748

Not adding cypress test since we don't have Redshift test DB in CI. cc: @prapullac

Merging in since tests have passed but status has not been updated.


test(python): add a 3rd nexus to an existing multipath device

Create 2 nexuses each with 2 replicas, connect to both nexuses over
NVMf, run fio against the block device. While fio is running, create
a 3rd nexus with the same 2 replicas and connect to it over NVMf.
Verify that the existing block device now has 3 paths.
Fixes CAS-975

bors try

bors try

bors try-

bors try

bors try

bors try

bors try

bors merge


Non-whitespace before first tag

Hello everyone, maybe someone has encountered such a problem
I use the latest version along with Nuxt 3 and when building, I constantly get the error "Non-whitespace before first tag"

I also ran into a problem

怎么解决，我现在也出现了这个问题

I have the same problem

I am encountering the same problem stack trace looks like this:

Same problem here. Running on macOS Sonoma with Node 20.

i am having the same problem, apparently this happens on macOS when a .DS_Store file happens to pop up in the svg directory. as it seems since Mojave you won't get .DS_Store to show even if you press "cmd + shift + ."
The module should filter out all non-svgs when parsing them.
Hotfix: sudo find ./ -name ".DS_Store" -depth -exec rm {} \; from within the svg folder


Deploy tool: CRIO targets file names with spaces do not transmit

File names with spaces do not transmit with new SSH transfer methods, showing the above error message. Using old webdav method file names with spaces did work.

Suspected cause: the command line call needs to escape the file name with quotes.
No test case was done with a file containing a space.  Need to add one.

Acceptance Criteria:

Files with spaces deploy successfully
Manually test

:( This is broken again. I need to make sure this is added to the manual deploy tests.


Pad the underlying fields when comparing StringTypes

While I'm not super savvy with the related crypto, I tracked the issue behind #986 down to the difference in the number of underlying fields when comparing strings of different sizes; I tried right-padding the shorter list of fields with the zero field for comparison purposes, and it seems to fix the issue.
Fixes #986.

As for picking Field::zero as the padding value (i.e. if it could inadvertently cause the comparison of strings of different lengths to result in true), I haven't been able to hit that value when testing different string values, and the Field::from_bits_le impl (specifically the mixing with the coefficient) suggests that it's impossible that it would ever be one of the underlying fields.

@Pratyush we should discuss the approach to fix the StringType, so we can resolve the ongoing CI failure.
The issue with this PR's approach is that padding leads to ambiguity in the length, causing false-positives (string match collisions). I believe we should employ the terminus bit approach by appending a 1 at the end of the bitstring.
Wdyt?

Yes, thanks for flagging. Closing PR now.


Wechaty Gateway Server: Assertion failed

1. Versions

What is your wechaty version?
Answer: 0.4.14

Which puppet are you using for wechaty? (padchat/puppeteer/padpro/...)
Answer: padplus_token (converted to donut token)
I got one donut token from the official merchant, but it is invalid after I switched the account. No new qrcode is forwarded, and my puppet hangs. (It might be another bug, but I am now hosting my own gateway server.)

What is your wechaty-puppet-XXX（padchat/puppeteer/） version?
Answer: wechaty-puppet-hostie 0.2.2

What is your node version? (run node --version)
Answer: I am using docker image, so...

What os are you using
Answer: Centos7 for gateway server, MacOS for dev

2. Describe the bug
This bug has been reported multiple times in the issues, but I did not see any reaction from the developers. I am reporting again to draw more attention and hopefully, I could get some suggestions from the official developers.
I have tried to update all the libraries to the latest version including the gateway docker image.
3. To Reproduce
https://github.com/wechaty/wechaty/issues/1988#issuecomment-635915025
Exactly the same steps here.
4. Expected behavior
Gateway server should forward the messages to puppet.
5. Actual behavior
The puppet side cannot get any message and it keeps reporting the same error as the gateway server.
6. Full Output Logs
VERB PuppetServiceImpl messagePayload()
00:04:58 VERB Puppet messagePayload(9144104376834225040)
Error [AssertionError]: Assertion failed
at new goog.asserts.AssertionError (/wechaty/node_modules/google-protobuf/google-protobuf.js:81:876)
at Object.goog.asserts.doAssertFailure_ (/wechaty/node_modules/google-protobuf/google-protobuf.js:82:257)
at Object.goog.asserts.assert [as assert] (/wechaty/node_modules/google-protobuf/google-protobuf.js:83:83)
at jspb.BinaryEncoder.writeUnsignedVarint64 (/wechaty/node_modules/google-protobuf/google-protobuf.js:458:77)
at jspb.BinaryWriter.writeUnsignedVarint64_ (/wechaty/node_modules/google-protobuf/google-protobuf.js:480:507)
at jspb.BinaryWriter.writeUint64 (/wechaty/node_modules/google-protobuf/google-protobuf.js:485:471)
at Function.proto.wechaty.puppet.MessagePayloadResponse.serializeBinaryToWriter (/wechaty/node_modules/@chatie/grpc/dist/generated/wechaty/puppet/message_pb.js:849:12)
at proto.wechaty.puppet.MessagePayloadResponse.serializeBinary (/wechaty/node_modules/@chatie/grpc/dist/generated/wechaty/puppet/message_pb.js:812:47)
at serialize_wechaty_puppet_MessagePayloadResponse (/wechaty/node_modules/@chatie/grpc/dist/generated/wechaty/puppet_grpc_pb.js:460:26)
at sendUnaryResponse (/wechaty/node_modules/grpc/src/server.js:98:15)
at sendUnaryData (/wechaty/node_modules/grpc/src/server.js:600:9)
at Object. (/wechaty/node_modules/wechaty-puppet-hostie/src/server/puppet-implementation.ts:529:16)
at Generator.next ()
at fulfilled (/wechaty/node_modules/wechaty-puppet-hostie/dist/src/server/puppet-implementation.js:5:58)
at processTicksAndRejections (internal/process/task_queues.js:97:5) {
reportErrorToServer: true,
messagePattern: 'Assertion failed',
code: 13
7. Additional context
I got this error every time when the gateway server receives one message.
[bug]

@inkpool you can try
docker run -e "WECHATY_PUPPET_PADPLUS_TOKEN=puppet_padplus_xxxxxxxxxxx"
    docker.pkg.github.com/wechaty/scala-wechaty/ding-dong-bot:0.0.16-SNAPSHOT


[WIP] Fix OpenHandler for all-verb routes

This fixes the detection for route entries without verb

Hey, this was marked as draft, do you think it'll work as is?

Hi,
sorry I had this draft in your repo that long x.x
IIRC I worked on adding something, but don't remember what it was .. the changes currently in here are probably fine


feat(ux/ui): implement toaster,support noIcon property, error toast dismissal, mobile on close

Alaska Airlines Pull Request
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes: #19, #9, #22, #26
Summary:
Created a toaster container to hold unlimited number of toasts.
Added noIcon property to allow the icon to be remove from toast UI.
Error toast must be dismissed by user - do not apply auto dismissal logic to error toasts.
For screen sizes < 767, allow the user to dismiss the toast by clicking anywhere on the toast.
Type of change:

[x] New capability

Checklist:

[x] My update follows the CONTRIBUTING guidelines of this project
[x] I have performed a self-review of my own update

By submitting this Pull Request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Pull Requests will be evaluated by their quality of update and whether it is consistent with the goals and values of this project. Any submission is to be considered a conversation between the submitter and the maintainers of this project and may require changes to your submission.
Thank you for your submission!
-- Auro Design System Team

fixed!


Colors!

Opening this PR so we can start a discussion about it if anybody wants to. We're getting to the point where we have enough arguments that we might want to think about a pipe-centric API versus loads of arguments to the same function, but the latter is how it's implemented now. This also has only been fully developed for lm models at this point but I'll make the minor updates needed to the functions for the other models soon and push them to this same PR before merging.
Here's a basic example:

For some reason I can't get it to work with texPreview::tex_preview() so the reprex is broken, but it renders fine for me for HTML and PDF.
The API
Currently, you can change the color of any variable name, coefficient (i.e., anything in greek) and the subscripts.

Colors must either be an HTML hex or a color recognized by LaTeX or standard R Markdown
Greek colors and subscript colors

You can pass a single color or a vector of colors. This is maybe a little less than ideal because if you wanted to change the color of just one coefficient, you would have to pass a vector that is something like c(rep("black", 4), "cyan", rep("black", 2)), and you'd have to make sure the one you were highlighting was in the correct spot. But it's also pretty easy to make them all different colors, as I've shown above (although some of those specific color choices are less than ideal).

Variable color

This is just like swap_names() where you pass a named vector with the names corresponding to the variable and the elements corresponding to the color.

Variable subscript colors

Same API as variable colors. One decision I made here though was to have all subscripts for a given variable be the same. We could allow this to vary too but that felt like too much work for now.

Alright, I think this is basically ready to go, including tests for each model type. You can take a look at the attached zip file (linked below) that has the HTML R Markdown output, if you'd like. These are the models I included in the tests. The only thing is I may still put a quick vignette together on this before merging.
A few caveats

merMods only colorize variable names and variable subscripts. I think eventually we'll want to support the greek and greek subscripts too but that was overly difficult for now so I decided to hold off, particularly because I think if we refactor to a layered approach the functions will have to be substantially rewritten anyway. I included a warning if those arguments are supplied to merMods.
I did not implement anything for the time-series models. @JaySumners, do you want to check-out this branch and add that? Or should we go without? As I've mentioned in the past, I am really unfamiliar with time-series models so I'm not sure how to best approach it.

testing results

We can go without ARIMA at the moment. I would probably color any linear portion like you are and then color AR, MA, SAR, and SMA portions separate or something like that.

Merging now!


fix: catalog source image

What type of PR is this?
/kind bug
What does this PR do / why we need it:
Fixes the incorrect catalog image that points to an invalid argocd-operator image
Have you updated the necessary documentation?

[ ] Documentation update is required by this PR.
[ ] Documentation has been updated.

Which issue(s) this PR fixes:
Fixes #?
How to test changes / Special notes to the reviewer:
Didn't rebuild the bundle, as the current bundle is updated for 0.13 (next) release.
Checked manual installation and works well.

Should we have a test in the CI to check for olm installation of the operator? if that is possible to add @varshab1210 - these are the steps - https://argocd-operator.readthedocs.io/en/latest/install/olm/

That's a great idea. Once we automate the release, we could set up a post-release job to verify the installations.


Use .mjs extension for ESM build.

Build tools that use native ESM (e.g. Vite) expect ESM builds to use the .mjs extension (unless the nearest package.json has "type": "module", in which case CJS builds should use the .cjs extension).
See: https://nodejs.org/api/packages.html#determining-module-system
Currently, both the ESM and CJS builds use .js -- this causes Node to assume they are both CJS:

Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.

Would you be willing to consider either adding "type": "module" to your package.json file, or renaming the built packages to e.g. color.esm.mjs and color.cjs.js?

Thank you for pointing that out! I just pushed a fix that:

Renames CJS exports to use .cjs
Adds type: "module" to package.json
Makes color.js be the ESM version (IIFE is now color.iife.js)

Let me know if that fixes it!

I had to revert type: "module" because it broke eleventy :( Hopefully the .cjs extension fixes it though?

Nevermind, it was the eleventy config. I re-added type: "module". The last 2 messages will self-destruct in 3, 2, 1…

Let me know if that fixes it!

Works for me, testing locally! I noted a few minor issues:

It looks like the main and module paths in package.json need to be updated to the newer build.
You may run into trouble compiling the 11ty site with "type": "module" set. Looks like you already solved this 🎉
Similarly, I think postcss.config.js needs to be renamed to .cjs for npm run build:css to work.

Let me know if that fixes it!

Works for me, testing locally! I noted a few minor issues:

It looks like the main and module paths in package.json need to be updated to the newer build.
~You may run into trouble compiling the 11ty site with "type": "module" set.~ Looks like you already solved this 🎉
Similarly, I think postcss.config.js needs to be renamed to .cjs for npm run build:css to work.

I forgot to post yesterday, all of these should be fixed now!


Update client_allocation_request.md

Description
Update client_allocation_request.md
Is this PR related with an open issue?
Related to Issue #
Types of changes

[ ] Fixing existing documentation / content in repo
[ ] Adding new content to the repo
[ ] Adding yourself to the miner registry (miners.md)

Thanks - good catch @fabriziogianni7


added jsonb_insert

added the jsonb_insert function.
Having issues with coupling the output with the three arguments because the output is null if any of the inputs are null and the second argument is a text array whereas the first and second are jsonb.
I could make the output Nullable<Json> but that reduced ergonomics of the function.
I would appreciate some guidance on how to move forward, perhaps a new trait?
Once this is reolved I will add the other variant of this function

@weiznich


make yoga threadsafe

Continuing https://github.com/facebook/yoga/pull/791
@nokia6686 is a former member of our team, so we are trying to pick up what he left and carry out the pull request.
Solution
Improved from previous solution with @jpap's suggestions.

Using atomic for gNodeInstanceCount and gConfigInstanceCount

std::atomic<int32_t> gNodeInstanceCount(0);
std::atomic<int32_t> gConfigInstanceCount(0);

Passing gDepth and gCurrentGenerationCount (renamed to depth and generationCount respectively) between function calls that stem from YGNodeCalculateLayout.
In YGNodeCalculateLayout, pass depth as value 0, to indicate the root depth.
Using #define with gPrintChanges and gPrintSkips to optimized out unnecessary logging and incrementing depth state.

#define PRINT_CHANGES false
#define PRINT_SKIPS false

Prevent memory leak by removing static keyword in YGConfigGetDefault

@davidaurelio Sorry for delaying this for too long, I was a little too busy last month.
In the new update, I've:

reverted the atomics
reverted the macros. I'll switch to constexpr in another pull request
reverted the static mistake

So this pull request will only have 2 changes now:

using uint8_t for generationCount
passing depth and generationCount via function parameters, instead of using global variables.

any update, @davidaurelio ?

@davidaurelio @jpap hello?

Hey all - this is still flagging our thread-sanitizer runs. It looks bad for Yoga's usage at a certain big company. Could the team find some way to fix or silence the warning with annotations?

@davidaurelio it's been several months. Could you take a look at this?

Hi folks, sorry for being unresponsive.
The aspect of this creating too much noise in sanitizers is new to me, sorry about that. I thought this was mainly due to concerns about the data races themselves.
Let me see how well this merges …


Contributor Console Integration (v2)

Summary
This issue will cover the time and effort for integrating the Contributor Console.
Background
For the most part, the ICLA flow is working for the new v2 Contributor Console. This task is to work through the integration challenges between the v2 Contributor Console and the API.
User Story
As a developer, I want to use the v2 Contributor Console for CCLAs.
Assumptions
None
Questions
None
Acceptance Criteria
The "done" criteria when this feature or problem is resolved. Such as:

Unit Tests added and running in CI
Functional Tests updated to cover feature, if applicable
Demonstrate the set of capabilities to the product team while the code is running in the STAGING environment.

References

v2 Contributor Console PRD

Closed - duplicate. of #1214


AttributeError: 'YellowBus' object has no attribute 'CATEGORY'

Hey me again!
Getting this when running "save as script"
File "/root/custom_nodes/ComfyUI-SaveAsScript/__init__.py", line 46, in save_as_script     ComfyUItoPython(workflow=workflow, output_file=sio)   File "/root/custom_nodes/ComfyUI-SaveAsScript/comfyui_to_python.py", line 530, in __init__     self.execute()   File "/root/custom_nodes/ComfyUI-SaveAsScript/comfyui_to_python.py", line 553, in execute     load_order = load_order_determiner.determine_load_order()   File "/root/custom_nodes/ComfyUI-SaveAsScript/comfyui_to_python.py", line 114, in determine_load_order     self._load_special_functions_first()   File "/root/custom_nodes/ComfyUI-SaveAsScript/comfyui_to_python.py", line 152, in _load_special_functions_first     if (class_def.CATEGORY == 'loaders' or AttributeError: 'YellowBus' object has no attribute 'CATEGORY'

Need some more information on this one. Sorry, I've been on a hiatus.


Add schema imports; remove track decorators

Thank you for your first PR @jbleyleSF .
As it written on your PR, I don’t  even need @track here, since there's no update to the variable.
I still newbie in LWC, and need to learn more about it.
Once again thank you, really appreciate it.

@arufian happy to help! @track was changed in 224. It's a new change so not everyone knows about it.
https://releasenotes.docs.salesforce.com/en-us/spring20/release-notes/rn_lwc_track.htm


Paste via ctrl+v have some problems [DESKTOP]

Is there an existing issue for this?

[X] I have searched the existing issues

Flutter Quill version
flutter_quill: ^9.2.6
Steps to reproduce

Just try to paste the text inside QUilEditor with Ctrl+V

Expected results
- First Expectation
The text should be pasted where I'm focusing on the editor for example in the 3rd line
- Second Expectation:
the QuillController .changes.listen should trigger an event
Actual results
1 Expectation :
The text pasted in the end of lines of the texts
2 Expectation :
the event listener is not being triggered
Code sample

  // Controller
  QuillController _controller = QuillController.basic();

  // Listener not veing triggred on Ctrl+V
    docListener = _controller.changes.listen((event) {
      logger.t('${_controller.document.toDelta().toJson()}');
      $collection.setContent(
        widget.note.id,
        jsonEncode(
          _controller.document.toDelta().toJson(),
        ),
      );
    });

  // Widget
    QuillToolbar.simple(
                configurations: QuillSimpleToolbarConfigurations(
                  controller: _controller,
                  showAlignmentButtons: true,
                  showRedo: false,
                  showUndo: false,
                  showSubscript: false,
                  showSuperscript: false,
                  showIndent: false,
                  color: $styles.colors.accent2,
                  embedButtons: [
                    ...FlutterQuillEmbeds.toolbarButtons(
                      videoButtonOptions: null,
                    ),
                  ],
                ),
              ),

Screenshots or Video
No response
Logs
In the previous version I was using  flutter_quill: ^9.2.2 and I didn't face these problems

Does the text has styling? if yes then this is known issue and I already added a todo for it

Yes I tried it from the search bar as a simple text it works but when I tried to copy from the website or vs-code not working

Fixed


Make Codecov Upload Reports in Any Case

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.61%. Comparing base (5625e56) to head (90d803c).
Report is 2 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #239      +/-   ##
============================================
- Coverage     94.97%   94.61%   -0.37%
- Complexity      271      361      +90
============================================
  Files            80       92      +12
  Lines          1135     1393     +258
  Branches         23       30       +7
============================================
+ Hits           1078     1318     +240
- Misses           43       57      +14
- Partials         14       18       +4

Flag
Coverage Δ

?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 13 files with indirect coverage changes


fix: release workflow

:tada: This PR is included in version 1.5.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


I cannot find TFS extension in Visual Studio Community

Is TFS extension available in Visual Studio Community latest version?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 87e2c464-3537-278a-da5a-5301af0fb6d5
Version Independent ID: 29cb95b8-5cc2-e816-2d2e-ea6945ae7a5d
Content: TF Version Control - Visual Studio for Mac
Content Source: mac/tf-version-control.md
Product: visual-studio-mac
GitHub Login: @asb3993
Microsoft Alias: amburns

Hi @kuzun9x, The TFS extension is available in all versions of Visual Studio for Mac. You'll have to install the extension through the extensions gallery via Visual Studio > Extensions...  as described here: https://docs.microsoft.com/visualstudio/mac/tf-version-control#installation

Yup, I have seen and installed this plugin.
Some day ago, i cannot find it in market.
Thanks,

Hi @kuzun9x,
I'm not sure what it can be. I can see the addin with VS Community. Can you try to refresh the repositories, restart Visual Studio and try again?

Regards.

The installation is fine. But after I downloaded the projects from TFVC, I'll get Load Failed: File Not Found error when trying to open the solution. The path in the error is the correct path.

@Wowotoe This area is just for issues with the documentation. This sounds like a bug in the product, so I'd recommend creating an issue on Developer Community


VSCodium on Ubuntu XFCE needs --privileged

Thanks for making this. You've saved me so many hours. Sending you a heads up on an issue I hit with my use-case. I don't fully understand why, but I thought you might want to know.
When I start the ubuntu-xfce and install VSCodium like this:
wget -qO - https://gitlab.com/paulcarroty/vscodium-deb-rpm-repo/raw/master/pub.gpg     | gpg --dearmor     | sudo dd of=/usr/share/keyrings/vscodium-archive-keyring.gpg
echo 'deb [ signed-by=/usr/share/keyrings/vscodium-archive-keyring.gpg ] https://download.vscodium.com/debs vscodium main'     | sudo tee /etc/apt/sources.list.d/vscodium.list
sudo apt update && sudo apt install codium

VSCodium(codium) never starts.
Use codium --verbose to show the error.
$ codium --verbose
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted

I found the answer here: https://github.com/jessfraz/dockerfiles/issues/350. Add --privileged to the Docker command. So:
docker run --privileged -it -p 33890:3389 danchitnis/xrdp:ubuntu-xfce $USERNAME $PASSWORD $SUDOBOOL

Works.
Again, thanks for creating this project. It's a huge help to me as I try to get consistent environments for learners.

Happy that it did help you. The issue is related to chrome's engine and its sandbox concept. There are some solutions here. Make sure that you are aware of the security risks when running docker in --privileged mode.


Error running openmp samples host_function and reduction..

Hi,
first test of aomp..
I'm running AOMP 11.5 on Ubuntu 19.10 on kernel 5.6+ ROCM 3.3..
running make I get:
Makefile:28: AOMP not found at /home/cosmic/rocm/aomp
/usr/lib/aomp/bin/clang -O3 -target x86_64-pc-linux-gnu -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx900    host_function.c -o host_function
but running I get:
./host_function
Testing myfun execution as a function pointer
 This is myfun writing to stderr
Testing myfun execution from device using hostrpc_fptr0
[GPU Memory Error] Addr: 0x7f3889e0c000 Reason: Page not present or supervisor privilege.
Memory access fault by GPU node-1 (Agent handle: 0x2341620) on address 0x7f3889e0c000. Reason: Page not present or supervisor privilege.

I tried running under root privileges but output is the same..
with reduction sample I get:
./reduction
[/home/estewart/git/aomp11/amd-llvm-project/openmp/libomptarget/plugins/hsa/impl/data.cpp:129] atmi_malloc failed: HSA_STATUS_ERROR_INVALID_ARGUMENT
terminate called after throwing an instance of 'std::system_error'
  what():  Resource deadlock avoided
Abortado (`core' generado)

Also note fortran sample simple_offload I get:
a(1)=    11.00000         a(2)=    22.00000
Success: if a diagnostic line starting with DEVID was output
maybe remains of older compiler as I do not get a DEVID line..

The first warning looks to be from the AOMP environment variable not being set to /usr/lib/aomp. ROCm is said to only support Ubuntu 16 and 18 (https://rocmdocs.amd.com/en/latest/Current_Release_Notes/Current-Release-Notes.html) so that very well may be the issue here. Have you tried on an older Ubuntu? What CPU/GPU are you working with? Looks like the gpu might be a gfx900?

please run lscpu and lsgpu and paste the output here.
by any chance  are you using a new AMD CPU+GPU (APU) laptop ?

If lsgpu does not work for you try
lspci | grep VGA

I may have dreamed up the command lsgpu, please run /opt/rocm/bin/rocminfo instead of lsgpu

Hi,
thanks for support..
@estewart08 I'm aware that ROCM doesn't support past 18.04.. hope ROCM can support Ubuntu 20.04 LTS in next few releases.. anyway I'm getting no major problems on ROCM (at least using the OCL ROCM drv)..
sorry for not giving more details.. CPU 6600k.. GPU: RX Vega 56 and I also have a NV Titan V on same PC installed.. also I forgot to say I was running the sample while a Nvidia Titan V was running the display using NV propietary driver..
rocminfo
ROCk module is loaded
cosmic is member of video group
=====================
HSA System Attributes
=====================
Runtime Version:         1.1
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:<PHONE_NUMBER>3709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE
System Endianness:       LITTLE

==========
HSA Agents
==========
*******
Agent 1
*******
  Name:                    Intel(R) Core(TM) i5-6600K CPU @ 3.50GHz
  Marketing Name:          Intel(R) Core(TM) i5-6600K CPU @ 3.50GHz
  Vendor Name:             CPU
  Feature:                 None specified
  Profile:                 FULL_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        0(0x0)
  Queue Min Size:          0(0x0)
  Queue Max Size:          0(0x0)
  Queue Type:              MULTI
  Node:                    0
  Device Type:             CPU
  Cache Info:
    L1:                      32768(0x8000) KB
  Chip ID:                 0(0x0)
  Cacheline Size:          64(0x40)
  Max Clock Freq. (MHz):   3900
  BDFID:                   0
  Internal Node ID:        0
  Compute Unit:            4
  SIMDs per CU:            0
  Shader Engines:          0
  Shader Arrs. per Eng.:   0
  WatchPts on Addr. Ranges:1
  Features:                None
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    32843864(0x1f52858) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Acessible by all:        TRUE
    Pool 2
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    32843864(0x1f52858) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Acessible by all:        TRUE
  ISA Info:
    N/A
*******
Agent 2
*******
  Name:                    gfx900
  Marketing Name:          Vega 10 XL/XT [Radeon RX Vega 56/64]
  Vendor Name:             AMD
  Feature:                 KERNEL_DISPATCH
  Profile:                 BASE_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        128(0x80)
  Queue Min Size:          4096(0x1000)
  Queue Max Size:          131072(0x20000)
  Queue Type:              MULTI
  Node:                    1
  Device Type:             GPU
  Cache Info:
    L1:                      16(0x10) KB
  Chip ID:                 26751(0x687f)
  Cacheline Size:          64(0x40)
  Max Clock Freq. (MHz):   1630
  BDFID:                   768
  Internal Node ID:        1
  Compute Unit:            56
  SIMDs per CU:            4
  Shader Engines:          4
  Shader Arrs. per Eng.:   1
  WatchPts on Addr. Ranges:4
  Features:                KERNEL_DISPATCH
  Fast F16 Operation:      FALSE
  Wavefront Size:          64(0x40)
  Workgroup Max Size:      1024(0x400)
  Workgroup Max Size per Dimension:
    x                        1024(0x400)
    y                        1024(0x400)
    z                        1024(0x400)
  Max Waves Per CU:        40(0x28)
  Max Work-item Per CU:    2560(0xa00)
  Grid Max Size:<PHONE_NUMBER>(0xffffffff)
  Grid Max Size per Dimension:
    x<PHONE_NUMBER>(0xffffffff)
    y<PHONE_NUMBER>(0xffffffff)
    z<PHONE_NUMBER>(0xffffffff)
  Max fbarriers/Workgrp:   32
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    8372224(0x7fc000) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Acessible by all:        FALSE
    Pool 2
      Segment:                 GROUP
      Size:                    64(0x40) KB
      Allocatable:             FALSE
      Alloc Granule:           0KB
      Alloc Alignment:         0KB
      Acessible by all:        FALSE
  ISA Info:
    ISA 1
      Name:                    amdgcn-amd-amdhsa--gfx900
      Machine Models:          HSA_MACHINE_MODEL_LARGE
      Profiles:                HSA_PROFILE_BASE
      Default Rounding Mode:   NEAR
      Default Rounding Mode:   NEAR
      Fast f16:                TRUE
      Workgroup Max Size:      1024(0x400)
      Workgroup Max Size per Dimension:
        x                        1024(0x400)
        y                        1024(0x400)
        z                        1024(0x400)
      Grid Max Size:<PHONE_NUMBER>(0xffffffff)
      Grid Max Size per Dimension:
        x<PHONE_NUMBER>(0xffffffff)
        y<PHONE_NUMBER>(0xffffffff)
        z<PHONE_NUMBER>(0xffffffff)
      FBarrier Max Size:       32
*** Done ***

lspci | grep VGA
03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XL/XT [Radeon RX Vega 56/64] (rev c1)
04:00.0 VGA compatible controller: NVIDIA Corporation GV100 [TITAN V] (rev a1)

Is this still failing with AOMP 13.0-2?

both pass for me with aomp 13.0-3 and released aomp_13.0-2
$ AOMP=~/rocm/aomp_13.0-3/ make run
./host_function
Testing myfun execution as a function pointer
This is myfun writing to stderr
Testing myfun execution from device using hostrpc_fptr0
This is myfun writing to stderr
This is myfun writing to stderr
This is myfun writing to stderr
This is myfun writing to stderr
This is myfun writing to stderr
This is myfun writing to stderr
This is myfun writing to stderr
This is myfun writing to stderr
This is myfun writing to stderr
This is myfun writing to stderr
Testing the host fallback of hostrpc_fptr0
This is myfun writing to stderr
Success
rlieberm@r4:~/git/aomp12/aomp/examples/openmp/host_function$ cd ../reduction/
rlieberm@r4:~/git/aomp12/aomp/examples/openmp/reduction$ AOMP=~/rocm/aomp_13.0-3/ make run
/home/rlieberm/rocm/aomp_13.0-3//bin/clang -O3 -target x86_64-pc-linux-gnu -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx906    reduction.c -o reduction
./reduction
The result is correct on target =<PHONE_NUMBER>00!
Success!


chore(deps): update dependency nuxt and @nuxt/kit to v3.2.3

📚 Description
Updates to Nuxt 3.2.3
📝 Checklist

[x] All commits follow the Conventional Commit format
[x] The PR's title follows the Conventional Commit format

:tada: This PR is included in version 5.0.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Checkpoint 1.3 backwards compatibility

This pull request introduces three main additions:

[x] 1.3 weights are now backwards compatible, meaning we can load checkpoints that had the .safetensors suffix incorrectly placed.
[x] The logic used to get the filenames of weights shards was moved from serialize.weights.load_weights to serialize.utils.get_path thanks to the addition of return_all_matches keyword.
[x] Addition of a test that verifies the capability to load weights when changing parallelism topology.

Hello,
Thank you for your PR.
As I understand it, you are verifying the checkpoint version. If it is 1.3, a different pattern will be used to retrieve the shards; otherwise, the existing method will continue. I suspect that this approach may not resolve all the issues, given that the checkpoint version 1.3 was only added to the main branch five days ago (see commit), whereas the bug was introduced a month and a half ago in this commit. Additionally, checking for version 1.2 seems problematic as it might disrupt the loading logic for all checkpoints before this commit.
Timeline:
Checkpoint version=1.2 -> bug introduced -> Checkpoint version=1.3
We have already merged a quick fix that utilized a script to modify the file name: PR #151. Could you build upon this solution? Perhaps making it automatic when this pattern is detected, instead of requiring users to run the script manually.
I really appreciate your contribution.

Thanks for the feedback. I implemented the suggested changes and tested it locally. Seems to be working ok. Let me know if you have any other comments.


Python dependency finder does not work for Python 3

Need to provide an alternate version of find_versions_by_attribute_template in source:src/dependency_finder/python.py
Imported from Bitbucket issue 81,  created by apdavison on 06-22-2011 at 16:14, last modified: 06-22-2011 at 17:19

fixed in r249
Trac comment by apdavison on 06-22-2011 at 17:19


New IPFS Client Maintainers Group

Hiya :)
In the IPFS org we’ve created a new group for maintainers of client libraries. This will make it easier for us to loop you into issues that may cause problems or necessitate changes in your client library.
If you could let us know the github usernames of all the active maintainers we can add them to the group and note them in the clients table.
Also, we have a great IPFS weekly newsletter. A few days before each edition goes out a new pull request is created in the repo. If you have any announcements related to your client library feed free to log a new issue in the repo or just comment on an upcoming newsletter pull request.

I've just updated the README to mention that I'm not maintaining this project any longer. If someone wants to maintain this, I'd be happy to transfer ownership.

I would be happy to be another maintainer.

@logicminds to clarify: do you still want to take over this project or will you be working on https://github.com/tbenett/ruby-ipfs-api instead?
Either way, I can link to the relevant repo / fork from this repo's README and, if necessary, transfer ownership of the gem on rubygems.

I would prefer to work on the ruby-ipfs-api as it appears to have more steam and offer more.  I would recommend adding the link.

Done. Let me know if you need anything else.


oai.islandora.burbank validation results: character encoding only

character encoding:

[x] identifier: Â shows up in the vernacular but Rikolti ignores it -- Â ""\u00c2"" appears in the identifier field for 19 records. These are not in the oai (see: https://burbankinfocus.org/oai2/request?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai:burbankinfocus.org:islandora_1965), but seem to pop up in the vernacular. Need to investigate
[x] format: Â ignored by Rikolti

Issues have been resolved based on review of latest 2023-12-21 validation report


Fix non_snake_case warning on generated identity functions

The Message derive macro currently generates Debug implementations that contain code that violates the non_snake_case linter rule^1. This PR changes the macro expansion to add an #[allow(...)] attribute for this rule.
Note: If there is consensus that the solution is OK, I'll fix any failing tests to make this PR ready to merge.

When will the new version with this change come out? Currently this makes error output a mess.


[SMALLFIX]Use static imports for standard test utilities

Use static imports for standard test utilities in /alluxio/core/common/src/test/java/alluxio/util/HFSUtilsTest.java

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3265/
Test PASSed.

alluxio-bot, merge this please


[eta-tour]: Bug - The current version of Eta is v0.7.0b2.

https://tour.eta-lang.org/start#3
Link the version to a/the github/doc page to make sure the version is always current.

@rolandtritsch Thanks for reporting. We could fix this in the coming version of Eta.

Outdated. Closing.


fix: device.setOrientation for iOS 16

Description

This pull request addresses the issue described here: https://github.com/wix/Detox/issues/3823

In this pull request, I have used the new iOS orientation APIs available from iOS 16 onwards, instead of the UIDevice orientation API's (which no longer work when building using Xcode 14 and running on iOS 16+).
Note: I need some help/advice on how to verify these changes locally. No matter how I run it doesn't seem to be pulling in my code changes. I've tried intentionally putting compilation errors in the code and the tests still build and run without issue. I've ran lerna bootstrap, following by yarn build:ios and yarn e2e:ios. The tests run but don't seem to be exercising any new code.

For features/enhancements:

[ ] I have added/updated the relevant references in the documentation files.

For API changes:

[ ] I have made the necessary changes in the types index file.

Note I believe the failing status checks are due to the fact that we are using iOS 16 APIs and therefore need to build using Xcode 14 (as opposed to 13).

@mrbrentkelly thanks for this - much appreciated! All the more so with the code seeming to have good potential. I hope @asafkorem could get around to taking a deeper look at it. In any case, Buildkite's failure is a bit obscure, and not very useful ATM. For what it's worth, I can stress out that AFAIK our CI agents do not sport xcode 14 just yet.

Sorry, I just noticed your note 🙂

Note: I need some help/advice on how to verify these changes locally. No matter how I run it doesn't seem to be pulling in my code changes. I've tried intentionally putting compilation errors in the code and the tests still build and run without issue. I've ran lerna bootstrap, following by yarn build:ios and yarn e2e:ios. The tests run but don't seem to be exercising any new code.

Try running detox rebuild-framework-cache

Our CI now has Xcode 14 (iOS 16 support), so we're finally able to continue with this PR 🙂

This PR would be super handy for us, it's been a real blocker since adopting iOS 16. Please can it be prioritised?

@AlexHayton @mrbrentkelly sorry for the delay, I'm releasing this now. @mrbrentkelly thanks for this great contribution!

Released as v20.2.2 🎉