Upgrade 4.22

Hi,
I know this is not particular a bug. But in 4.22 there are quiete a few changes in the UBT which are not yet in FastBuild. Some classes have changed completely. I will have a look into the code but it seems quite a bit of work which I am not sure I can handle on my own. Anybody worked through it so far?

I got some stuff fixed. Like the path that needs to be converted to a string. But some of the classes in the buildpipline have changed completely. So the sorting of actions do not work anymore. Unfortunately I am not that familiar with Fastbuild to get my head around that any time soon. Is there a better documentation that could point me in the right direction?

Hi @meiwen333
I've stumbled upon the same issue as you. For the path, I simply changed action.CommandPath.Contains() to action.CommandPath.FullName.Contains().
As far as I'm aware there are no other useful docs for this but the idea is pretty simple.
Fastbuild distributes the action but most actions require items produced by other actions as prerequisite. I'm looking for another way to get those ProducingActions as well.
The best source for me is the XGE executor (which is from Incredibuild) in UnrealBuildTool/Executors and the changes to their workflow between 4.21 and 4.22.
I'll let you know if I make any progress.

We got it to work with some small changes (one of which (trhe regex) is really hacky). You're welcome to try it. It worked for us but I can't guarantee it's 100% error-free
FASTBuild.zip

Hi @MrPatry ,
thanks for the update.
Did you also experience all the missing precrocessor macros?
Seems similar to this thread:
https://answers.unrealengine.com/questions/813258/updated-to-420-now-i-cant-build-error-c4668.html
Anyway thanks for posting the solution to the upgrade :)
Cheers Pascal

No we did not, sorry

We got it to work with some small changes (one of which (the regex) is really hacky). You're welcome to try it. It worked for us but I can't guarantee it's 100% error-free
(Update 12:17PM: New version attached, some error happened when compiling c files in LiveCoding)
FASTBuild.zip

Hi @MrPatry   I use your code in vs2017 but  error occured
error described that:
opening file 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Redist\MSVC\14.13.26020\x64\Microsoft.VC141.CRT\msvcp140.dll' in Compiler ToolManifest

Hi @MrPatry I use your code in vs2017 but error occured
error described that:
opening file 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Redist\MSVC\14.13.26020\x64\Microsoft.VC141.CRT\msvcp140.dll' in Compiler ToolManifest

Make sure you're using at least VS2017 15.9 and have .Net Framework 4.6.2 targetting pack installed

In my case I changed the path to the correct version:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Redist\MSVC\14.16.27012\x64\Microsoft.VC141.CRT
It is in line 617-618 or 624-625
if (VCEnv.Compiler == WindowsCompiler.VisualStudio2015)
				{
					AddText(string.Format("\t\t'{0}/VC/redist/x64/Microsoft.VC{1}.CRT/msvcp{2}.dll'\n", VCInstallDir.ToString(), platformVersionNumber, platformVersionNumber));
					AddText(string.Format("\t\t'{0}/VC/redist/x64/Microsoft.VC{1}.CRT/vccorlib{2}.dll'\n", VCInstallDir.ToString(), platformVersionNumber, platformVersionNumber));
				}
				else
				{
					//VS 2017 is really confusing in terms of version numbers and paths so these values might need to be modified depending on what version of the tool chain you
					// chose to install.
					AddText(string.Format("\t\t'{0}/VC/Redist/MSVC/14.13.26020/x64/Microsoft.VC141.CRT/msvcp{1}.dll'\n", VCInstallDir.ToString(), platformVersionNumber));
					AddText(string.Format("\t\t'{0}/VC/Redist/MSVC/14.13.26020/x64/Microsoft.VC141.CRT/vccorlib{1}.dll'\n", VCInstallDir.ToString(), platformVersionNumber));
				}

Just check which version you have and adjust the path.

Anyboday experiencing these errors:
C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared\intsafe.h(66): error C4668: '__midl' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif'

Thx @MrPatry ! My own fix for the command-line parsing issue was a lot less clean/more hacky.. The other small changes you made were what was missing on my end. I'm just hitting one small issue:
just before the build completely finishes, this happens:
1>5> -Build: 351 ms	D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-DerivedDataCache.dll
1>9> -Build: 1158 ms	D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-RHI.dll
1>11>-Build: 381 ms	D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-Json.dll
1>12>-Build: 638 ms	D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-ApplicationCore.dll
1>8> -Build: 535 ms	D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-Projects.dll
1>7> -Build: 451 ms	D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-ShaderCompilerCommon.dll
1>10>-Build: 423 ms	D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-InputCore.dll
1>1> -Build: 1494 ms	D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-VulkanShaderFormat.dll
1>3> -Build: 3792 ms	D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-Core.dll
1>2> -Build: 3429 ms	D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-Slate.dll
1>4> -Build: 4265 ms	D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-CoreUObject.dll
1>6> -Build: 113871 ms	D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\UE4Editor-Engine.dll
1>Alias: all
1>Failed due to missing file: D:\GitHub\UE4.22\UnrealEngine\Engine\Source\Action_1744
1>Saving DepGraph 'D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\fbuild.fdb'
1>Saving DepGraph Complete in 0.068s

I looked a further into this issue and saw that this actions are not explicitly created in the bff file, just it's command line argument is in there:
// "D:\GitHub\UE4.22\UnrealEngine\Engine\Build\Windows\cl-filter\cl-filter.exe" D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\UE4Editor\Development\LiveCoding\xxhash.c.txt -- "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\cl.exe"  @"D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\UE4Editor\Development\LiveCoding\xxhash.c.obj.response" /showIncludes
ObjectList('Action_1743')
{
	.Compiler = 'UE4Compiler'
	.CompilerInputFiles = "D:\GitHub\UE4.22\UnrealEngine\Engine\Source\Developer\Windows\LiveCoding\Private\External\xxhash.c"
	.CompilerOutputPath = "D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\UE4Editor\Development\LiveCoding"
	.CompilerOptions = '/Zc:inline /nologo /Oi /c /Gw /Gy /Zm850 /wd4819 /D_CRT_STDIO_LEGACY_WIDE_SPECIFIERS=1 /D_SILENCE_STDEXT_HASH_DEPRECATION_WARNINGS=1 /D_WINDLL /D_DISABLE_EXTENDED_ALIGNED_STORAGE /source-charset:utf-8 /execution-charset:utf-8 /Ox /Ot /GF /Ob2 /errorReport:prompt /EHsc /Z7 /MD /bigobj /fp:fast /Zo /Zp8 /wd4463 /wd4838 /I . /I Developer\Windows\LiveCoding\Private /I Runtime /I Runtime\Core\Public /I ..\Intermediate\Build\Win64\UE4Editor\Inc\CoreUObject /I Runtime\CoreUObject\Public /I Developer /I Developer\Settings\Public /I ..\Intermediate\Build\Win64\UE4Editor\Inc\LiveCoding /I Developer\Windows /I Developer\Windows\LiveCoding\Public /I "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\INCLUDE" /I "C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" /I "C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" /I "C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" /I "C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" /I "C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" /DIS_PROGRAM=0 /DUE_EDITOR=1 /DENABLE_PGO_PROFILE=0 /DUSE_VORBIS_FOR_STREAMING=1 /DWITH_DEV_AUTOMATION_TESTS=1 /DWITH_PERF_AUTOMATION_TESTS=1 /DUNICODE /D_UNICODE /D__UNREAL__ /DIS_MONOLITHIC=0 /DWITH_ENGINE=1 /DWITH_UNREAL_DEVELOPER_TOOLS=1 /DWITH_APPLICATION_CORE=1 /DWITH_COREUOBJECT=1 /DUSE_STATS_WITHOUT_ENGINE=0 /DWITH_PLUGIN_SUPPORT=0 /DWITH_PERFCOUNTERS=1 /DUSE_LOGGING_IN_SHIPPING=0 /DWITH_LOGGING_TO_MEMORY=0 /DUSE_CACHE_FREED_OS_ALLOCS=1 /DUSE_CHECKS_IN_SHIPPING=0 /DWITH_EDITOR=1 /DWITH_SERVER_CODE=1 /DWITH_CEF3=1 /DWITH_LIVE_CODING=1 /DWITH_XGE_CONTROLLER=1 /DUBT_MODULE_MANIFEST=\"UE4Editor.modules\" /DUBT_MODULE_MANIFEST_DEBUGGAME=\"UE4Editor-Win64-DebugGame.modules\" /DUBT_COMPILED_PLATFORM=Win64 /DUBT_COMPILED_TARGET=Editor /DUE_APP_NAME=\"UE4Editor\" /DWIN32=1 /D_WIN32_WINNT=0x0601 /DWINVER=0x0601 /DPLATFORM_WINDOWS=1 /DNDEBUG=1 /DUE_BUILD_DEVELOPMENT=1 /DUE_IS_ENGINE_MODULE=1 /DUE_ENABLE_ICU=1 /DWITH_VS_PERF_PROFILER=0 /DWITH_DIRECTXMATH=0 /DWITH_MALLOC_STOMP=1 /DCORE_API=DLLIMPORT /DCOREUOBJECT_API=DLLIMPORT /DSETTINGS_API=DLLIMPORT /DLIVECODING_API=DLLEXPORT /TC /W0  /Fo"%2" "%1" '
	.CompilerOutputExtension = '.c.obj'
}

// "C:\WINDOWS\system32\cmd.exe" /C "copy /Y "D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\ThirdParty\Windows\DirectX\x64\dxcompiler.dll" "D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\dxcompiler.dll" 1>nul"
// "C:\WINDOWS\system32\cmd.exe" /C "copy /Y "D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\ThirdParty\Windows\DirectX\x64\dxil.dll" "D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\dxil.dll" 1>nul"
// "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\link.exe" @"D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\ShaderFormatD3D\ShaderCompileWorker-ShaderFormatD3D.dll.response"
Copy('Action_1746_dummy')
{
	.Source = 'D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\ShaderFormatD3D\ShaderCompileWorker-ShaderFormatD3D.dll.response'
	.Dest = 'D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\ShaderFormatD3D\ShaderCompileWorker-ShaderFormatD3D.dll.response.dummy'
	.PreBuildDependencies = {
		'Action_200', ;SharedPCH.Core.NoUndef.cpp
		'Action_1108', ;Module.ShaderFormatD3D.cpp
		'Action_278', ;ShaderCompileWorker-Core.lib
		'Action_534', ;ShaderCompileWorker-RenderCore.lib
		'Action_763', ;ShaderCompileWorker-ShaderPreprocessor.lib
		'Action_765', ;ShaderCompileWorker-ShaderCompilerCommon.lib
	}
}

Executable('Action_1746')
{
	.Linker = 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\link.exe'
	.Libraries = 'Action_1746_dummy'
	.LinkerOptions = '/TLBOUT:"%1" /Out:"%2" @"D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\ShaderFormatD3D\ShaderCompileWorker-ShaderFormatD3D.dll.response" '
	.LinkerOutput = 'D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-ShaderFormatD3D.dll'
}

This action, along with some other actions (like 1745 in my case) are missing as well. Checking the ActionTypes, all of them are BuildProject or WriteMetaData steps as expected from CreateBffFile().
Since this function hasn't really changed in comparison to our working setups for earlier engine versions, I'm in the dark about why these actions have suddenly become a problem.

Try with this new version, it should fix this problem
FASTBuild.zip

@MrPatry Thx again for the quick answer. I just figured it out myself and fixed it slightly differently. Here's my solution as well - with what I had to change to make caching work as well.
FASTBuild.zip

And it seems that the win sdk is the culprit. Which version are you using @MrPatry ?
@meiwen333 I'm using 10.0.16299.0

hello  @meiwen333
with all this setup do you have buld failure, that are complaining of missing Lib files... ?
11>UnrealBuildTool : error : Failed to produce item: C:\PerforceRoot\U9\Unreal-Stable\Engine\Plugins\Tests\RuntimeTests\Intermediate\Build\Win64\UE4Editor\Development\RuntimeTests\UE4Editor-RuntimeTests.lib
those kind?

Hi,
@msolemajor
I didn't have these errors.
I got it up and running with a change that was suggested in this thread:
https://github.com/liamkf/Unreal_FASTBuild/issues/29

Hey guys, just to let you know that instead of having to do the regex trick, you can simply set bGenerateDependenciesFile to false in CppCompileEnvironment.cs
It will also allow incremental builds to work.
Also, I suggest adding this to your BuildConfiguration.xml file to prevent lib linking issues
<?xml version="1.0" encoding="utf-8" ?>
<Configuration xmlns="https://www.unrealengine.com/BuildConfiguration">
	<BuildConfiguration>
		<bEditorDependsOnShaderCompileWorker>false</bEditorDependsOnShaderCompileWorker>
	</BuildConfiguration>
</Configuration>

Cheers

Hey Guys!
I've integrated @MrPatry file but I have a weird behavior: every time I try to build the project it looks like the previous actions listed are rebuilt again and again.
ex:
I launch a build and only have one cpp to build. The build complete as expected. If I launch the build again the same cpp is built again without any changes.
Does it ring a bell to anyone?
Cheers

@oliviergaertner I think this was fixed with @MrPatry latest comment allowing incremental builds to work:

Hey guys, just to let you know that instead of having to do the regex trick, you can simply set bGenerateDependenciesFile to false in CppCompileEnvironment.cs
It will also allow incremental builds to work.

According to some threads on UDN setting bGenerateDependenciesFile  to false might generate some linking issues down the road.
Are experiencing any? Like when adding UFunctions etc?

IMO setting bGenerateDependenciesFile to false should does the trick, but as it doesn't generate dependencies file so it cannot do incremental build properly. After I turned it to false and I had to rebuild all files every time.
There should be a way to generate dependencies file with FastBuild instead of cl-filter.exe.

@oliviergaertner @nullbus After merging @MrPatry's work, I am experiencing full rebuild every time. Is this expected behavior at this moment with FBuild on 4.22?
Anyone tried putting 4.21's UBT back to 4.22?

I made some fix to generate dependency files with FastBuild and got it working in 4.22 Windows Editor(with VS2019). As it adds more pass so the compilation time has increased. And if you are building engine, there are still some issues that can be fixed or detoured manually:

I encountered C1001 compiler error when compiler tries to pass #import directive in the engine source code. If you look at the errorous source code, you might know these are easily turned off setting macro variables.
Use English as the primary language pack of the Visual Studio. I used Korean(my mother tongue) at first, but it generated empty dependency files.

I attached my FASTBuild.cs file here but it differs much from original code to fit my usage. So please use at your own risk.
FASTBuild.zip

I made some fixes to generate dependency files with FastBuild and got it working in 4.22 Windows Editor(with VS2019). As it adds more pass so the compilation time has increased. And if you are building engine, there are still some issues that can be fixed or detoured manually:

I encountered C1001 compiler error when compiler tries to pass #import directive in the engine source code. If you look at the errorous source code, you might know these are easily turned off setting macro variables.
Use English as the primary language pack of the Visual Studio. I used Korean(my mother tongue) at first, but it generated empty dependency files.

... And also I think there are some possible improvements:

Distribute dependency generation as it is almost same as normal compile process.
....

I attached my FASTBuild.cs file here but it differs much from original code to fit my usage. So please use at your own risk.
FASTBuild.zip

I've tried using this and am getting this error:
2>   at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional)
2>   at System.IO.Path.InternalGetDirectoryName(String path)
2>   at UnrealBuildTool.FASTBuild.AddCompileAction(Action Action, Int32 ActionIndex, List1 DependencyIndices) in C:\Program Files\Epic Games\UE_4.22\Engine\Source\Programs\UnrealBuildTool\System\FASTBuild.cs:line 854 2>   at UnrealBuildTool.FASTBuild.CreateBffFile(List1 InActions, String BffFilePath, List`1 LocalExecutorActions) in C:\Program Files\Epic Games\UE_4.22\Engine\Source\Programs\UnrealBuildTool\System\FASTBuild.cs:line 1220
Not sure whats wrong, might be due to an update since this is a few months old. Hopefully someone smarter than me can figure out how to make this work with 4.22

Not sure if related, but I got an error in AddCompileAction on line 958. It was due to the LastIndexOf command which in some cases returned -1. Adding a second >0 check solved that error for me.

@meiwen333 Hey, thanks for the reply. Do you think you could upload your fastbuild.cs? I got your last uploaded version working, but only after adding "bEnableUndefinedIdentifierWarnings = false;" to my build.cs which I'm certain will cause issues down the line.

@wout276  Hey, thanks so much for the reply and uploading your fastbuild.cs. Do you think you could upload your most recent fastbuild.cs?  I got your last uploaded version working, but only after adding "bEnableUndefinedIdentifierWarnings = false;" to my build.cs which I'm certain will cause issues down the line, so I'd like to see if you did anything I'm missing.

@JacobShawC
Setting bEnableUndefinedIdentifierWarnings = false; is unfortunate indeed but since we don't experience many issues due to it being disabled, I have never searched for any solutions for that. Afaik, it's something everyone working with fastbuild after 4.19 is doing.
We recently switched to @nullbus' fastbuild.cs and doing the extra pass to generate the dependency files. However, since we are heavily using caching and the extra pass is not distributed nor cached, it feels like it's slower in our case than doing a full rebuild every time. Can't tell for sure yet though.
Anyway, here is our current fastbuild.cs file. It's mostly the same as @nullbus' file now but with some minor changes.
FASTBuild.zip

It's been long time I have done it before. For the file I posted early, there is a problem that build distribution not started until generating bunch of dependency files. (For engine build, I had to wait 5+ minutes with 12 threads). I don't know why it's not starting because its dependency have already finished.
But when I reversed the dependency between 'obj file' and 'dependency txt file' in bff file, build distribution started almost immediately as I expected. If you are building with remote machine(s), it would show better performance.
I cannot share my file right now because it needs more improvement(including merging @wout276's work). But I think it's kinda easy to implement.

Hi @nullbus , I don't completely understand why reversing the dependency would result in an overall speedup or what exactly you mean by reversing the dependency. Changing the dependency from obj->txt to txt->obj and then passing the dependency txt's to fastbuild as targets does indeed start the build immediately. Although, in my tests, the total time is the same because the dependency files still get created afterwards.
I think the only way to speed this up would be to make distribution work for the extra pass or to do everything in one pass. Shouldn't it be possible to use cl-filter (which creates a cl.exe child process) to generate the txt files as well as the object files? I've tried a little bit to get it working but I've had no luck so far.

@wout276 I understand your situation, as generating dependency files are only processed in locally now. I also usually waited 1-2 more minutes for dependency files.
And yes, (I guess) cl-filter.exe's original purpose is creating a dependency file with a object file. But I wonder it also could be done with FastBuild, which cannot handle multiple output files with single ObjectList node. It is also discussed at FastBuild issue last year. IMHO distributed cl-filter pass could be possible after that issue has resolved.
And I found another issue that incremental build almost not working. The cause was about dependency file pass(second pass). To make process faster and to make dependency file without generating object file, I used /E flag that emits preprocessed source files to stdout. And this huge amount of output mixed with dependency info(/showIncludes emits information to stderr) blocks cl-filter to process properly. I tried a lot to solve that problem but I got no luck and I had to make modified cl-filter that handle stderr and stdout separately.

@nullbus Thx for sharing your way of working, I'll do some more testing and we'll go with the solution that's currently fastest for us.
It's a hacky workaround but if we're not satisfied yet, I may look into archiving the output files together and referring that archive as the outputfile to fastbuild and de-archiving on the client side. It will also create overhead but with a fast archiving algorithm, it could be worth it.

Hi @nullbus @wout276 , I'm trying to add fastbuild into the engine these days and I successfully builded the engine with your files, Thanks for your jobs!
But I can not fix the incremantal build issue. The engine does full build every time even with no modifications, and the build takes unacceptable time althought the the fastbuild cache works.
I tried everything i can do but got no progress, any ideas?

Hi @ShuraLiu, with the changes made by @nullbus, a full rebuild shouldn't happen anymore. It does however rebuild all dependency files every time, even with caching enabled. Unfortunately, there's currently no solution for this.
I suggest you test for your own case whether a full cached rebuild is faster than the partial rebuild with dependency files and choose the appropriate one. Unfortunately, building will in any case be slower than in UE4.21. It will probably stay this way for future engine versions as well, until Fastbuild supports multiple output files or someone finds and implements another workaround.

Can someone help me maybe? I'm using FastBuild.cs from  @wout276 and while building from source and Distributed Compilation I get this error:
1>4> Obj: D:\UnrealEngine\release\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Development\CoreUObject\Module.CoreUObject.5_of_7.cpp.obj
1>PROBLEM: D:\UnrealEngine\release\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Development\Core\Module.Core.6_of_11.cpp.obj
1>PROBLEM: C:\Users\fox\AppData\Local\Temp.fbuild.tmp\0x00000000\core_1003\Module.Core.6_of_11.cpp.obj
1>Module.Core.6_of_11.cpp
1>c1xx : fatal error C1356: unable to find mspdbcore.dll
1>EXEC : Failed to build Object. error : 2 (0x02) Target: 'C:\Users\fox\AppData\Local\Temp.fbuild.tmp\0x00000000\core_1003\Module.Core.6_of_11.cpp.obj'
1>-> Obj: D:\UnrealEngine\release\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Development\Core\Module.Core.9_of_11.cpp.obj <REMOTE: DESKTOP-NDLRD8J>

https://github.com/Ronsenval/UnrealEngineCompilationUsingFastBuild
This is a new repo by @Ronsenval , I didn't test his changes yet but he might have found a nice solution for the dependency generation. It might be interesting for at least some of you.

I've just integrated @Ronsenval solution for 4.23 but at the moment I have some link issues (very long link time and linker memory goes crasy)

When I tested his solution the last few days, I had very long build times since it rebuilded big parts of the engine even though I didn't change a letter of code. I was able to setup my own version for 4.23 though by adding the Intel ISPC compiler. I guess currently it's still a matter of choosing the implementation with the downside that's the least bad for your own situation..

So is development for this now dead? We're now approaching 4.24 and still no official release for a compatible version.
I've unfortunately had to skip using fastbuild as the fixes posted here had too many issues.

*cricket cricket

As Incredibuild prices are a little too much for starting indie, and my particular refusal to quit against my better judgement I'm working on porting FASTBuild to 4.24 (but I'm just tinkering in my free time), can't promise if I'll be able to do it and even when. But as I fixed some PhysX bugs in UE4 before (NDA source, cannot publish it anywhere) I hope it won't be too hard.

@Hisamera The problem is not getting it to work, there's still a few people/companies using this. The problem is just that changes were made to UnrealBuildTool that aren't compitable with Fastbuild. Thus requiring workaround and losing a lot of the performance gain in comparison to before and making the development time not quite worth it.
On top of that, I found some hints in the code of 4.25 and 4.26 that a native fastbuild implementation might be coming to the official UE4 releases soon(?) - with the needed BuildTool changes. Also Oculus has implemented fastbuild to some extend in it's version of UE4, but didn't really solve the need for an extra dependency pass afaik.
These reasons together are why development appears to be dead currently. You can never get the performance gains you could get before without making drastic changes to the build tool or Fastbuild itself.

@wout276 I'm not doing it for profit (I'm working on porting FASTBuild to 4.24 (but I'm just tinkering in my free time) ). I never would have been where I currently am, if I just did things that were profitable, or worth the hassle. I fixed broken PhysX implementation in UE4, even though turning it off would have been sufficient, that should tell you a lot about my ambitions.
Secondly, I highly respect your willingness to give me your edited file, but I'm doing it precisely because it's not easy. I'm like a homing rocket when it goes for problems with C++ irrelevant of where it originated, whenether in UE4 source, or projects source, because I did things the hard way first time. The only thing that is totally opaque to me is UBT, writing a FASTBuild integration is a quickest way for me to learn it. I'm doing programming stuff for over 10 years now, recently professionally, and I never learned quicker than when I was faced with a problem.
Thirdly I looked at a lot of implementations, and attempts to fix them, but I decided to write it from scratch, and I'm nearing finishing (My file already has 40KB of code, and probably will double) local compilation (so all cores are used not just one), from my understanding caching would be rather easy to implement (as FASTBuild just does it itself), and distributed could take a long time, but company where I'm employed is trying to give me a exclusive access to one of their lesser PCs. All of that was done in my free time during a week nontheless, I'm optimistic about finishing it this weekend.
Fourthly, It won't be a wasted effort for ME, even if UE4.26 will have it natively, as I will learn a ton of stuff about UBT and other stuff.
Fifthly, thats a last thing that keep nagging me, and I want to return to writing modding tools for Stellaris as I always wanted, but I couldn't for a long time XD

edit: If you want I can send you a functional version for UE4.24 but we ditched it since.

What did you ditch it for? Other than Incredibuild I'm not aware of an alternative.

@Hisamera I should have clarified it is still extremely useful for building local only, as it's only the distributed build step that is hindered by Fastbuild's limitations. If your implementation is finished and you're willing to share it, I would like to have a look at it out of curiosity :)
@ricekab We didn't replace it, it's still the best alternative and if we really need it again, I can have it running in a few hours. There's just some inconveniences when working from a custom binary build that's build with fastbuild on pc's that don't have the exact same setup.

Experimental fastbuild support in unreal engine staging branch:
https://github.com/EpicGames/UnrealEngine/tree/release-engine-staging/Engine/Source/Programs/UnrealBuildTool/Executors/Experimental

In UE 4.26 it wont work unfortunately. Tested with Sample Project using two machines.


Ubuntu's rviz kinetic got SIGABRT in libOgreMain.so.1.9.0

Got this fault (coredump in attachment):
rviz: /build/ogre-1.9-mqY1wq/ogre-1.9-1.9.0+dfsg1/OgreMain/src/OgreNode.cpp:630: virtual void Ogre::Node::setScale(const Ogre::Vector3&): Assertion `!inScale.isNaN() && "Invalid vector supplied as parameter"' failed.

Thread 1 "rviz" received signal SIGABRT, Aborted.
0x00007ffff69dc428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
54      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff69dc428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff69de02a in __GI_abort () at abort.c:89
#2  0x00007ffff69d4bd7 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x7ffff3c245e8 "!inScale.isNaN() && \"Invalid vector supplied as parameter\"", file=file@entry=0x7ffff3c24528 "/build/ogre-1.9-mqY1wq/ogre-1.9-1.9.0+dfsg1/OgreMain/src/OgreNode.cpp", line=line@entry=630,
    function=function@entry=0x7ffff3c24a20 "virtual void Ogre::Node::setScale(const Ogre::Vector3&)") at assert.c:92
#3  0x00007ffff69d4c82 in __GI___assert_fail (assertion=0x7ffff3c245e8 "!inScale.isNaN() && \"Invalid vector supplied as parameter\"", file=0x7ffff3c24528 "/build/ogre-1.9-mqY1wq/ogre-1.9-1.9.0+dfsg1/OgreMain/src/OgreNode.cpp", line=630,
    function=0x7ffff3c24a20 "virtual void Ogre::Node::setScale(const Ogre::Vector3&)") at assert.c:101
#4  0x00007ffff3a42d8f in Ogre::Node::setScale(Ogre::Vector3 const&) () from /usr/lib/x86_64-linux-gnu/libOgreMain.so.1.9.0
#5  0x00007fffb69022f7 in rviz::ShapeMarker::onNewMessage(boost::shared_ptr<visualization_msgs::Marker_<std::allocator<void> > const> const&, boost::shared_ptr<visualization_msgs::Marker_<std::allocator<void> > const> const&) () from /home/t4g/repos/cnord/catkin_ws/devel/lib/librviz_default_plugin.so
#6  0x00007fffb68fd1de in rviz::MarkerBase::setMessage(boost::shared_ptr<visualization_msgs::Marker_<std::allocator<void> > const> const&) () from /home/t4g/repos/cnord/catkin_ws/devel/lib/librviz_default_plugin.so
#7  0x00007fffb68d8eaa in rviz::MarkerDisplay::processAdd(boost::shared_ptr<visualization_msgs::Marker_<std::allocator<void> > const> const&) () from /home/t4g/repos/cnord/catkin_ws/devel/lib/librviz_default_plugin.so
#8  0x00007fffb68d85f8 in rviz::MarkerDisplay::processMessage(boost::shared_ptr<visualization_msgs::Marker_<std::allocator<void> > const> const&) () from /home/t4g/repos/cnord/catkin_ws/devel/lib/librviz_default_plugin.so
#9  0x00007fffb68d922c in rviz::MarkerDisplay::update(float, float) () from /home/t4g/repos/cnord/catkin_ws/devel/lib/librviz_default_plugin.so
#10 0x00007ffff7a808fd in rviz::DisplayGroup::update(float, float) () from /opt/ros/kinetic/lib/librviz.so
#11 0x00007ffff7b4c049 in rviz::VisualizationManager::onUpdate() () from /opt/ros/kinetic/lib/librviz.so
#12 0x00007ffff1f80d2a in QMetaObject::activate(QObject*, int, int, void**) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#13 0x00007ffff1f8d5c8 in QTimer::timerEvent(QTimerEvent*) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#14 0x00007ffff1f81bb3 in QObject::event(QEvent*) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#15 0x00007ffff746405c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#16 0x00007ffff7469516 in QApplication::notify(QObject*, QEvent*) () from /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#17 0x00007ffff1f5238b in QCoreApplication::notifyInternal(QObject*, QEvent*) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#18 0x00007ffff1fa75ed in QTimerInfoList::activateTimers() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#19 0x00007ffff1fa7b29 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#20 0x00007ffff14a9197 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#21 0x00007ffff14a93f0 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#22 0x00007ffff14a949c in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#23 0x00007ffff1fa87cf in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#24 0x00007ffff1f4fb4a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#25 0x00007ffff1f57bec in QCoreApplication::exec() () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#26 0x00000000004011cb in main ()

Run evironment:
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /opt/ros/kinetic/lib/rviz/rviz...(no debugging symbols found)...done.
Starting program: /opt/ros/kinetic/lib/rviz/rviz --version
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffdd772700 (LWP 13020)]
[ INFO] [1540317904.529952330]: rviz version 1.12.16
[ INFO] [1540317904.529993333]: compiled against Qt version 5.5.1
[ INFO] [1540317904.530004022]: compiled against OGRE version 1.9.0 (Ghadamon)

rviz-sigabrt-inscale-isnan.coredump.gz

Thanks for the details summary, I don't have time to investigate, but perhaps it can be a lighting rod for other people with the issue and you guys can help one another.

In additional, here is the listing of ldd /opt/ros/kinetic/lib/rviz/rviz:
https://gist.github.com/theg4sh/0e916104c3513896a354568560cb72a4

Sorry for troubling, that was my bad. After ros update, I forgot to rebuild rviz from repository which i built earlier (a few months ago). Looks like works good now.


Upload signature on collection detail page

When the flag can_upload_signatures is set it takes precedence over the backend signing. Therefore we are hiding the Sign all buttons on the collection and namespace detail pages, and changing the Sign version x.x to bring up the certificate upload modal.
In addition it fixes the waitForTask function to return error if the task is not running and not completed so we can report failures.
After with the can_upload_signatures = true:

)

(restarting once more, the test failures seem unrelated, but...)

@himdel I re run it, but the same imports_filter test fails every time. I checked out locally and the tests is passing there. It seems pretty unrelated though.

@himdel same tests fails here too: https://github.com/ansible/ansible-hub-ui/pull/2228

Agreed, imports_filter is broken on master, feel free to ignore it for now :). (I started looking into it in https://github.com/ansible/ansible-hub-ui/pull/2165)
But I'm also seeing ✖collection_detail.js ✖collection.js ✖collections_list.js here

That's new, I didnt see them before, let me look into it

@himdel I cannot reproduce these locally, and sometimes other tests are failing

Restarting tests, hopefully that should tell us if it's something in this PR or coming from elsewhere :)

Passing, merging


[JetBrains Toolbox Recent Projects] shell script location problem

Description
Extension – JetBrains Projects
Author: @gdsmith
Hi, I faced the script path problem.

and I checked my permission, looks like I have the permission
I have no idea what's wrong, if need any context just tell me, thanks help

Steps To Reproduce
Computer: Mac Pro / M1 / MacOS Ventura 13.5

install raycast and JetBrains Toolbox Recent Projects extension
open raycast, use JetBrains command, and click Search Recent Projects

Current Behaviour

Expected Behaviour
https://timdeschryver.dev/blog/enable-shell-scripts-for-the-jetbrains-toolbox-app

This doesn't seem to be a API error. Can you open an extension issue instead: https://github.com/raycast/extensions/issues/new?assignees=&labels=extension%2Cbug&projects=&template=extension_bug_report.yml&title=[Extension+Name]+... so that the authors are properly aware of this?


[SQL Profiles] Feature request: deny certain data types

The SQL profiler provides mechanisms for denying profiling based on table/column name patterns.
However, this can become maintenance intensive if, e.g. we want to deny profiling for all columns that are of certain data types.
As an example, the Snowflake VARIANT/OBJECT/ARRAY data type is quite unsuited for profiling.
It would be great if DataHub would allow defining a list of data types which should be denied for profiling.
Furthermore, it would be even better if we could allow/deny cwertain metrics based on the data types.

This PR will probably fix it: #4328


Save a reference to the LCP and CLS targets in case they are removed from the DOM

Fixes #561

Test here: https://disappearing-cwv-node.glitch.me/index.html
Maybe we should add this to the test suite?
And maybe we should do this for CLS too?

Yes, I think we want a test for this behavior. It should be possible to test.

Added tests for LCP, CLS and even the INP scenario added in #477

Can you add the same logic for metric -> lcpEntry -> element?

@dmitiiv there's a request not to save the full element (see #580) but instead just the target due to the amount of memory that can be taken up. This makes reporting the element back kinda pointless (since it can be got from the lcp entry).
Would the selector be sufficient for you without the element?

Yes, I've seen the issue. Removing the element (without saving) makes the fix almost useless for the purpose of obtaining information about the target element in a Single Page Application (SPA). It is critical to keep statistics without losing the details of the problem. However, if we still have an element selector, it could allow us to safely retain the necessary information.
Briefly, yes its not that was expected but better if we don't have even a selector

Yes we would still look to save the element selector (and in fact this would be the better selector since it would be calculated while the element is still part of the DOM, rather than once it's been removed).

There are a number of tradeoffs being made here, balancing different needs from various users of this library.
I think the root of all these issues is that currently the library has its own internal getSelector() function that users cannot modify, so some users have asked for the library to expose the underlying DOM element as a solution. This gives users more flexibility, but it comes at a cost of requiring the library to store strong references to these DOM elements for all users, even users that don't need this functionality (because the default selector logic works fine for them).
I wonder if instead of the library making decisions about these tradeoffs, it would be better to give users full control, in the form of letting them customize the target selector generation logic.
Perhaps something like an option to pass their own getTargetSelector function:
onLCP(metricCallback, {
  getTargetSelector: (element) => {
    // Users can add whatever logic they want to generate a selector,
    // some other identifier, or even just return the element itself.
    return myCustomStringifyLogic(element);
  }
});

This option would be available for all metric functions in the attribution build.

Regarding whether to run the getSelector() logic at reporting time or at the time when the entry is dispatched, there are also tradeoffs here. I don't think the getSelector() is slow or anything, but it does have to traverse up the DOM tree for each element, which (depending on the size of the DOM and the number of page interactions, layout shifts, etc.) could result in a lot of processing. That's the main reason I originally choose to generate the selector at reporting time.
While we could make this change and always calculate at entry-dispatch time, any users who need this behavior can already get it by using the reportAllChanges option, so this change would impose the slower behavior on all users, even those who don't need it.
To be clear, I'm not necessarily opposed to this change, I just want to make sure we're thinking about all of the trade offs before making a decision that impacts all users of the library.


Laravel 8 Issues and RuntimeException after running composer require

When initially setting up the module I get the following error when using composer require:
Problem 1
    - Root composer.json requires area17/twill-api dev-develop -> satisfiable by area17/twill-api[dev-develop].
    - area17/twill-api dev-develop requires illuminate/support ^9.12 -> found illuminate/support[v9.12.0, ..., 9.x-dev] but these were not loaded, likely because it conflicts with another require.

You can also try re-running composer require with an explicit version constraint, e.g. "composer require area17/twill-api:*" to figure out if any version is installable, or "composer require area17/twill-api:^2.1" if you know which you need.

Re-running composer require using composer require area17/twill-api:* as suggested by the error message, installs the module correctly. with these versions:
- Locking area17/twill-api (dev-main 0166f5a)
  - Locking laravel-json-api/core (v2.3.0)
  - Locking laravel-json-api/eloquent (v2.1.1)
  - Locking laravel-json-api/encoder-neomerx (v2.0.1)
  - Locking laravel-json-api/exceptions (v1.1.1)
  - Locking laravel-json-api/laravel (v2.4.0)
  - Locking laravel-json-api/neomerx-json-api (v5.0.1)
  - Locking laravel-json-api/spec (v1.2.0)
  - Locking laravel-json-api/validation (v2.1.0)

However yields the following error when artisan attempts to auto-discover these packages after the composer require call:
  RuntimeException

  Server v1 does not exist in config or is not a valid class.

  at vendor/laravel-json-api/core/src/Core/Server/ServerRepository.php:69
     65▕
     66▕         $class = $this->config()->get("jsonapi.servers.{$name}");
     67▕
     68▕         if (empty($class) || !class_exists($class)) {
  ➜  69▕             throw new RuntimeException("Server {$name} does not exist in config or is not a valid class.");
     70▕         }
     71▕
     72▕         try {
     73▕             $server = new $class($this->app, $name);

      +12 vendor frames
  13  [internal]:0
      Illuminate\Foundation\Application::Illuminate\Foundation\{closure}()

      +5 vendor frames
  19  artisan:37
      Illuminate\Foundation\Console\Kernel::handle()
Script @php artisan package:discover --ansi handling the post-autoload-dump event returned with error code 1

From my investigations of both the documentation available here and laravel-json-api, there's no clear-cut way to get this module up and running that I can find.
I'm currently running Laravel v8.83.25 with a fresh install of Twill, with no other additions/features/functionality

Hello @Luke-Arch, sorry for the long delay before looking into that. I will comment for other people who might end up here.
twill-api depends on Laravel JSON:API package and so, the minimum Laravel version supported would be Laravel - ^8.76 or ^9.0 according to their documentation.
Regarding the other error about the server v1 that does not exist, I updated the documentation and installation process (along making it much simpler than early dev version). If you update you package to the current develop branch and follow the instruction, you should not run into this issue.
I am closing this for now, but please re-open if you run into this issue again.


Added support for iOS and Android

Fixes #353.
Fixes #354.
This adds (long overdue) support for building for Android (ARM64/ARM32/x64/x86) and iOS.
Note that due to limitations in GDExtension the iOS build does not support running in the iOS simulator.

🐐


GH-2868: corrected option '--target' in help

This fixes #2868
I am, however really unsure about PR'ing into main, so feel free to reject this.

@nils-a yeah, PR'ing into main isn't the correct course of action here.
If this is indeed going to go out as a bug fix, we will need to first create a hotfix branch, and this PR would need to be re-directed to that branch.


[SMALLFIX] Improve handling for the serving thread hanging

The serving thread is supposed to be responsive to interrupts such that this will never happen, but in case the thread doesn't die after 5 minutes, we should stop the process and log the stack of the serving thread.
The PropertyKey is added with isHidden=true since users should never need to see this configuration, but it could be useful for developer debugging purposes, or to avoid the System.exit in case it causes issues.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/18599/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/18600/
Test PASSed.


are different kernels supported?

Is it possible to use thebelab with kernels other than Python?

Sure, check out the configuration: https://thebelab.readthedocs.io/en/latest/config_reference.html

lovely, thank you!
On Mon, 28 Oct 2019 at 22:10, Anton Akhmerov<EMAIL_ADDRESS>wrote:

Sure, check out the configuration:
https://thebelab.readthedocs.io/en/latest/config_reference.html
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/minrk/thebelab/issues/170?email_source=notifications&email_token=ABCBUPRZEESGVSKXUQ2RUVLQQ5IMZA5CNFSM4JGABJQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECOM5HY#issuecomment-547147423,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABCBUPUFMMF3OFEWVC7TUBLQQ5IMZANCNFSM4JGABJQQ
.


Ping seems quite volatile

Hey,
I replaced HAProxy with Hopper as a proxy and noted the following:
With HAProxy, my ping appears to be stable at around ~20:

With Hopper, it likes to jump around quite a lot and is way higher than it was:

(This behavior is also seen by different users from different parts of the country with different ISPs)
Both Revers Proxies connect to the same Minecraft Proxy and are hosted by the same Hoster. And both Revers Proxies are using the Proxy Protocol.
I can give you both Revers Proxies IPs in case you are interested and want to investigate that.

Hi, does this happen when there are multiple players connected on the same proxy or does it happen with only a player connected too?

Hi, does this happen when there are multiple players connected on the same proxy or does it happen with only a player connected too?

Happens in both cases, with just me and with dozens of players connected

I just released a commit with what I think might be the solution to the problem. If not then I will continue to investigate.

It definitely improved the volatile part. It no longer jumps around every 2 to 3 seconds, especially not up to +100, however it is still consistently around 1.5x to 2x higher than with HAProxy.

Basically, HAProxy uses splice on linux to zero-copy transfer data from the client socket to the server socket.
The only problem is that splicing between sockets requires the creation of a pipe. So a total of 2 pipes per connection, and creating pipes is an "expensive" operation. HAProxy solves this by having a pipe pool, which is probably what I will implement in hopper too.

Hi there!
Is there an ETA on the new method? :)

@faultydev @Alex1607 sorry for my absence. The fix was just to disable the Nagle algorithm by setting nodelay to true on the sockets.
Rolling out this fix in a few minutes


[BUG] Invalid preKey ID

Describe the bug
A clear and concise description of what the bug is.
Some of the messages I've received from events['messages.upsert'] have Invalid PreKey ID, does anyone have any idea why this is happening and how to fix it?
To Reproduce
Steps to reproduce the behavior:

Created a new connection
Create event listener for events['messages.upsert']
Send message to server number
Some messages have Invalid PreKey ID

Expected behavior
Receive the contents of the message according to what we receive on WhatsApp
Environment (please complete the following information):

Is this on a server? yes
What do your connectOptions look like?
const sock = makeWASocket({        printQRInTerminal: true,        patchMessageBeforeSending: (message) => {          const requiresPatch = !!(            message.buttonsMessage ||            message.templateMessage ||            message.listMessage          );          if (requiresPatch) {            message = {              viewOnceMessage: {                message: {                  messageContextInfo: {                    deviceListMetadataVersion: 2,                    deviceListMetadata: {},                  },                  ...message,                },              },            };          }          return message;        },        auth: state,        msgRetryCounterMap,        markOnlineOnConnect: false,        shouldIgnoreJid: jid => isJidStatusBroadcast(jid),        defaultQueryTimeoutMs: undefined      });
Do you have multiple clients on the same IP? Yes
Are you using a proxy? No

Additional context
This Problem does not occur in all messages

Any update? I am facing same issue but inside Jering javascript node, package from dotnet

up

No update for this issue :(

up

anyone know how to solve this issue ?

UP

UP

I encountered this problem today. And it only resolved after I sent the first message.
What I'm thinking of testing: On the mobile phone, set up a welcome message. So, when receiving a first message, the mobile will trigger a response. And from then on, the next messages work normally. It's not the ideal solution, just a temporary fix.

Any update?

auth: {
	creds: state.creds,
	keys: makeCacheableSignalKeyStore(state.keys, logger),
}

Any update?

will work on this issue

I don't quite get why but authState methods request for a pre-key that does not exist in keys and was never set. Here are logs after testing:
Settings keys:
setting keys: {"pre-key":{"144":{"private":{"type":"Buffer","data":[0,163,112,193,154,175,125,158,32,72,180,111,115,130,141,236,181,144,185,179,239,250,108,89,75,223,190,238,161,5,209,84]},"public":{"type":"Buffer","data":[79,70,23,71,138,5,159,183,231,196,89,52,38,45,206,14,254,159,243,62,170,17,123,155,96,188,118,147,165,240,151,108]}}}}
Trying to get pre-key immediately after fetching new message upsert:
reading key pre-key, ids: 71

up

up

same problem

@PurpShell ^ this is losing a lot of messages.

1 month later, time to make some serious progress on libsignal

looked into this on the libsignal level, found that the prekey chain is getting lost / devoured / deleted somehow

there is a quick fix to this for now (stop all deletions on auth store level) but it's better to command it from above on to why invoke deletion of the prekey pairs

do you guys get this using useMultiFileAuthState too?
I can't for the life of me figure out why when using an external DB I get the preKeyId error but not when using local files

do you guys get this using useMultiFileAuthState too? I can't for the life of me figure out why when using an external DB I get the preKeyId error but not when using local files

Same here, i cant get preKeyId when use external DB, but when using local files all fine

UP!!

do you guys get this using useMultiFileAuthState too? I can't for the life of me figure out why when using an external DB I get the preKeyId error but not when using local files

Same here, i cant get preKeyId when use external DB, but when using local files all fine

hey guy, could you share the part of your code using local files? I mean, the part that you create, read and write/update it

up


patch: apply drizzle eslint rules to ctx.db

Closes #
✅ Checklist

[x] I have followed every step in the contributing guide (updated 2022-10-06).
[x] The PR title follows the convention we established conventional-commit
[x] I performed a functional test on my final commit

Changelog
Add ctx.db to the drizzleObjectName list for the drizzle eslint rules to be applied when using ctx.db in trpc routes.
Previously, it would only apply if you import { db } from "@/db"

Screenshots
[Screenshots]
💯

git checkout from wrong branch, my bad 🤦🏻


Setup Sonarcloud connected mode (IDE integration for Intellij)

Type of change
Select the type of your PR

Enhancement / new feature

Description
The Sonar plugin for IntelliJ has the ability to use rules from project definition in Sonarcloud directly in the IDE.  This configuration can be committed to a project's repo in order to give all the project's developer's the same experience   The feedback the developer gets in the IDE should match the feedback they'll receive from the CI workflow.  That should improve productivity.
As Kroxylicious already as a public organisation as Sonarcloud, kroxylicious and defines project for kroxylicious and kroxylicious-junit5-extension, we can benefit from this integration.
https://docs.sonarsource.com/sonarlint/intellij/team-features/connected-mode-setup/#bind-using-shared-configuration
Additional Context
I've been trying out the IDE integration for a few weeks without noticing any downsides, so I thought it is worth sharing the config.
Here's how it looks for me.

Checklist
Please go through this checklist and make sure all applicable tasks have been done

[ ] Write tests
[ ] Make sure all tests pass
[ ] Review performance test results. Ensure that any degradations to performance numbers are understood and justified.
[ ] Make sure all Sonarcloud warnings are addressed or are justifiably ignored.
[ ] Update documentation
[ ] Reference relevant issue(s) and close them after merging
[ ] For user facing changes, update CHANGELOG.md (remember to include changes affecting the API of the test artefacts too).

Having switched branches I think my intellij was in a bad state as it created a .connectedMode.json file this time...

Having switched branches I think my intellij was in a bad state as it created a .connectedMode.json file this time...

Let me see how my laptop reacts to appearance of the connectedMode.json file for the first time.

gah, what a horrific UI experience.
From what I can tell it ignored the connectedMode.json and then prompted me to create it (with the same content).  I'm baffled
Maybe we ditch this PR and just have some words in the DEVELOPER.md file?

gah, what a horrific UI experience.
From what I can tell it ignored the connectedMode.json and then prompted me to create it (with the same content). I'm baffled
Maybe we ditch this PR and just have some words in the DEVELOPER.md file?

Yikes. I hoped it was just mine install being confused...
Maybe @robobario or @gracegrimwood could give it a test on a clean system and see how they go before we give up on automation?
If they have the same experience adding notes to the Dev guide is a good move.

Apparently I had the plugin installed but never set up, so I gave this a go. I checked out this branch, restarted IntelliJ (that bit is important because it didn't do anything at all before I restarted it), and then it popped up with a message that there was "an existing connected mode configuration file" and asked if I wanted to use it. Once I did that (it asked for a token to set the connection up) it created the following .idea/sonarlint.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="SonarLintProjectSettings">
    <option name="bindingEnabled" value="true" />
    <option name="projectKey" value="kroxylicious_kroxylicious" />
    <option name="serverId" value="kroxylicious" />
  </component>
</project>

So it needs a user token from SonarCloud to get it setup properly, but otherwise that .sonarlint/connectedMode.json file works correctly for me. It didn't ask me to recreate it at all.

thanks @gracegrimwood .  it be good to hear @robobario 's experience.
so I suppose we could commit .idea/sonarlint.xml too, but we'd need to stipulate a serverId in the DEV_GUIDE.md.  We'd also need to check Intellij's behaviour if that server id wasn't present.

I got the same popup as Grace, generated a token (logging in with github) and it appears to be applying sonar rules. When I closed IDEA it wrote out the .idea/sonarlint.xml same as for Grace:
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="SonarLintProjectSettings">
    <option name="bindingEnabled" value="true" />
    <option name="projectKey" value="kroxylicious_kroxylicious" />
    <option name="serverId" value="kroxylicious" />
  </component>
</project>

maybe better to gitignore .idea/sonarlint.xml? The serverId was automatically populated when it popped up the dialogue to connect.

I'll add that ignore to the PR

This just needs a review tick, then I think it is good to merge.


Add prefab exporter

https://github.com/jessinthecloud/ValheimJsonExporter/blob/cfeee7ca69bb5bc457d645d1502a4ebc985db003/ValheimJsonExporter/ValheimJsonExporter.cs#L35

Not necessary -- piece/piecetable is what I want


test: disable lldb test as it requires privileges to run on macOS

Resolves #13413

@bors r+

:pushpin: Commit cd1f08899eaa0629d99e503bfeb5ef8df31bd091 has been approved by epage
It is now in the queue for this repository.

:hourglass: Testing commit cd1f08899eaa0629d99e503bfeb5ef8df31bd091 with merge 9a6bafd9560af3a4813a01db8e9ca98190014c01...

:sunny: Test successful - checks-actions
Approved by: epage
Pushing 9a6bafd9560af3a4813a01db8e9ca98190014c01 to master...

:sunny: Test successful - checks-actions
Approved by: epage
Pushing 9a6bafd9560af3a4813a01db8e9ca98190014c01 to master...

:eyes: Test was successful, but fast-forwarding failed: 422 Changes must be made through a pull request.


Eventbus propagation

Which problem is this PR solving?
Adds context propagation through the EventBridge putEvents methods
Short description of the changes
Uses the TraceHeader available from the SDK.
Sadly this only works with the X-Ray propagator, because it seems like AWS won't accept another type of propagator in the TraceHeader field.
The good part is that the AWS Lambda instrument already works without any change

Sorry, I screwed up. Reopened there: https://github.com/open-telemetry/opentelemetry-js-contrib/pull/1297


Is it possible to show 'skip' button instead of back button

Is there any provision to show a skip button replacing probably the back button? Is this currently doable?

was wondering the same


Support disable permission check on workspace

Description

Backport #216 to workspace branch
Support disable permission check on workspace

Issues Resolved

Screenshot

Testing the changes

Check List

[ ] All tests pass

[ ] yarn test:jest
[ ] yarn test:jest_integration
[ ] yarn test:ftr

[ ] New functionality includes testing.
[ ] New functionality has been documented.
[ ] Update CHANGELOG.md
[ ] Commits are signed per the DCO using --signoff

Codecov Report

Merging #228 (2b38868) into workspace (a74a6ee) will decrease coverage by 10.86%.
The diff coverage is n/a.

@@              Coverage Diff               @@
##           workspace     #228       +/-   ##
==============================================
- Coverage      66.22%   55.36%   -10.86%
==============================================
  Files           3422     1190     -2232
  Lines          65748    24050    -41698
  Branches       10587     4029     -6558
==============================================
- Hits           43540    13315    -30225
+ Misses         19571    10078     -9493
+ Partials        2637      657     -1980

Flag
Coverage Δ

Linux_1
?

Linux_2
55.36% <ø> (+<0.01%)
:arrow_up:

Linux_3
?

Linux_4
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

...ore/server/saved_objects/service/lib/repository.ts
89.43% <ø> (ø)

... and 2594 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Noticed that there are 3 unrelated commits in this PR, maybe should be dropped?

@Hailong-am , could you explain why we add this new flag?

Noticed that there are 3 unrelated commits in this PR, maybe should be dropped?

good suggestion, i have drop the 3 unrelated commits. will raise separate PR for backporting test cases.

@Hailong-am , could you explain why we add this new flag?

update it in PR description

@Hailong-am , could you explain why we add this new flag?

update it in PR description

If sets to false, all ACL check will be skipped?

If sets to false, all ACL check will be skipped including user level, right?

Yes, that will disable ACL totally.


Not bundling with ruby < 2.2

I'm having this annoying error everytime I have to bundle with celluloid-io in ruby < 2.2, where nio4r 2.0.0 resolves and breaks because it dropped support for such rubies. I don't know exactly the reasons for nio4r 2.0.0 not supporting lower rubies (kwargs perhaps?), but this change should therefore be reflected in celluloid-io to avoid headaches (for now I'm resolving nio4r versions myself in the Gemfiles, but the problem surges in libraries depending on it).
A couple of suggestions:

set the dependency from nio4r as "~> 1.2" (0.17 doesn't use byte buffers)
stop supporting 2.2 in celluloid-io

Upgrade to the latest bundler and and ruby RUBY_VERSION to the top of your Gemfile. Or add your own constraints on the nio4r version.

My use case is a library which uses celluloid-io as a dependency, and is installed as a system gem (no gemfile). Upgrading can't be done without a Ruby version manager. And there is no rails 5.
Can't this be resolved on the gemspec level? Like "if Ruby greater than add this dep, otherwise had this dep." ?

If you have a recent Bundler and add ruby RUBY_VERSION to the top of your Gemfile, it should do it automatically.
You can also add transitive dependencies to your Gemfile, and specify version constraints on them.
This is going to become increasingly a problem so long as you use an EOL Ruby, FYI.

I've just described you a use-case where bundler isn't used. Yes, I know how to handle these things using a Gemfile, it's not about that.
FWIW I'm not asking you to keep supporting EOL'ed versions of ruby, I understand the maintenance burden, I'm just asking you not to abruptly "break the build" needlessly. I say needlessly, because I've been using in production (and developing against) celluloid-io 0.16-0.17 since at least ruby 2.0, and it "just works". Not asking for corner case fixes for EOL'ed rubies as well.
I'll also add that the readme states "support ruby 1.9.3 up to 2.3.1" (yes, if you use nio4r < 2). I'll also point that celluloid-io master is currently breaking in travis CI due to a "laxed" dependency of a gem (listen) which broke support for rubies < 2.2 in version 3. In their case, they "communicated" that support termination using semver, and the celluloid suite could just pick it up.
This is just a suggestion of a solution that could be added in celluloid: communicate support termination using versioning. I know, it's not 1.x yet so one can do whatever, but one could just release an 0.18 version terminating support to EOL'ed rubies. Would any of these (or the previous) suggestions be something we could agree on? I could PR smth if we could.
Again, I'd like to expose my use-case: customer deploys a VM (behind a proxy) with stock ruby 2.0, and tools are installed system-wide. Tools (mostly general-purpose scripts, some deamons) use an in-house library using celluloid-io to configure remote devices. Can't influence much the decision about ruby version manager or ruby upgrades. And no rails 5.

Your problem is fundamentally a dependency resolution problem, and it both has a solution and one bundler can find for you.
I'm sorry you're not using bundler, and I'm sorry you want to use EOL Rubies, and I am sorry I have not updated the README of every last project to remove the EOL Rubies (I am still systematically going through them to do this, but there are a lot of them and it takes time).
But fundamentally: this is not a problem with this gem (and hence why I closed the issue). This is a problem with your current deployment system failing to resolve a solution.
I will eventually get to Celluloid::IO and correct some of the problems you mentioned, but I may or may not change the dependency relationship between celluloid-io and nio4r.
I would strongly suggest getting off EOL'd Rubies and adopting bundler for your deployments.

About the rubies, again not my call, but correlating the ruby EOL policy with the versions shipped in linux distributions and the rate of upgrades, it's hard to keep up without a version manager. But I agree, my system problem.
I just wanted to expose my use case and provide arguments, discuss a possible agreement and offer my contribution. If you think it is an edge case (I agree, it is) not worthy of specific handling, I'll have to accept.


Where is the win32-x64-59_binding.node？

I can't install "node-sass": "4.5.3" by npm.
`><EMAIL_ADDRESS>install C:\git\static-angular\node_modules\node-sass

node scripts/install.js

Downloading binary from https://github.com/sass/node-sass/releases/download/v4.5.3/win32-x64-59_binding.node
Cannot download "https://github.com/sass/node-sass/releases/download/v4.5.3/win32-x64-59_binding.node":
HTTP error 404 Not Found`

I have the same error for linux linux_musl-x64-59_binding.node:
><EMAIL_ADDRESS>install /builds/CEVP/cevp-web/node_modules/node-sass
> node scripts/install.js

Downloading binary from https://github.com/sass/node-sass/releases/download/v4.5.3/linux_musl-x64-59_binding.node
Cannot download "https://github.com/sass/node-sass/releases/download/v4.5.3/linux_musl-x64-59_binding.node":

HTTP error 404 Not Found

@baihech , seems like a duplicate. See this issue: https://github.com/sass/node-sass/issues/2129
Solution would be to downgrade the node to v8

@ox-michaelradionov I' try.

Duplicate of https://github.com/sass/node-sass/issues/2129


Where is the source code?

Having a hard time finding the source code for the project. I thought I would fork the project and tinker with is a bit but am having a hard time finding the source code? Is is not open source? If it is where can I find the code for it?

Sorry to disappoint you, the app is not open source.


Faster compression using hash table of history buffer

Store a hash table of two-byte pairs for each entry in the history table. That allows faster searches through history: when searching for a match, hash the first two bytes that are being searched, then look up the hash value in the table.
This uses more memory to store the hash table (e.g. ~4 kB), but gives a significant speed benefit.

Done in C code version 0.7.0


Unable to download this video - Value cannot be null

Try this sample video:
https://www.youtube.com/watch?v=PpG0z-npFIY
It will return this error:

Is this related to this bug?

@xamarin-enthusiast  @Tyrrrz  @floatas I am getting same error even in release 1.5.3


[bug] Dev server not starting

Currently, if you want to start the dev server, you will get the following error:
➜ npm run dev

><EMAIL_ADDRESS>dev
> email dev

✔ Emails preview generated
⠋ Installing dependencies...

up to date, audited 584 packages in 3s

55 packages are looking for funding
  run `npm fund` for details

7 vulnerabilities (2 moderate, 5 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

✔ Dependencies installed

><EMAIL_ADDRESS>dev
> next dev 3000

error - Invalid project directory provided, no such directory: /Users/bastiaanverhaar/Desktop/Darwin/outbound-communications/.react-email/3000

Specs:

OS: MacOS 13.1
Terminal: iTerm2 with Fish shell
NodeJS: v18.13.0
Package manager: NPM & Yarn

I found that the error is due to an incorrect shell command:
const startDevServer = (packageManager: string, port: string) => {
  shell.exec(`${packageManager} run dev -p ${port}`, { async: true });
}

This code should be:
const startDevServer = (packageManager: string, port: string) => {
  shell.exec(`${packageManager} run dev -- -p ${port}`, { async: true });
}

Will open a PR for it

Having the same issue on Windows, specs:

OS: Windows 11 Home 22H2
NodeJS: v18.13.0
Package Manager: npm

1.7.10 should fix this.


feat: supported display all types of broadway's names

relate on #12

@feng19 to fix the CI issue, you need to run mix deps.unlock --unused. Please take a look.

@feng19 to fix the CI issue you need to run mix deps.unlock --unused. Please take a look.

If I upgrade plug_cowboy to 2.5.1 that has no problem about it.
Upgrade it?

If I upgrade plug_cowboy to 2.5.1 that has no problem about it.
Upgrade it?

@feng19 yes, I think it's fine to upgrade. :+1: It's only used in development.

@feng19 thank you! :purple_heart:

❤❤❤


esdoc-publish-html-plugin includes non-configurable "class" directory for classes

This is a problem when using Jenkins as a CI tool; its "Publish Javadoc" post-build action results in code being served by Jetty, which silently filters out paths with a class part. To work around this, I've needed to include this horrible hacky monkeypatching "plugin" in my ESDoc configuration:
const DocBuilder = require('esdoc-publish-html-plugin/out/src/Builder/DocBuilder').default;

if (!DocBuilder.prototype.__gOFN) {
  DocBuilder.prototype.__gOFN = DocBuilder.prototype._getOutputFileName;
  DocBuilder.prototype._getOutputFileName = function (doc) {
    return this.__gOFN(doc).replace(/^class\//, 'cls/');
  };
}

module.exports = {}

Any ideas for a cleaner solution? Jenkins in this case is in a hosted environment and is not configurable at the required level to fix this there.

I filed a JENKINS-53095 for this issue.  I'm pretty sure it is just a problem with publishHTML, not Jetty.
Please vote up the issue if this impacts you. :smiley:


docs(wiki/stories-1.0-update): fix e2e project registration with ts-node

Merging https://github.com/angular/angular-cli/pull/5635 instead since it does the same but was submitted first.

:cry:


dash instead of underscore

Coverage remained the same at 19.77% when pulling bbfa0f38801c39e1116711d8a148be9932437bf1 on api-to-dash into df12c650daee393efea34e84bd14a8ececf97807 on master.


39 Moving the Front end application from Go to Next.js

Fixes https://github.com/open-telemetry/opentelemetry-demo/issues/39.
Changes
In this PR we are introducing the new front-end version. In this version, we are focusing on having a browser-side client written in Typescript.
The frontend was built based on the proposal presented during the last SIG call (July 25th 2022) and can be found here: https://docs.google.com/presentation/d/1klWGRAcFYJLBrXtMMRGHJvwQaMt1JPWnO7qQzFdJti4/edit?usp=sharing
Next Steps:

Add instrumentation.
Add unit tests and integration tests.

For significant contributions please make sure you have completed the following items:

[ ] Appropriate CHANGELOG.md updated for non-trivial changes
[x] Design discussion issue https://github.com/open-telemetry/opentelemetry-demo/issues/39

Video demo
https://www.loom.com/share/c7b1c559346e4a4c8df4e670ef14b558
Screenshots

Make sure to go ahead & sign the cla @xoscar

Thanks for your contribution, @xoscar!
Would you update the PR title to capture the intention? It's hard to understand what "39 web application" is unless someone reads through the details (and later it will cause pain if someone want to search old PRs).

Could we get the checks passing on this to move it forward?

I believe that we can also get rid of the social network icons, we are not using them within the FE.

I didn't mark the PR as need work because I'm will be OoO for some time, but went through all files and added my review!
This was a really massive one 😅

Ah, btw...
The loadgen is failing because the FE url is different now.
Maybe we could consider either moving from 8081 to 8080 or updating the loadgen endpoint as well.

Ah, btw... The loadgen is failing because the FE url is different now. Maybe we could consider either moving from 8081 to 8080 or updating the loadgen endpoint as well.

I fixed the endpoint for the loadgen but the scenarios are no longer "valid" as the app does not support the same POST/GET endpoints. I'll update it with better next as part of a follow up PR

@wph95 , @cedricziel , @mic-max , @open-telemetry/demo-approvers - ready for final reviews

Hey @mic-max thanks for reviewing the PR 😁 I just updated it based on your changes, let me know if that's good enough to merge the changes. @cartersocha I think we need to press the button one last time 😄

@wph95 , @cedricziel , @mic-max , @open-telemetry/demo-approvers - ready for final reviews

I don't know how to effectively review any PR that has more than a hundred files. Maybe it should be changed to a series of smaller PRs?

@wph95 , @cedricziel , @mic-max , @open-telemetry/demo-approvers - ready for final reviews

I don't know how to effectively review any PR that has more than a hundred files. Maybe it should be changed to a series of smaller PRs?

The idea of one big PR was to not have intermediate commits where the frontend might be broken, this PR adds all of the features the GO version used to have plus the new redesigns.
The next ones should be fairly small as we will only focus on one section like:

Instrumentation.
E2E tests.

Thanks @xoscar! Huge contribution 🌟

Thank you @xoscar for this!!! 💯


"Terms defined by reference" using wrong spec names

https://infra.spec.whatwg.org/#index-defined-elsewhere says [encoding-1] instead of [ENCODING], mismatching https://infra.spec.whatwg.org/#informative

This may also be related to things going in the wrong category of informative vs. normative references https://github.com/whatwg/infra/pull/156#issuecomment-333126813

Yup, these are slightly different manifestations of the same bug; the latter is being tracked with more detail in #1106.

Looks like both bugs are fixed now, yay!


More details about the training implementation

Hi, thanks for your neat and valuable work! By the way, I would like to know about the training batch size for KITTI and Waymo.
According to the paper, it seems that the batch_size for PV-RCNN on KITTI and Waymo are set as 8 and 64 respectively. I would like to make sure these parameters are right as I can not find them in the configuration files.
By the way, have you tested the influence of batch size on KITTI or Waymo?

Moreover, have you used the road map data for KITTI dataset?

Hi, I use Batch 8 for KITTI (without using road map) and do not perform experiments on Waymo dataset. And I do not test other batch size for experiments.

Got that, thanks!

Hi, I use Batch 8 for KITTI (without using road map) and do not perform experiments on Waymo dataset. And I do not test other batch size for experiments.

So, you've also set batch size as 8 for all experiments. Right?

Table 9.

Hi, I use Batch 8 for KITTI (without using road map) and do not perform experiments on Waymo dataset. And I do not test other batch size for experiments.

For the result in Table 9, it that model trained with road map info leveraged? Actually, I failed to reproduce that performance without using road map even if I eval all the ckpts from epoch1 to epoch80 considering that the performance deviation may exist. Instead, when I use the road map info (by enabling it in the config file), I can get a slightly-lower but similar result as stated in Table 9.

By the way, I notice that the "road map" is always set as "True" in the configuration files given in this repo. So I would like to  check whether the road map info is used for the experiments listed in your main paper. Thanks!

Sorry for the late reply. I didn't find the response updated here. I checked the code and did not find the "road map" in the config. Can you point out what the "road map" is for me?

Oh, I mean the "ROAD_PLANE" configuration parameter. For example, the one in the config file below.
https://github.com/dvlab-research/VFF/blob/1e95c52d0bf35c872cde9227e9ab4fcdb1fd7fdb/tools/cfgs/kitti_models/VFF_VoxelRCNN.yaml#L21

Hi, this is only used to put the sampled GT instance into the current scenes, like that in https://github.com/dvlab-research/VFF/blob/54ff6ef470d0bfbe0dda0c2598ba31e12dfcf7a8/pcdet/datasets/augmentor/database_sampler.py#L205

Got that, thanks! I have another question for you. If I would like to use the DB-sampler on the NuScenes dataset, how can I get the road_plane info? Could you give me any suggestion on that? Thanks!

Hi, maybe you can refer to this repo. I implement the cross-modality DB sampler for nuScenes dataset.

Got that, thanks!


Listener doesn't output sound if there's no Camera on the scene

Hit this when trying to use AudioStreamPlayer3D in a 2-D world to get its advanced panning/rotation support. Essentially I'm syncing the X and Z coordinates of AudioStreamPlayer3D nodes with the X and Y coordinates of a Node2D parent, as well as their rotations. This seems to work OK for sound sources, but it failed for Listener and took days plus help to debug.
Essentially, I can't have a Listener as a child of a Node2D (or even a Spatial) and get audio. Instead, I have to:

Create a separate Viewport.
Create a Camera as a child of the viewport. I don't want 3-D rendering, so I hope this prevents any visible rendering.
Create a Listener as a child of that camera.
Enable the 3-D listener property on the parent Viewport.

Once I've done that, I can sync AudioStreamPlayer3D nodes with a Node2D parent. I can also sync the Listener with another Node2D, but I can't use the parent-child relationship. I have to tag it.
I looked through the listener source, and don't immediately see where it would fail if there isn't a Camera in its ancestry.
Here is a fairly minimal reproduction of what I tried and failed. The reproduction uses a Spatial as the Listener parent, but it'd also be nice if I could use a Node2D. I figured a Spatial parent would be the least likely to fail though, but it did.
Using a custom built Godot 3.1.1.
Thanks.

The title is inaccurate. This happens if there's no Camera in the scene, so 3D rendering is inactive. This might be intended and just needs being documented.

I think @Calinou may have mentioned wanting to add a warning if there are 3D nodes in the scene but no 3D camera. Is that an appropriate way to address this? I don't want users spinning their wheels trying to figure out why the pattern they used for 2D doesn't work in 3D, but at the same time I think that the current behavior does make sense to keep. So documenting it and maybe adding a warning seems like the best plan here.

There's absolutely a use-case for sound-only 3D. @ndarilek can expand on why if he's available but the basic version is that Godot does not support first-person audio listeners in top-down 2D games. I wish I was around for more of the 4.0 development cycle because there are so many things I'd like to rework. That's something that I wish we could support, for example.

I think we only have the Doppler effect in 3D, so having support for audio-only 3D makes sense too.

Sound-only 3D can be used to make games suitable for visually impaired players, such as AudioQuake, where sounds are used as spatial cues instead (or additionally to) 3D models. If this can be supported that's definitely nice to have.

Perhaps listeners should also work independently of cameras?


Added INTRODUCED to compliment FIX

As per https://github.com/ossf/osv-schema/issues/125#issuecomment-1486206362

Also can you cut a 1.4.1 release with this? I need this ASAP to convert all our Linux Kernel entries to 1.4.1 (with INTRODUCED in the URLs which 90% have or so)

Also can you cut a 1.4.1 release with this? I need this ASAP to convert all our Linux Kernel entries to 1.4.1 (with INTRODUCED in the URLs which 90% have or so)

Will do, once we get this merged. Just waiting for a second approval from @chrisbloom7 :)

Fixes #125

Also, the next version should be 1.5.0 (rather than 1.4.1) as these changes introduce new features.


[MXSession resetRoomsSummariesLastMessage] freezes the app

In https://github.com/matrix-org/riot-ios-rageshakes/issues/19, the system detects a time change (which is unexpected) which leads to the call of [MXSession resetRoomsSummariesLastMessage] that seems to freeze the app.
This call also generates a unexpected load of [MXFileStore commit] whereas only one should do the job.

done


Crashed application RTMP || IP camera

Hi all.
I'm writing an application, I used your libraries to transfer video to the server by RTMP.
Has connected all as it is necessary in a gradle and the manifest.
There are MainActivities and SurfaceView, I wanted to start broadcasting video from SurfaceView directly to the server, as in your examples. But there was an error and the application closes as soon as it opens.
He swears on two lines:
        // create builder
        RtmpCamera1 rtmpCamera1 = new RtmpCamera1 (videostreamPreviewSf, (ConnectCheckerRtmp) this);
        // start stream
        rtmpCamera1.startStream ("rtmp: //adress...");

The error is here:
(ConnectCheckerRtmp) this

You need implement ConnectCheckerRtmp in your class.

@pedroSG94
I have certain events in SurfaceView that are not related to the camera, but I want to transfer them to the server via RMTP.
RtmpCamera1 is not suitable, because she is for working with the camera
Advise me what to use?

You can do a "hacky way". You can do stream with openglview and use AndroidViewFilter to send other view of course this will use camera but your view will overlay it so you can set same size to openglview than your surfaceview and overlay all camera content.
You can see OpenglRtmpActivity to know how to use openglview and androidviewfilter. If you need more info you can see this issue still closed:
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/issues/125

@pedroSG94
Sorry that I'm flooding this thread.
The matter is that in SurfaceView I have a translation from another camera and I need to transfer it to the server. OpenglRtmp - I understand it works with objects

If your camera is device camera (You still have device camera in use so you can't open with the library) nothing to do by default. You need modify the library.
If camera is like a IP camera and you want re stream it to other server you can try use AndroidViewFilter like I said but if not work you can modify the project to stream using opengl to copy surface frames to surface from VideoEncoder class (like I do using OpenglView that copy camera frames in surfaceview surface and in VideoEncoder surface) or send h264 and aac bytebuffer directly to rtmp or rtsp modules.
Other way (maybe not useful for you is stream your device screen like in DisplayRtmpActivity class

@pedroSG94
I decided to use RtmpDisplay - it's quite a convenient thing.
At the debugging stage, everything went fine until he started complaining about the microphone

This look audio permission missed. Remember get permissions on runtime if you use API23+

Settings gradle I have such

conditions added
            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
                if (rtmpDisplay.prepareAudio() && rtmpDisplay.prepareVideo()) {
                    rtmpDisplay.startStream("rtmp://adress...", resultCode, data);
                }
            }

No, no. You need write code to popup permission on runtime if your device use android 6.0+.
Also you need write audio permission in manifest. Please see MainActivity in this project to know how to do runtime permissions. If you don't understand the activity search in google a more easy example.

@pedroSG94
Thanks for answers!
At the moment, fixed this by turning on Microfan for the application via Settings

I learned how to process h.264 video and display it on my screen from an IP camera.
By their own methods, I wanted to use your example for streaming to the server.
I want to achieve the same result as you RtmpCamera1.
but I just send this video to the server, like .startStream (url);
RtmpDisplay unfortunately did not come up to me
Can you suggest something? pls

You need use SrsFlvMuxer class for it. See this class: RtmpCamera1
The idea is get h264 frames as ByteBuffer and MediaCodec.BufferInfo.

ByteBuffer: This contain the frame of h264.
MediaCodec.BufferInfo contain info about h264 frame If you have not it You can simply create one and override size and presentationTimeUs parameters (size is the size of h264 frame, presentationTimeUs is the time when frames was created. You have a example of how to override timeUs in VideoEncoder class).

Useful extra info:

You can stream without audio so you can discard all related with audio in RtmpCamera2 class.
Set pps and sps is not necessary SrsFlvMuxer will get it for you if anyone is setted.

@pedroSG94
All of the above is already implemented, below is a snippet of code. It is called when the camera is connected
        // The callback for receiving the raw H264 video data for camera live view
        mReceivedVideoDataCallBack = new VideoFeeder.VideoDataCallback() {

            @Override
            public void onReceive(byte[] videoBuffer, int size) {
                //Log.d(TAG, "camera recv video data size: " + size);
                if (useSurface) {
                    VideoStreamDecoder.getInstance().parse(videoBuffer, size);
                } else if (mCodecManager != null) {
                    mCodecManager.sendDataToDecoder(videoBuffer, size);
                }

            }
        };

If I understand correctly, do I need to use the SrsFlvMuxer class functionality to transfer video over RTMP?

Yes. You need use this method:
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/rtplibrary/src/main/java/com/pedro/rtplibrary/rtmp/RtmpCamera1.java#L95
Remember start/stop srsFlvMuxer too:
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/rtplibrary/src/main/java/com/pedro/rtplibrary/rtmp/RtmpCamera1.java#L69
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/rtplibrary/src/main/java/com/pedro/rtplibrary/rtmp/RtmpCamera1.java#L80

@pedroSG94
There is a separate class of VideoStream, where I found the time to implement it
rtmp.getH264Data (buffer, bufferInfo);
and there is a MainActivity, where I create a surfaceView, initialize the scope and run
.startStream ("rtmp: // ...");
How to connect all this, so that everything will work now?)

I don't understand you but if you can use sendVideo and start methods of SrsFlvMuxer class, stream should work.

@pedroSG94
I use the camera c documentation for dji and when the application tries to decode using the function getH264Data (..., ...);
see line 640. ( https://pastebin.com/ha99q0DB )
Slightly changed the function getH264Data, removed all unnecessary there and left
   @override
   public void getH264Data (ByteBuffer h264Buffer, MediaCodec.BufferInfo info) {
     if (Build.VERSION.SDK_INT> = Build.VERSION_CODES.JELLY_BEAN_MR2) {
         mediaMuxer.writeSampleData (videoTrack, h264Buffer, info);
     }
     getH264DataRtp (h264Buffer, info);
   }

Now in my SurfaceView I get a black screen and nothing is sent to the server anyway :-(

I have a similar case when I use file stream.
I recommend you first render a surface of surfaceview with decoder to test that you are doing it correctly. And then render a VideoEncoder surface like in  my decoder rendering and send this data to stream.
You can find surface to render with decoder in this line, try emulate this class to decode your frames:
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/encoder/src/main/java/com/pedro/encoder/input/decoder/VideoDecoder.java#L69
If it works correctly you can now encode frames using surface of VideoEncoder class like in this method:
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/encoder/src/main/java/com/pedro/encoder/video/VideoEncoder.java#L342

@pedroSG94
Swears on VideoEncoder on line 309. ( https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/encoder/src/main/java/com/pedro/encoder/video/VideoEncoder.java#L309 )
The code in MainActivity when initializing objects is as follows
         rtmpCamera1 = new RtmpCamera1 (videostreamPreview, this);

         if (Build.VERSION.SDK_INT> = Build.VERSION_CODES.LOLLIPOP) {
             rtmpCamera1.startStream ("rtmp: // adress ...");
         }

         if (Build.VERSION.SDK_INT> = 21) {
             videoEncoder = new VideoEncoder (rtmpCamera1);
             videoEncoder.getDataFromSurfaceAPI21 ();
             showToast ("Run in 21 SDK");
         }

This error look a o prepared VideoEncoder:
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/encoder/src/main/java/com/pedro/encoder/video/VideoEncoder.java#L69

/*Change this to other method. Use your own configuration.
    You don't need use MediaExtractor because you should know stream resolution, etc.
    The idea is create your own MediaFormat to avoid nullpointer in prepareVideo method*/
    videoDecoder.initExtractor();

I do not fully understand what should be in videoDecoder.initExtractor ();
in videoDecoder initExtractor inherits String, and MediaFormat is a different data type.
I got confused about how to use initExtractor (
help ))

You need create your own MediaFormat like in VideoEncoder class:
https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/encoder/src/main/java/com/pedro/encoder/video/VideoEncoder.java#L101

@pedroSG94
I already had something similar from DJI
Used in the next class
Line 16 and 20
( https://pastebin.com/w30eNCiv )

initExtractor should look like:
  public boolean initExtractor() throws IOException {
    decoding = false;
    videoFormat = MediaFormat.createVideoFormat("video/avc", 1088, 1088); //use same resolution than videoEncoder class
    mime = videoFormat.getString(MediaFormat.KEY_MIME);
    return true;
  }

@pedroSG94
First question.
It seems that everything was fine, but now when you try to create a connection through rrtmp.start ("rtmp: // ....");
@pedroSG94
An error occurs that references VideoDecoder.java 79 line

Second question
The sendH264Buffer(ByteBuffer byteBuffer) method is analogue to decodeVideo ()?
If so, where and how should I call it? or he himself called when I go to the class where he is?
Displays as an inactive class

I see multiple errors in decoder. Also I can see that you not prepared VideoEncoder class as you can see with log:
VideoEncoder need be prepared, VideoEncoder not enabled

Error in decoder:

You need call initExtractor, prepareVideo and then start decoder in this order.
Start method in VideoDecoder should be changed You need use decodeVideo when you receive a frame from your camera not with a loop in a thread and send byte[] to this method to feed this buffer and delete all related with mediaextractor in this method.

I understand what you want to explain to me, but the fact is that my level of work with Media resources is not so big, so you are my hope :) Forgive me if I ask stupid questions
Instead of decodeVideo(), I should now use sendH264Video(byte[] ByteBuffer).
i.e. sendH264Video() should be almost the same as in the DJIVideoDecodeStream class as a decodeFrame()
There is a difference between decodeFrame() and sendH264Video () - this is what the first uses DJIF library inputFrame = frameQueue.poll (); and the like, and in the second case (in our) VideoExtractor
I really hope for your help.
VideoDecoder.java ( https://pastebin.com/L00CcZ2S )
DJIVideoStreamDecoder.java ( https://pastebin.com/jrSAG48Q )
rRTMP.java (this is ur old TestH264Stream) ( https://pastebin.com/DErXLUPS )

Your DJIVideoStreamDecoder is decoding from a surface if you set it a not null surface so you don't need my VideoDecoder.
Also rRtmp class should be used outside of DJIVideoStreamDecoder.
So I think you have 2 ways to do it:

Get yuv data from your IYuvDataListener interface and use this data like I do in RtmpCamera1base. To do it you need know yuv frame type (NV21, YV12, I420, NV12, etc).
Set a valid surface using VideoEncoder surface and send it to DJIVideoStreamDecoder using "init(Context context, Surface surface)" method (This way is most efficient).

This is a little code that could work for you:
public class TestH264Stream implements GetH264Data, ConnectCheckerRtmp {

  //private VideoDecoder videoDecoder;
  private DJIVideoStreamDecoder decoder;
  private VideoEncoder videoEncoder;
  private SrsFlvMuxer srsFlvMuxer;
  private int width, height;

  public TestH264Stream() {
    //videoDecoder = new VideoDecoder(this, this);
    decoder = DJIVideoStreamDecoder.getInstance();
    videoEncoder = new VideoEncoder(this);
    srsFlvMuxer = new SrsFlvMuxer(this);
  }

  public void prepare(int width, int height) {
    this.width = width;
    this.height = height;
    videoEncoder.prepareVideoEncoder(width, height, 30, 1200 * 1024, 0, false, 2,
        FormatVideoEncoder.SURFACE);
    /*Change this to other method. Use your own configuration.
    You don't need use MediaExtractor because you should know stream resolution, etc.
    The idea is create your own MediaFormat to avoid nullpointer in prepareVideo method*/

  }

  public void start(Context context, String url) {
    videoEncoder.start();
    decoder.init(context, videoEncoder.getInputSurface());
    srsFlvMuxer.setVideoResolution(width, height);
    srsFlvMuxer.start(url);
  }

  public void stop() {
    decoder.stop();
    videoEncoder.stop();
    srsFlvMuxer.stop();
  }

  //callback to get frame info. This should be called a time with the first encoded frame.
  @Override
  public void onSPSandPPS(ByteBuffer sps, ByteBuffer pps) {
    srsFlvMuxer.setSpsPPs(sps, pps);
  }

  //callback called when a frame is encoded.
  @Override
  public void getH264Data(ByteBuffer h264Buffer, MediaCodec.BufferInfo info) {
    srsFlvMuxer.sendVideo(h264Buffer, info);
  }

  @Override
  public void onVideoFormat(MediaFormat mediaFormat) {
    //useful is you want record. See some Base to know about it.
  }

  @Override
  public void onConnectionSuccessRtmp() {
    //do something
  }

  @Override
  public void onConnectionFailedRtmp(String reason) {
    //do something
  }

  @Override
  public void onDisconnectRtmp() {
    //do something
  }

  @Override
  public void onAuthErrorRtmp() {
    //do something
  }

  @Override
  public void onAuthSuccessRtmp() {
    //do something
  }
}

To do second way you need use rRtmp and replace VideoDecoder to DJIVideoStreamDecoder.

Really working option.
There is a small problem when I initialize the connection to MainActivity. On my SurfaceView on the phone, the video from the camera stops showing, but broadcast to the server is sent. True in poor quality, green lines appear and ruffle
Line 200 - 202. Maybe I'm in the wrong place declaring?
( https://pastebin.com/L0FTCf8m )
p.s. video in media server

Lose preview in surfaceview is normal because you replace surface of your surfaceview to surface of VideoEncoder that send to media server.
If you want both as same time you need use opengl to copy surface frames like I do in OpenGlView (it is not easy if you don't know opengl).
And bad image result can has multiple reason but my first suspect is line 200 that set resolution of VideoEncoder(I think). This resolution should be original resolution of camera that you get not surfaceview size.
Other possibility could be some bad configuration on decoder or encoder(color format is a possibility) or maybe glitch on your devices mediacodec or opengl(it is frequent on custom rooms and rare but possible with officials rooms)

Sent by mail, can look =)

HI @Rishat7c
You could provide how your solution is. The links to https://pastebin do not work anymore.
Thanks


Can't recognize videos

There is a moodle course with around 50 videos, this extension of course doesn't go beyond the resources of page 1, so it can't find them and download them :(

Hi @Kreijstal,
While it doesn't support downloading videos, the extension does support downloading resources from all the Resources page. If your course url is as so https://lms.monash.edu/course/view.php?id=42395&section=13#13, replacing view with resources and dropping the section param will get you there https://lms.monash.edu/course/resources.php?id=42395
I'll close this for now but feel free to reopen it should you have any further questions

Hi @Kreijstal,
While it doesn't support downloading videos, the extension does support downloading resources from all the Resources page. If your course url is as so https://lms.monash.edu/course/view.php?id=42395&section=13#13, replacing view with resources and dropping the section param will get you there https://lms.monash.edu/course/resources.php?id=42395
I'll close this for now but feel free to reopen it should you have any further questions

that's ok https://github.com/marcelreppi/moodle-buddy can download videos, that's great!

that's ok https://github.com/marcelreppi/moodle-buddy can download videos, that's great!

It looks good! Didn't know it was a thing, thanks for pointing it out :)

It looks good! Didn't know it was a thing, thanks for pointing it out :)


Removing extra/erroneous line in ADLS .NET SDK getting started article

Removing extra/erroneous line in ADLS .NET SDK getting started article.

Hi @matt1883, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

It looks like you're working at Microsoft (mahi). If you're full-time, we DON'T require a contribution license agreement.

If you are a vendor, DO please sign the electronic contribution license agreement. It will take 2 minutes and there's no faxing! https://cla.azure.com.

TTYL, AZPRBOT;

#sign-off

@matt1883 : Thanks for your contribution to the Azure documentation! The author, @nitinme, has been notified to review your proposed change.


Derive poem_openapi::Object for structs containing Protobuf well known types

I would like to have the option to derive the poem_openapi::Object for structs generated from a GRPC definition. These sometimes contain Protobuf Well Known Types.
Specifically, the ones that map to serde_json::Value would be very handy to have.
pub struct GrpcGeneratedExampleStruct {
    #[prost(uint32, repeated, tag = "1")]
    pub field1: ::prost::alloc::string::String,
    #[prost(message, optional, tag = "2")]
    pub properties: ::core::option::Option<::prost_wkt_types::Struct>,
}

If I could derive Object on this struct, I could instantly pass the struct through to a GRPC microservice from a standard HTTP one. This would greatly reduce the boilerplate code I have in place now.

Implementation has been merged.