Engine: BSD: Address SIGSEGV error on Loading / Creating a Project

valgrind-log.zip
Description
The editor partially loads the project and then crashes after a few seconds. It immediately crashes after project creation.
Prerequisites
Was on the project selection screen waiting to open a project.
Steps to Reproduce

Build latest source code with CMake.
Install to the SDK build directory using gmake install
Run WorldEditor from inside of install-root/sdk/2024.2/freebsd/x86/bin/ directory
Load up or create new project.

Expected behavior:
Full editor loads.
Actual behavior:
Crashes while loading editor and engine.
Reproducibility:
Consistently.
Versions

Additional Information
I decided to run both gdb and Valgrind to get to the bottom of this. Attached is the Valgrind log.
Note the Invalid write of size 8 on line 400 and then the Address 0x70 is not stack'd on line 413. There's other errors but MaterialInstance::setTransform is the crashing function in both debugger applications.

I'm not fully sure how to fix this yet but this may be related to the crashes others have been running into.

MaterialInstance::setTransform is a simple function. I can assume it's tries to call setTransform with nullptr of MaterialInstance object? Could you show a call stack?

Seems like the issue in Renderable::setMaterialsList some materials was nullptr and have not created an instance for it. Which leads to nullptr exception

Have added a checker. But I assume some standard resources was not installed properly because of CMake build

That's part of why I might attempt to build Qt from source. Many of the libraries (especially the standard ones outside of the current build system) are ports of the Linux ones. I noticed several Qt library related errors in that log

it's not related Qt. Please check resources dir in SDK

Running gmake install produces this:
CMake Error at build/cmake_install.cmake:45 (file):
  file INSTALL cannot find "/plugins/imageformats": No such file or
  directory.
Call Stack (most recent call first):
  cmake_install.cmake:48 (include)

and then it complains about not having a local Qt installation and errors out with that too.

it's CMake issues imageformats is directory not file

Thank you for your patience, by the way. We got this. I implemented your current commit manually and still getting a crash but also it loaded a bit further. Debugging now

.embedded is "virtual" directory. it's means this resource located NOT in project directory. It has many inputs most of them in "resources" directory. Check SDK build from the archive which I mentioned yesterday

It's not in the Linux one. I'll try from the Windows archive.

I mean the .embedded doesn't exist. Should it? I've already copied that folder from the Linux archive. Copying from the Windows one to see if there's a difference

Implementing your fix and adding a couple more checks, I was able to get this stacktrace with gdb:
0x0000000823e8c516 in PipelineContext::draw(Camera*) ()
   from /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/../lib/libengine-editor.so
(gdb) backtrace
#0  0x0000000823e8c516 in PipelineContext::draw(Camera*) ()
    at /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/../lib/libengine-editor.so
#1  0x0000000823ef3246 in RenderSystem::update(World*) ()
    at /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/../lib/libengine-editor.so
#2  0x000000089dad2358 in RenderGLSystem::update(World*) ()
    at /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/plugins/librendergl-editor.so
#3  0x0000000823ef370a in RenderSystem::renderOffscreen(World*, int, int) ()
    at /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/../lib/libengine-editor.so
#4  0x000000089dad23cb in RenderGLSystem::renderOffscreen(World*, int, int) ()
    at /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/plugins/librendergl-editor.so
#5  0x000000000032c6d1 in IconRender::render(QString const&, QString const&) ()
#6  0x0000000000332282 in ImportQueue::onImportFinished() ()
#7  0x0000000000333f2b in QtPrivate::FunctorCall<QtPrivate::IndexesList<>, QtPrivate::List<>, void, void (ImportQueue::*)()>::call(void (ImportQueue::*)(), ImportQueue*, void**) ()
#8  0x0000000000333e9d in void QtPrivate::FunctionPointer<void (ImportQueue::*)()>::call<QtPrivate::List<>, void>(void (ImportQueue::*)(), ImportQueue*, void**) ()
#9  0x0000000000333de2 in QtPrivate::QSlotObject<void (ImportQueue::*)(), QtPrivate::List<>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) ()
#10 0x000000082e656672 in ??? () at /usr/local/lib/qt5/libQt5Core.so.5
#11 0x0000000823f19fd5 in AssetManager::importFinished() ()
    at /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/../lib/libengine-editor.so
#12 0x0000000823e03a5f in AssetManager::onPerform() ()
    at /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/../lib/libengine-editor.so
#13 0x0000000823f19b08 in AssetManager::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) ()
    at /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/../lib/libengine-editor.so
#14 0x000000082e656736 in ??? () at /usr/local/lib/qt5/libQt5Core.so.5
#15 0x000000082e65b803 in QTimer::timerEvent(QTimerEvent*) () at /usr/local/lib/qt5/libQt5Core.so.5
#16 0x000000082e64b5b7 in QObject::event(QEvent*) () at /usr/local/lib/qt5/libQt5Core.so.5
#17 0x0000000827c99b43 in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
    at /usr/local/lib/qt5/libQt5Widgets.so.5
#18 0x0000000827c9aeee in QApplication::notify(QObject*, QEvent*) ()
    at /usr/local/lib/qt5/libQt5Widgets.so.5
#19 0x000000082e622df1 in QCoreApplication::notifyInternal2(QObject*, QEvent*) ()
    at /usr/local/lib/qt5/libQt5Core.so.5
#20 0x000000082e6748b4 in QTimerInfoList::activateTimers() () at /usr/local/lib/qt5/libQt5Core.so.5
#21 0x000000082e676381 in ??? () at /usr/local/lib/qt5/libQt5Core.so.5
#22 0x0000000837a13ee1 in ??? () at /usr/local/lib/libglib-2.0.so.0
#23 0x0000000837a1428f in ??? () at /usr/local/lib/libglib-2.0.so.0
#24 0x0000000837a14326 in g_main_context_iteration () at /usr/local/lib/libglib-2.0.so.0
#25 0x000000082e675915 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/local/lib/qt5/libQt5Core.so.5
#26 0x000000082e61f786 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) ()
    at /usr/local/lib/qt5/libQt5Core.so.5
#27 0x000000082e623471 in QCoreApplication::exec() () at /usr/local/lib/qt5/libQt5Core.so.5
#28 0x000000000030d5d4 in main ()

I can assume pipeline was not fully loaded. Could you check if pipeline tools plugin loaded? Pipeline is a special game resource which describes the order to render image and tasks were needed to render it.

 [PluginManager] Can't load plugin: /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/plugins/libpipelinetools.so
 [PluginManager] Can't load plugin: /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/plugins/libshadertools.so
 [PluginManager] Can't load plugin: /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/plugins/libmotiontools.so

Right on the money.

When I get off of work, I found some more places some checks can be added. Probably won’t fully solve the underlying problem but your suggestions have given me a direction on what to look for while I have downtime at work.

I thing you should check dependencies for this plugins using ldd or analog on BSD platform

Got it. I’ll still implement those checks locally since they won’t harm anything. There’s no library linkage errors left for the builds so far but I can recheck after work.
My assumptions are it’s the platform and system libraries first over your code for the same reasons you’ll suggest it’s directly related up your code and your build system. I’m enjoying work on this, to be honest. Thank you.😎

Yes it's bit addictive =)

Took a glance at the CMake files for all the unloaded plugins and they should link correctly. graph-editor is a dependency in all. I don’t have a Linux install right now but building a version with CMake and running natively from those builds could rule out if it’s the platform or  the CMake build system. Getting a VM setup with an install to test on should be doable on my end. It doesn’t even need to run well, just be able to build and launch the editor.

That stacktrace brought up a Camera pointer. There’s at least one unchecked one in Pipelinecontext that could be null. I took a brief look at the RenderSystem RenderGL code and didn’t see anything right off the bat.
I’ll check the fonts too. The crash happens right after they get loaded in and sometimes they and certain image formats will get mangled by git when cloning from my experiences. It’s a known bug that pops up every so often with the git port on the platform.

When you setup the CMake system, did you assume a build inside tree or in an external folder? I think I found the problem.
I’m also confused about how the engine creates the virtual resource directory. I see the code looking for the resources and then assuming .embedded exists in memory. Definite black magic to me. That’s why I wasn’t sure if I should look for a hidden folder in the resources folder or not.

I think you should check dependencies for this plugins using ldd or analog on BSD platform. They all uses grapheditor module

Just got home from work. Here's the BSD ldd logs
ldd-logs.zip
Everything is linked correctly as it stands.

My current assumption is that each of the failed plugins generated a warning of some type that I mindlessly didn't think much of. I'm building them individually and piping the warnings to a log. I'm also comparing my findings to the previous Linux build logs from your runners. I found some possible leads. Will post my findings here in a bit.

This doesn't exist in the other logs and is mighty suspicious to me:
/home/brad/git/thunder/engine/includes/editor/converters/animconverter.h:0: Note: No relevant classes found. No output generated.
Here's the source code for that: https://github.com/thunder-engine/thunder/blob/master/engine/includes/editor/converters/animconverter.h
pipeline-warnings.log
And attached is the warning log for pipelinetools. It's not generating all of the headers even though they have no issues with the Linux build.

When you setup the CMake system, did you assume a build inside tree or in an external folder? I think I found one of the problems.
I’m also confused about how the engine creates the virtual resource directory. I see the code looking for the resources and then assuming .embedded exists in memory. Definite black magic to me. That’s why I wasn’t sure if I should look for a hidden folder in the resources folder or not.

I think I forgot to prepare right install instructions fro CMake. Me personally using QBS because it's convenient to use it in Qt Creator IDE

I think you should check dependencies for this plugins using ldd or analog on BSD platform. They all uses grapheditor module

Just got home from work. Here's the BSD libraries, plugins and executables ldd logs. There's nothing that's linked wrong as far as I can see. ldd-logs.zip
ldd-exe.log

Ok seems like all dependencies were resolved. Then it's missilery why it don't loading =/

This doesn't exist in the other logs and is mighty suspicious to me:
/home/brad/git/thunder/engine/includes/editor/converters/animconverter.h:0: Note: No relevant classes found. No output generated.
Here's the source code for that: https://github.com/thunder-engine/thunder/blob/master/engine/includes/editor/converters/animconverter.h
pipeline-warnings.log
And attached is the warning log for pipelinetools. It's not generating all of the headers even though they have no issues with the Linux build.
Seems to be not relevant or useful. It's just a standard moc warning stating that it doesn't need to process that header.

I don't fully understand what relevant classes it's looking

When you setup the CMake system, did you assume a build inside tree or in an external folder? I think I found one of the problems.
I’m also confused about how the engine creates the virtual resource directory. I see the code looking for the resources and then assuming .embedded exists in memory. Definite black magic to me. That’s why I wasn’t sure if I should look for a hidden folder in the resources folder or not.

I think I forgot to prepare right install instructions fro CMake. Me personally using QBS because it's convenient to use it in Qt Creator IDE

Honestly, I'm more comfortable with CMake and alternative systems like Facebook's Buck. I completely understand. Qt is very new to me overall so scripting with qbs is a bit of trial and error for me.

I would like to improve CMake with your help if u don't mind. It's a good feedback from you! Thank you!

Gladly. I won't have time this morning to fully work on #783 but getting the build systems worked out properly would greatly improve the project for users. Until the BSD port is in a more stable state, I won't be looking at porting that part of the build to qbs just yet. I don't want to introduce too many bugs to the other platforms and qbs not having an official binary build on FreeBSD (specifically) could mean problems for the end user.

I think you should check dependencies for this plugins using ldd or analog on BSD platform. They all uses grapheditor module

Just got home from work. Here's the BSD libraries, plugins and executables ldd logs. There's nothing that's linked wrong as far as I can see. ldd-logs.zip
ldd-exe.log

Ok seems like all dependencies were resolved. Then it's missilery why it don't loading =/

I'm still looking further into this. My current new assumption is that either there's a bug in graph-editor that's depended on only by the unloaded plugins or one of the thirdparty dependencies they all share is causing problems.
An example test I've done so far that failed: I upgraded physfs to 3.2.0. It's fully backwards compatible with the original version used (it was created for use with I want to say the Linux port of Unreal Tournament 2003 - could be remembering wrong), so its maintainer (icculus) has kept it backwards compatible to support his old port jobs and a large number of retro and modern systems. That's heavily used in the resource loading system for Thunder.

graph-editor depending on uikit and qt.

I was meaning you may have been on point with asking me to check graph-editor. All of the failed plugins depend on graph-editor but it's also possible it could be uikit or one of the shared third-party dependencies. I like to make broad assumptions and then narrow them down.

is it possible to check plugins dependencies in hierarchy way

I can try. It's definitely a weakness.

Uikit isn't building properly is what it looks like. That plugin must be loaded first and placing it next to WorldEditor (to be comparible to how it is for Windows), it's still not loading. Your latest PR did push the loading screen to about 75% though.

I don't know how useful this is but all of the plugins that won't load (minus UIKit) have errors where the module name and information is in the compiled library. These are from the compiled debug build of each. With how modules are loaded, I noticed that you seem to look for just the basic information and then load from there. I could be misreading, but your code flows well.
dump.txt
These only seem to exist for these plugins but I haven't dove too far into all of them. These also don't exist in the Linux or Windows counterparts from the Github runner builds.

It's Independence Day here but I didn't feel like going out. I took a several hour nap instead. I had this dream about a possible fix and I'm testing it out right now. The runtime path (RPATH) for Linux and FreeBSD must be configured slightly differently.
Example: a library needs to call another library in its same directory. Linux automatically links them correctly. FreeBSD didn't in this case, hence a possible reason for the plugins not properly loading. What I've noticed is they don't load when I run from the sdk build but do run when running from the build directory before install.
FreeBSD
brad@5k-lame-brain ~/g/t/i/s/2/f/x/b/plugins [1]> ldd libshadertools.so | grep editor
ldd: libshadertools.so: Shared object "libshadertools.so" not found, required by "ldd"
libshadertools.so: exit status 1

Linux
brad@5k-lame-brain ~/D/T/l/r/i/s/2/l/x/b/plugins> /compat/linux/bin/bash ldd libshadertools.so | grep editor
	libgraph-editor.so => /home/brad/Downloads/Thunder/linux/release/install-root/sdk/2024.2/linux/x86_64/bin/plugins/./../../lib/libgraph-editor.so (0x0000000801243000)
	libengine-editor.so => /home/brad/Downloads/Thunder/linux/release/install-root/sdk/2024.2/linux/x86_64/bin/plugins/./../../lib/libengine-editor.so (0x0000000801c00000)
	libnext-editor.so => /home/brad/Downloads/Thunder/linux/release/install-root/sdk/2024.2/linux/x86_64/bin/plugins/./../../lib/libnext-editor.so (0x00000008012da000)
	libuikit-editor.so => /home/brad/Downloads/Thunder/linux/release/install-root/sdk/2024.2/linux/x86_64/bin/plugins/./../../lib/../lib/libuikit-editor.so (0x0000000801a7c000)
	libphysfs-editor.so => /home/brad/Downloads/Thunder/linux/release/install-root/sdk/2024.2/linux/x86_64/bin/plugins/./../../lib/../lib/libphysfs-editor.so (0x000000080139b000)
	libglfw-editor.so => /home/brad/Downloads/Thunder/linux/release/install-root/sdk/2024.2/linux/x86_64/bin/plugins/./../../lib/../lib/libglfw-editor.so (0x00000008024e1000)
	libfreetype-editor.so => /home/brad/Downloads/Thunder/linux/release/install-root/sdk/2024.2/linux/x86_64/bin/plugins/./../../lib/../lib/libfreetype-editor.so (0x000000080283e000)
	libzlib-editor.so => /home/brad/Downloads/Thunder/linux/release/install-root/sdk/2024.2/linux/x86_64/bin/plugins/./../../lib/../lib/../lib/libzlib-editor.so (0x00000008013b2000)

I keep looking at the code and don't see anything that would directly be problematic so far. Doing extra checks for nullptrs hides but doesn't solve the problem of the plugins not loading. Those plugins are most likely why the shader and pipeline files won't load. Fonts and textures are loading though (according to this):
 [PluginManager] Can't load plugin: /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/uikit-editor.so
 [PluginManager] Can't load plugin: /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/plugins/libmotiontools.so
 [PluginManager] Can't load plugin: /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/plugins/libshadertools.so
 [PluginManager] Can't load plugin: /home/brad/git/thunder-build/install-root/sdk/2024.2/freebsd/x86/bin/plugins/libpipelinetools.so
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/editor/gizmos/camera.png
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/editor/gizmos/directlight.png
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/editor/gizmos/pointlight.png
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/editor/gizmos/spotlight.png
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/engine/textures/invalid.png
 Converting: /home/brad/git/thunder-build/install-root/s
dk/2024.2/resources/editor/gizmos/postprocess.png
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/editor/gizmos/soundsource.png
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/engine/textures/ui.png
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/engine/fonts/Roboto.ttf
 [ FileIO ] Can't open file .embedded/DefaultMesh.shader
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/engine/meshes/cube.fbx
 [ FileIO ] Can't open file .embedded/DefaultMesh.shader
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/engine/meshes/plane.fbx
 [ FileIO ] Can't open file .embedded/DefaultMesh.shader
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/editor/meshes/cone.fbx
 [ FileIO ] Can't open file .embedded/DefaultMesh.shader
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/editor/meshes/bone.fbx
 [ FileIO ] Can't open file .embedded/DefaultMesh.shader
 Converting: /home/brad/git/thunder-build/install-root/sdk/2024.2/resources/editor/meshes/sphere.fbx
 [ FileIO ] Can't open file .embedded/DirectLight.shader
 [ FileIO ] Can't open file .embedded/DefaultSprite.shader
 [ FileIO ] Can't open file .embedded/DefaultSprite.shader
 [ FileIO ] Can't open file .embedded/DefaultPostEffect.shader
 [ FileIO ] Can't open file .embedded/Deferred.pipeline
fish: Job 1, './install-root/sdk/2024.2/freeb…' terminated by signal SIGSEGV (Address boundary error)

I have it in a separate branch, but upgrading glslang from 11.8.0 to 11.13.0 didn't solve the shadertools plugin not loading issue. I've gotten some feedback from the BSD porting community on the dumps and outside of the iterator warning, they're probably not related to the core cause. Main piece of feedback that stood out to me:
What I'm saying is valid syntax in one version of the SPIR-V compiler might not be valid in another. That does not necessarily mean you have to upgrade anything, but that might explain the observed difference between platforms.
Getting this solved would still possibly need some shader tweaks to get fully working.

I'm not sure if this is a hit or not, but all of the modules that load are the only modules that share a graph-editor dependency, specifically the abstractnodegraph.h and graphnode.h headers.
brad@5k-lame-brain ~/g/thunder (embedded-black-magic-2)> grep -uwR modules/editor/* --exclude-dir=modules/editor/grapheditor -e "graph" | grep "\.h"
modules/editor/motiontools/editor/animationedit.ui:   <header>editor/graph/graphview.h</header>
modules/editor/motiontools/converter/entrystate.h:#include <editor/graph/abstractnodegraph.h>
modules/editor/motiontools/converter/animationcontrollergraph.h:#include <editor/graph/abstractnodegraph.h>
modules/editor/pipelinetools/editor/pipelineedit.ui:   <header>editor/graph/graphview.h</header>
modules/editor/pipelinetools/converter/pipelinetaskgraph.h:#include <editor/graph/graphnode.h>
modules/editor/pipelinetools/converter/pipelinetaskgraph.h:#include <editor/graph/abstractnodegraph.h>
modules/editor/shadertools/converter/shadergraph.h:#include <editor/graph/graphnode.h>
modules/editor/shadertools/converter/shadergraph.h:#include <editor/graph/abstractnodegraph.h>
modules/editor/shadertools/converter/shadergraph.cpp:#include <editor/graph/nodegroup.h>
modules/editor/shadertools/converter/rootnode.h:#include <editor/graph/graphnode.h>
modules/editor/shadertools/converter/rootnode.h:#include <editor/graph/abstractnodegraph.h>
modules/editor/shadertools/converter/functions/function.h:#include <editor/graph/graphnode.h>
modules/editor/shadertools/editor/materialedit.ui:   <header>editor/graph/graphview.h</header>

Following this particular lead while drinking coffee and getting ready for work.

I can try. It's definitely a weakness.

Have found lddtree utility
https://stackoverflow.com/questions/1488527/hierarchical-ldd1

I don't know how useful this is but all of the plugins that won't load (minus UIKit) have errors where the module name and information is in the compiled library. These are specific dumps from those. With how modules are loaded, I noticed that you seem to look for just the basic information and then load from there. I could be misreading, but your code flows well.
dump.txt
These only seem to exist for these plugins but I haven't dove too far into all of them. These also don't exist in the Linux or Windows counterparts from the Github runner builds.

I have it in a separate branch, but upgrading glslang from 11.8.0 to 11.13.0 didn't solve the shadertools plugin not loading issue. I've gotten some feedback from the BSD porting community on the dumps and outside of the iterator warning, they're probably not related to the core cause. Main piece of feedback that stood out to me:
What I'm saying is valid syntax in one version of the SPIR-V compiler might not be valid in another. That does not necessarily mean you have to upgrade anything, but that might explain the observed difference between platforms.
Getting this solved would still possibly need some shader tweaks to get fully working.

You are not getting even close to shader compilation yet. If plugin "shadertools" will not loaded, it will not start to convert shaders. The update of GLSL version wiil not solve this problem. We need to understand what wrong with load.

I don't know how useful this is but all of the plugins that won't load (minus UIKit) have errors where the module name and information is in the compiled library. These are specific dumps from those. With how modules are loaded, I noticed that you seem to look for just the basic information and then load from there. I could be misreading, but your code flows well.
dump.txt
These only seem to exist for these plugins but I haven't dove too far into all of them. These also don't exist in the Linux or Windows counterparts from the Github runner builds.

I have it in a separate branch, but upgrading glslang from 11.8.0 to 11.13.0 didn't solve the shadertools plugin not loading issue. I've gotten some feedback from the BSD porting community on the dumps and outside of the iterator warning, they're probably not related to the core cause. Main piece of feedback that stood out to me:
What I'm saying is valid syntax in one version of the SPIR-V compiler might not be valid in another. That does not necessarily mean you have to upgrade anything, but that might explain the observed difference between platforms.
Getting this solved would still possibly need some shader tweaks to get fully working.

You are not getting even close to shader compilation yet. If plugin "shadertools" will not loaded, it will not start to convert shaders. The update of GLSL version wiil not solve this problem. We need to understand what wrong with load.

My thoughts exactly. It was feedback I was given for a possible issue that will come after.

I can try. It's definitely a weakness.

Have found lddtree utility
https://stackoverflow.com/questions/1488527/hierarchical-ldd1

Thanks for sharing that. Would that help you out on seeing what’s happening on my end? When you say hierarchy, tree makes more sense to me in this case. I’ll grab this after work. 🤘

Logs says clearly: [PluginManager] Can't load plugin: This means code bellow failed: QLibrary *lib = new QLibrary(path); if(lib->load()) {
Based on my experience with QT it happens when lib dependencies was not fully resolved. I think the issue is in RPATH for some of sub-dependecies. Do u have lddtree ?

Grabbing it after work. You answered my previous question remarkably. RPATH is different between Linux and BSD for sure. I have some ideas after looking at RPATH with readelf

I’m a little surprised you weren’t using https://doc.qt.io/qt-5/qpluginloader.html but at the same time, rolling your own plugin system does give you more control and is one less thing that can fully break your system. I spent most of yesterday following the plugin structure and how you render the ui and everything else.

QPluginLoader is just a wrapper. Plugins in my case bit more specific. I'm not using Qt in game binaries. Editor only. But many modules can be used in game. So I was need more universal system in my case.

Makes perfect sense. Great explanation. I’ve noticed you limit QT usage to where it’s needed but that’s it. I don’t remember if I posted it but the crash stacktrace showed missing Qt implementations (hence part of why I thought about building it from source).

I would like to remove Qt completely one day. But unfortunately it's too far from this.

I’ll still peruse the grapheditor code while at work. I’m following that lead and using it to understand how it works since it’s a common thread that only the broken modules have. The Plugin specific code is uniform across the board, so it’s not an issue.

One thing I’m not sure about is how Qt would break these plugins but not cause the other ones to not load (unless they haven’t loaded because these ones failed to load). One thing that might be good is to implement extra logging for debugging when a plugin doesn’t load. Debug builds are helping some with finding why things aren’t working for me but having the extra information won’t hurt.

QLibrary just using syscalls to load library dynamically. If library can't be loaded (because of dependencies) it just returns nullptr handler. At least texturetools plugins is loading normally. In your case only 3 plugins is failed to load. All 3 has dependency graph-editor. It provides the editor of graphs. grapth-editor uses uikit to visualize this graphs. So I bet the issue is in paths to search this dependencies. probably you can check this two libs and it's rpaths

For your Linux binaries, RPATH directly adds the local QT’s internal gcc/lib directory to it. That’s the only thing I’m observing. Do you have alternatives for that if that isn’t the solution?

FreeBSD
brad@5k-lame-brain ~/g/t/i/s/2/f/x/b/plugins [1]> ldd libshadertools.so | grep editor
ldd: libshadertools.so: Shared object "libshadertools.so" not found, required by "ldd"
libshadertools.so: exit status 1

This is also not an issue. I tested again and the Linux ldd and BSD version handle paths slightly differently.
ldd ./libshadertools.so is the correct way on BSD while ldd libshadertools.so seems valid for the Linux tool.

For your Linux binaries, RPATH directly adds the local QT’s internal gcc/lib directory to it. That’s the only thing I’m observing. Do you have alternatives for that if that isn’t the solution?

The texturetools plugin is also using Qt inside. (It has a sprite editor). But loading correct.

I love how you have your directories structured. Having editor, components, resources, etc where needed in the modules helps quite a bit with readability. Everything with the editor subdirectory relies on Qt in some form.

please check where graph-editor and uikit-editor is located in your case?

For your Linux binaries, RPATH directly adds the local QT’s internal gcc/lib directory to it. That’s the only thing I’m observing. Do you have alternatives for that if that isn’t the solution?

The texturetools plugin is also using Qt inside. (It has a sprite editor). But loading correct.

Are you guiding me to the solution or guessing like me? Texturetools loading correctly using Qt  is dependent on a difference in implementation of similar elements but aligns with where my train of thought was going.
I was asking for an alternative because you know your project better than me but keep on suggesting it’s the RPATH when my own tests are leaning me away from that.

For your Linux binaries, RPATH directly adds the local QT’s internal gcc/lib directory to it. That’s the only thing I’m observing. Do you have alternatives for that if that isn’t the solution?

The texturetools plugin is also using Qt inside. (It has a sprite editor). But loading correct.

Are you guiding me to the solution or guessing like me? Texturetools loading correctly using Qt is dependent on a difference in implementation of similar elements but aligns with where my train of thought was going.
I was asking for an alternative because you know your project better than me but keep on suggesting it’s the RPATH when my own tests are leaning me away from that.

Unfortunately guessing. I don't have BSD to check. But all points to RPATH. What is your assumptions?

For your Linux binaries, RPATH directly adds the local QT’s internal gcc/lib directory to it. That’s the only thing I’m observing. Do you have alternatives for that if that isn’t the solution?

The texturetools plugin is also using Qt inside. (It has a sprite editor). But loading correct.

Are you guiding me to the solution or guessing like me? Texturetools loading correctly using Qt is dependent on a difference in implementation of similar elements but aligns with where my train of thought was going.
I was asking for an alternative because you know your project better than me but keep on suggesting it’s the RPATH when my own tests are leaning me away from that.

Unfortunately guessing. I don't have BSD to check. But all points to RPATH. What is your assumptions?

Not unfortunate at all. We’re on the same page then.
Most of the BSDs follow the standard Linux directory structures outside of where userland (ported tools and libraries that aren’t a part of the base go here) software goes. That’s what “/usr/local” is all about in my CMake updates.
The sdk structure and build directories will be the exact same between platforms.

Another assumption would be Clang vs GCC. Mac’s clang and BSD’s are different under the hood, so what works correctly on a Mac won’t always translate to the BSD version. Gotta love custom flavors of tools. 😅

Could you make an archive of your install dir from cmake?

When I get off work, I can send it. It’ll be too big for Github. A debug build directory ends up at just under 1 GB in size. The sdk debug build I haven’t checked but release sdk is slightly smaller than the Linux one.

I don't think the build dir will be useful for me. could you just zip an install dir?

I misread your original message. The sdk release install is slightly smaller. I might be able to manually add the system Qt libraries to it and them link correctly. I’ll need to test that out. Without them, lddtree will give the wrong info

or you just can list install dir structure reversely for now. I just would like to check everything is in place

I haven’t tried it yet. I’m used to that being linked without any intervention. That’s why when some of the modules couldn’t find specific Qt modules was strange.

I haven’t tried it yet. I’m used to that being linked without any intervention. That’s why when some of the modules couldn’t find specific Qt modules it was strange.

If you just building from source for own it will work. but if you a going to make SDK you need to bundle all dependencies.

Heading home for lunch. Have about 3 hours before going back in so I’ll work with your feedback. If it’s as simple as that, I’ll be relieved. If not, I still have reading to do.

Or you just can list install dir structure recursive for now. I just would like to check everything is in place. so you refer to Qt libs outside install dir right? this would require to change RPATH isn't it?

tree.txt
Reverse tree incoming

I don't think it's an issue, but are you using x86 architecture?

I don't think it's an issue, but are you using x86 architecture?

64-bit. That's on my todo list. The way CMake currently determines architecture is a bit off. For FreeBSD 14.x and older, 32-bit support will remain actively supported upstream (don't know about the other BSDs) but 15+ is going exclusively 64-bit and dropping 32-bit (outside of compatibility requirements for certain software like WINE or certain browsers). It shouldn't cause an issue for my tests but would be a problem if a maintainer from the repos takes on getting it officially in there.

test.zip
Please apply this patch

test.zip Please apply this patch

That is a beautiful patch.

As I said it's in RPATH =)

Is that Qt though?

No. Will check it tomorrow, It's too late for me now =) need to sleep.

I'm looking at it right now, too. Expect a possible PR.

Still possibly related to Qt:
BSD libuikit-editor.so symbols
brad@5k-lame-brain ~/g/t/i/s/2/f/x/lib> nm -gC libuikit-editor.so | grep UiEdit
000000000014b4f0 T UiEdit::changeEvent(QEvent*)
00000000001491a0 T UiEdit::onActivated()
000000000014b490 T UiEdit::resizeEvent(QResizeEvent*)
00000000001489f0 T UiEdit::readSettings()
0000000000148c60 T UiEdit::writeSettings()
0000000000149240 T UiEdit::onObjectCreate(QString)
0000000000149400 T UiEdit::onObjectsChanged(QList<Object*> const&, QString, Variant const&)
0000000000149300 T UiEdit::onObjectsDeleted(QList<Object*>)
00000000001492d0 T UiEdit::onObjectsSelected(QList<Object*>, bool)
000000000014aba0 T UiEdit::saveElementHelper(pugi::xml_node&, Widget*)
000000000014a4c0 T UiEdit::loadAsset(AssetConverterSettings*)
0000000000149220 T UiEdit::onUpdated()
000000000014a6f0 T UiEdit::saveAsset(QString const&)
0000000000147fd0 T UiEdit::UiEdit()
0000000000147fd0 T UiEdit::UiEdit()
0000000000148e10 T UiEdit::~UiEdit()
0000000000148be0 T UiEdit::~UiEdit()
0000000000148be0 T UiEdit::~UiEdit()
000000000014d2e0 W Ui_UiEdit::retranslateUi(QWidget*)
000000000014b8b0 W Ui_UiEdit::setupUi(QWidget*)
0000000000148e60 T UiEdit::isModified() const
000000000014a150 T UiEdit::propertyTag(MetaProperty const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const
0000000000149010 T UiEdit::componentGroups() const
0000000000148ea0 T UiEdit::suffixes() const
                 U vtable for UiEdit
0000000000148e40 T virtual function non-virtual override offset : -16 UiEdit::~UiEdit()
0000000000148df0 T virtual function non-virtual override offset : -16 UiEdit::~UiEdit()
Process 46274, 'nm' from job 1, 'nm -gC libuikit-editor.so | gre…' terminated by signal SIGBUS (Misaligned address error)

Linux libuikit-editor.so symbols
brad@5k-lame-brain ~/D/T/l/r/i/s/2/l/x/lib> nm -gC libuikit-editor.so | grep UiEdit
0000000000081ff0 T UiEdit::changeEvent(QEvent*)
00000000000820a0 T UiEdit::onActivated()
00000000000c3330 T UiEdit::qt_metacall(QMetaObject::Call, int, void**)
00000000000c32d0 T UiEdit::qt_metacast(char const*)
0000000000089400 T UiEdit::resizeEvent(QResizeEvent*)
0000000000086170 T UiEdit::readSettings()
0000000000087d10 T UiEdit::writeSettings()
0000000000081e70 T UiEdit::onObjectCreate(QString)
0000000000084ca0 T UiEdit::onObjectsChanged(QList<Object*> const&, QString, Variant const&)
0000000000081f00 T UiEdit::onObjectsDeleted(QList<Object*>)
0000000000106ce0 D UiEdit::staticMetaObject
0000000000081ee0 T UiEdit::onObjectsSelected(QList<Object*>, bool)
00000000000825e0 T UiEdit::saveElementHelper(pugi::xml_node&, Widget*)
0000000000087fc0 T UiEdit::loadAsset(AssetConverterSettings*)
0000000000081e60 T UiEdit::onUpdated()
0000000000088270 T UiEdit::saveAsset(QString const&)
00000000000863c0 T UiEdit::UiEdit()
00000000000863c0 T UiEdit::UiEdit()
0000000000087f90 T UiEdit::~UiEdit()
0000000000087f30 T UiEdit::~UiEdit()
0000000000087f30 T UiEdit::~UiEdit()
0000000000081e30 T UiEdit::isModified() const
00000000000c30e0 T UiEdit::metaObject() const
0000000000082120 T UiEdit::propertyTag(MetaProperty const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const
00000000000860a0 T UiEdit::componentGroups() const
0000000000085ff0 T UiEdit::suffixes() const
00000000001069f0 V typeinfo for UiEdit
00000000000e4c08 V typeinfo name for UiEdit
0000000000106a08 V vtable for UiEdit
0000000000087fb0 T virtual function non-virtual override offset : -16 UiEdit::~UiEdit()
0000000000087f80 T virtual function non-virtual override offset : -16 UiEdit::~UiEdit()

The BSD one is missing the qt_metacall symbol that the plugin system looks for (if I've been understanding the docs right).
I bet the other bad plugins have the same issue. Will work on adding Qt5 to the RPATH just in case.

I'm looking at it right now, too. Expect a possible PR.

I’ll watch and learn on this. I have a breakage from the C++ 17 code that I’m testing out before I post anything. Right now, it looks pretty simple to fix, just requires a bit of trial and error but I will probably have some questions about code implementation when I get to that point. 😁

Have cc2ef43 solved the boot up issue?

Have cc2ef43 solved the boot up issue?

See here: https://github.com/thunder-engine/thunder/pull/791 .

There's still some bugs that need to be ironed out but I can't believe WorldEditor is running natively. The performance is a bit bad at the moment but that could be related to https://github.com/thunder-engine/thunder/issues/789 . With the debug build, this was the output. Like suggested by the FreeBSD developer above, it looks like at least one of the shaders may need some tweaking.
world-editor-first-flight.txt

I guess this issue can be closed. For shader compilation errors I'm suggesting to create a new one.

I guess this issue can be closed. For shader compilation errors I'm suggesting to create a new one.

Definitely closed and noted.


Dependabot version update no space left on device

Is there an existing issue for this?

[X] I have searched the existing issues

Package ecosystem
Terraform
Package manager version
Terraform
Language version
Latest
Manifest location and content before the Dependabot update
No response
dependabot.yml content
https://github.com/ministryofjustice/analytical-platform/blob/main/.github/dependabot.yml
Updated dependency
No response
What you expected to see, versus what you actually saw
Expect to see pull requests updated for all dependencies
What we are seeing
many references to no space left on device
and
updater | +--------------------------------------------------------------+
updater | |                Dependencies failed to update                 |
updater | +----------------------+---------------------------------------+
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/aws        | dependency_file_not_resolvable        |
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/kubernetes | dependency_file_not_resolvable        |
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/aws        | dependency_file_not_resolvable        |
updater | | hashicorp/kubernetes | private_source_authentication_failure |
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/kubernetes | unknown_error                         |
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/kubernetes | unknown_error                         |
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/helm       | unknown_error                         |
updater | | hashicorp/helm       | unknown_error                         |
updater | | hashicorp/aws        | unknown_error                         |
updater | | hashicorp/kubernetes | unknown_error                         |
updater | | pagerduty/pagerduty  | unknown_error                         |
updater | | hashicorp/aws        | unknown_error                         |
updater | +----------------------+---------------------------------------+

Native package manager behavior
No response
Images of the diff or a link to the PR, issue, or logs
https://gist.github.com/jacobwoffenden/ab818a03c762c4edc216870f2f667ef1
Smallest manifest that reproduces the issue
No response

I didn't realise Dependabot on GitHub Actions had gone GA https://github.blog/2024-05-02-dependabot-on-github-actions-and-self-hosted-runners-is-now-generally-available/, have enabled!


Add description icon to table and column

Signed-off-by: Timothy Lee<EMAIL_ADDRESS>Co-authored-by: Even Wei<EMAIL_ADDRESS>
** PR checklist **

[x] Ensure you have added or ran the appropriate tests for your PR.
[x] DCO signed

What type of PR is this?
Feature
What this PR does / why we need it:
Show description in single report

Conflicted.


3D Tiles - Support batch table binary

For https://github.com/AnalyticalGraphicsInc/3d-tiles/issues/32
This adds support for a batch table binary section to b3dm, i3dm, and pnts.
To do:

[x] Need instancedWithBatchTableBinary tileset for testing. (This causes one test to fail)
[ ] Possibly remove handleTranslucent check for pnts, should be able to support transparent and opaque together.

In the Sandcastle example with the new batched point cloud, picking works well when zoomed out, but when zoomed in, it often flickers between the back and front faces of the sphere as the mouse moves, which is expected.
No action now, but when we do, for example, the edge-preserving blur, we'll need to determine how to "blur" the pick ids, e.g., think more like nearest texture filtering, not linear.

Can you verify that test coverage is solid?
Looks great!

In the Sandcastle example with the new batched point cloud, picking works well when zoomed out, but when zoomed in, it often flickers between the back and front faces of the sphere as the mouse moves, which is expected.

I should be able to make this better by using normals and doing back-face culling on the points.

Ah..... I accidentally committed some large pnts tiles, I'll squash my changes with smaller versions.

Updated.
This also lays the foundation for dynamic styling in the shader. Here is a point cloud that stores a secondaryColor (shades of red) and temperature (3d noise values) as per-point properties in the batch table, loaded in as vertex attributes and styled in the vertex shader.

This also lays the foundation for dynamic styling in the shader...

Very nice.

Do you still plan to do this in this PR?

[ ] Possibly remove handleTranslucent check for pnts, should be able to support transparent and opaque together.

Should the arraybuffer/view memory concerns we discussed offline be addressed in this PR or another?

Do you still plan to do this in this PR?

Possibly remove handleTranslucent check for pnts, should be able to support transparent and opaque together.

I think I may pass on that for later. I'm a bit worried about the performance hit associated with doing all the pass/translucent checks per point. As of now supporting opaque and translucent together is only an issue for batched points, and maybe just setting the render state to translucent may be enough.

Should the arraybuffer/view memory concerns we discussed offline be addressed in this PR or another?

I'll handle it here.

I think I may pass on that for later. I'm a bit worried about the performance hit associated with doing all the pass/translucent checks per point. As of now supporting opaque and translucent together is only an issue for batched points, and maybe just setting the render state to translucent may be enough.

Good plan.

Just those comments.  Really liking the direction here.

The points batched example in Sandcastle comes up blank here: http://cesium-dev.s3-website-us-east-1.amazonaws.com/cesium/batch-table-binary/Apps/Sandcastle/index.html?src=3D Tiles.html&label=Showcases
Does it for you?

Updated.

The points batched example in Sandcastle comes up blank here: http://cesium-dev.s3-website-us-east-1.amazonaws.com/cesium/batch-table-binary/Apps/Sandcastle/index.html?src=3D Tiles.html&label=Showcases

This works now.
The memory issues are also addressed. Model doesn't hold on to the array buffer: https://github.com/AnalyticalGraphicsInc/cesium/blob/master/Source/Scene/Model.js#L3697.

Part of #3241


Quiz stalls

Describe the bug
Doing the quiz, I answered the question but the next one does not unlock
My local app state:

Evolution of Money I question 1 is answered, but it says "Earn 2 sats" on the button
Question 2 is greyed out earn 2 sats - > To unlock, answer ...
When trying to answer question 1 again to move forward, I get an error toast message:
"Error quiz question was already claimed" without unlocking question 2

To Reproduce
IDK how to reproduce
Expected behavior
Quiz should continue
Smartphone (please complete the following information):

Device: iPhone 12
OS: iOS 16.6.1
Blink app <IP_ADDRESS>8

if you try again today, are you able to do the next section?
maybe this something we need to clarify, but only one section can be done per day

I restarted the app and that resolved it. I cannot say if doing it a day later fixed it though. I did restart not the same day so that might confound the bug


Fix a false negative for RSpec/FilePath when spec file has multiple describe.

Fix #1106
RSpec/FilePath does not register offenses when the spec file has multiple describe, such as the following code.
describe MyClass, '#foo' do
end

describe MyClass, '#bar' do
end

But the spec file has a single ‘describe’, registers offenses.
describe MyClass, '#foo' do
^^^^^^^^^^^^^^^^^^^^^^^^ Spec path should end with `my_class*foo*_spec.rb`.
end

I think this is a false negative, so could you please approve this PR?

Before submitting the PR make sure the following are checked:

[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[ ] Updated documentation.
[ ] Added an entry to the CHANGELOG.md if the new code introduces user-observable changes.
[ ] The build (bundle exec rake) passes (be sure to run this locally, since it may produce updated documentation that you will need to commit).

If you have created a new cop:

[ ] Added the new cop to config/default.yml.
[ ] The cop is configured as Enabled: pending in config/default.yml.
[ ] The cop documents examples of good and bad code.
[ ] The tests assert both that bad code is reported and that good code is not reported.
[ ] Set VersionAdded in default/config.yml to the next minor version.

If you have modified an existing cop's configuration options:

[ ] Set VersionChanged in config/default.yml to the next major version.

What would be the suggested file path in this case?

I suggest checking the diff of offences resulted by this change against https://github.com/pirj/real-world-rspec, and specifically rspec-core and rspec-expectations repos, to my best memory they have had several top-level describes in the same file.
Basically, this change would prevent from having multiple top-level example groups. Some prefer to;
RSpec.describe A, '#foo' do
end
RSpec.describe A, '#bar do
end

and others
RSpec.describe A do
  describe '#foo' do
  end
  describe '#bar' do
  end
end

I believe both styles are ok.
Even if we want to provide a toolbox to enforce consistency between those two styles, this is unlikely a job for RSpec/FilePath cop.

Thank you for pointing this out.

I believe both styles are ok.
Even if we want to provide a toolbox to enforce consistency between those two styles, this is unlikely a job for RSpec/FilePath cop.

You are totally right.
I'm sorry, I should have discussed this in the issue beforehand.
So, could I close this Pull Request because we don't think it's necessary?

Thank you for the time you put in it.
I guess the PR can be closed. And a PR with a cop that would suggest combining/splitting top-level groups with method specifiers is welcome!


fix: Add DAG as default in UI filter and reorder

DAG should be on by default

Back-ported to 2.7


Make server kick client

.Is there some way to make the server kick a client? For example, something like
for client in server.get_clients():     if client.toofast:        server.kick_client(client)

You can send a message to the client to shut down process_net_event
Server :
def kick(client,reason):      client.send_message("kick":"reason) # Send a message containing "kick" with content reason
Client :
Your code, imports, definitions etc
kick=False
@Client.event
def kick(reason):
global kick
kick = True
print(reason) # you can add a reason if you want. If not replace the reason variable on the server to any value you want
while kick=False:
Client.process_net_event()
Other stuff


Change output stream format to H264.RemoteStream

Since the new H264 parser does not accept RemoteStream with type: :packetized, a suggestion has been made to change it to H264.RemoteStream with alignment: :nalu. This will necessitate changes in membrane_rtp_plugin.
Discussion of this proposed solution is strongly encouraged below 😄

It doesn't seem right to change it here - the H264.RemoteStream was created to allow passing AVCDecoderConfigurationRecord with SPS and PPS, so:

The new parser seems to be bugged and ignores the :decoder_configuration_record from H264.RemoteStream. That record should be parsed as well to properly handle such format
Should handle %RemoteStream{content_format: H264, type: :packetized} the way it currently handles %H264.RemoteStream{alignment: :nalu}

To sum up - IMO the changes should be done in the parser, not here (cc: @varsill @DominikWolek)

On our internal meeting we decided that this is the way to go so I am approving


Feature "Right-Click in File Tree to Beautify other files" not working

While the plugin works fine when invoked via the command palette or plugins menu, right clicking in the file tree generates the following error (w/ stack trace provided). Apologies if I'm reporting this wrong, long time lurker first time participator.
/var/folders/jb/jnrxxr6n5w9fssk3jr_2qpjc0000gn/T/115630-39496-1uce8b3.cfg:1 Expected a number for output_tab_size, got undefined /var/folders/jb/jnrxxr6n5w9fssk3jr_2qpjc0000gn/T/115630-39496-1uce8b3.cfg:2 Expected a number for input_tab_size, got undefined Parsing: /var/folders/jb/jnrxxr6n5w9fssk3jr_2qpjc0000gn/T/input115630-39496-a0yrfr as language C
/var/folders/jb/jnrxxr6n5w9fssk3jr_2qpjc0000gn/T/115630-39496-1uce8b3.cfg:1 Expected a number for output_tab_size, got undefined
/var/folders/jb/jnrxxr6n5w9fssk3jr_2qpjc0000gn/T/115630-39496-1uce8b3.cfg:2 Expected a number for input_tab_size, got undefined
Parsing: /var/folders/jb/jnrxxr6n5w9fssk3jr_2qpjc0000gn/T/input115630-39496-a0yrfr as language C
Hide Stack Trace
Error: /var/folders/jb/jnrxxr6n5w9fssk3jr_2qpjc0000gn/T/115630-39496-1uce8b3.cfg:1 Expected a number for output_tab_size, got undefined
/var/folders/jb/jnrxxr6n5w9fssk3jr_2qpjc0000gn/T/115630-39496-1uce8b3.cfg:2 Expected a number for input_tab_size, got undefined
Parsing: /var/folders/jb/jnrxxr6n5w9fssk3jr_2qpjc0000gn/T/input115630-39496-a0yrfr as language C
    at /Users/jonathan/.atom/packages/atom-beautify/src/beautifiers/beautifier.coffee:289:25
    at tryCatcher (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/babel-core/node_modules/bluebird/js/main/util.js:26:23)
    at Promise._settlePromiseFromHandler (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/babel-core/node_modules/bluebird/js/main/promise.js:503:31)
    at Promise._settlePromiseAt (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/babel-core/node_modules/bluebird/js/main/promise.js:577:18)
    at Promise._settlePromises (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/babel-core/node_modules/bluebird/js/main/promise.js:693:14)
    at Async._drainQueue (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/babel-core/node_modules/bluebird/js/main/async.js:123:16)
    at Async._drainQueues (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/babel-core/node_modules/bluebird/js/main/async.js:133:10)
    at MutationObserver.Async.drainQueues (/Applications/Atom.app/Contents/Resources/app.asar/node_modules/babel-core/node_modules/bluebird/js/main/async.js:15:14)
    at process._tickCallback (node.js:367:9)

Could you run the Atom command from the Command Palette: Atom Beautify: Help Debug Editor. Paste the contents of that into a Gist with the file extension .md (it is a markdown file), then paste the link to it here for me to see.
So the right click beautify in the file tree does call different code than would the command palette beautify however they should be similar for the most part (beautification).
If you would be interested in doing some debugging yourself, that would be very helpful.
The menu item Beautify File calls the command atom-beautify:beautify-file. See https://github.com/Glavin001/atom-beautify/blob/master/menus/atom-beautify.cson#L8
The command atom-beautify:beautify-file is added here: https://github.com/Glavin001/atom-beautify/blob/master/src/beautify.coffee#L446
beautifyFile function is implemented here: https://github.com/Glavin001/atom-beautify/blob/master/src/beautify.coffee#L233-L240
beautifyFile calls beautifyFilePath which is implemented here: https://github.com/Glavin001/atom-beautify/blob/master/src/beautify.coffee#L187-L231
You are able to change the debugging levels by going into Atom Beautify's package settings, and changing the value of Logger Level option to something like verbose (declared at https://github.com/Glavin001/atom-beautify/blob/master/src/config.coffee#L14-L18 ). This will now show logs with level verbose and up, which is basically everything, in your Atom JavaScript Developer Console. So you can add debugging statements using the logger, such as logger.verbose('well, this happened').
Hope that helps. Thanks!

Here's the debug output: https://gist.github.com/toadhats/1d0b04e003bc9a3ad6f7

also I'm pretty clueless so forgive me if this is really dumb, but aren't all the functions you just mentioned called when beautify is used via palette/menu as well?


Upgrade Pyjion to Python 3.6

This change upgrades Pyjion to Python 3.6.  It supports the new word based opcode, supports the new opcodes that have been added/changed in 3.6, and uses the frame evaluation function to hook into CPython.
Pyjion is also now a C extension module and JITing is enabled when the module is imported.  There are also several functions exposed to check the status of Pyjion, enable/disable JITing, as well as configuring the JIT threshold.

@DinoV,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

I believe this PR will close #198 , #204 , #208 , and #209 . That will leave only #199 as the only issue left under the Python 3.6 milestone.


Validator ctor default

From #8 this is a request for fixing and improving default handling for validator ctor.
"The constructor of the validator requires me to specify against which fileformat (2007/2010/2013) to validate. How can I determine the file format of an existing document? If that's possible, why doesn't the default constructor of the validator do so, instead of defaulting to 2007?"

I don't believe this is possible. Is there anything in the OpenXML schema that gives this information? @tomjebo  @EricWhiteDev


Broken To header when writing "surname, firstname" if the name contains UTF-8 characters

E.g. in German is it common to write a name like "surname, firstname". When writing Jóhn Doe, you address an e-mail or letter to "Doe, Jóhn". Because the expression contains a comma, in a MIME header has to be quoted.
In lettre 0.10.0-rc.3 the To header is encoded as To: "Doe, =?utf-8?b?SsOzaG4i?=<EMAIL_ADDRESS>You will notice that the trailing quotation mark is base64 escaped, which makes some mail providers reject the message, e.g. web.de, gmx.de, and freenet.de. It's well in their right to reject such a message, because the To header is mis-encoded, compare  https://tools.ietf.org/tools/msglint/
Instead of To: "Doe, =?utf-8?b?SsOzaG4i?=<EMAIL_ADDRESS>you should either

encode the whole name, including the quotation marks: To: =?utf-8?b?IkRvZSwgSsOyaG4i?=<EMAIL_ADDRESS>E.g. Gmail uses this option.
or don't encode the quotation mark: To: "Doe, =?utf-8?b?SsOyaG4=?="<EMAIL_ADDRESS>
Code to generate the encoded header:
let m = MessageBuilder::new()
    .to(Mailbox::new(
        Some(r#""Doe, Jóhn""#.parse().unwrap()),
        "john.doe@example.com".parse().unwrap(),
    ))
    .from("me@example.com".parse().unwrap())
    .envelope(
        Envelope::new(
            Some("me@example.com".parse().unwrap()),
            vec!["john.doe@example.com".parse().unwrap()],
        ).unwrap()
    )
    .body(Vec::new())
    .unwrap()
    .formatted();
print!("{}", std::str::from_utf8(&m).unwrap());

Very similar to #626

Instead of To: "Doe, =?utf-8?b?SsOzaG4i?=<EMAIL_ADDRESS>you should either

encode the whole name, including the quotation marks: To: =?utf-8?b?IkRvZSwgSsOyaG4i?=<EMAIL_ADDRESS>E.g. Gmail uses this option.
or don't encode the quotation mark: To: "Doe, =?utf-8?b?SsOyaG4=?="<EMAIL_ADDRESS>

As discussed in https://github.com/lettre/lettre/issues/626#issuecomment-930349634 I'm working on a fix and just to double-check how others solved this issue I sent emails to myself using the Fastmail web interface, which I trust more than Gmail, and here are the findings:

Original
Encoded
Decoded

John
John
John

John Doe
"John Doe"
"John Doe"

Jòhn
=?UTF-8?Q?J=C3=B2hn?=
Jòhn

Jòhn Doe
=?UTF-8?Q?J=C3=B2hn_Doe?=
Jòhn Doe

So it seems like we can avoid putting in the quotation marks if the name-addr part of the email needs to be encoded

#700 fixed the problem. Thank you very much!


Missing GDS????

Was able to final get a bit further.
Realized I had to set up AuroaDB and enter my info there.
Getting further but now it fails trying to excute first call in function:
Am I missing the GDS library?
def cluster_genres_with_gds(neo4j):
    result = neo4j.run("""
    CALL gds.graph.exists($name) YIELD exists WHERE exists CALL gds.graph.drop($name) YIELD graphName
    RETURN graphName + " was dropped." as message
    """, name='genre-has-artist')

Full Output from Start
KTEL-MBP-1312:neo4j-playlist-builder djk-tel$ python3 neo4j_spotify_playlist_builder.py
dropping and creating constraints...
creating tracks...
creating albums...
creating artists..
creating genres..
Linking tracks to albums, genres, and artists...
Clustering genres using the GDS library and creating super-genres...
Traceback (most recent call last):
  File "/Volumes/Panko/zz Programming Transfers/Machine Learning/Neo 4J/neo4j-playlist-builder/neo4j_spotify_playlist_builder.py", line 338, in <module>
    load_graph_using_spotify_api()
  File "/Volumes/Panko/zz Programming Transfers/Machine Learning/Neo 4J/neo4j-playlist-builder/neo4j_spotify_playlist_builder.py", line 63, in load_graph_using_spotify_api
    cluster_genres_with_gds(neo4j)
  File "/Volumes/Panko/zz Programming Transfers/Machine Learning/Neo 4J/neo4j-playlist-builder/neo4j_spotify_playlist_builder.py", line 114, in cluster_genres_with_gds
    result = neo4j.run("""
  File "/Users/djk-tel/Library/Python/3.9/lib/python/site-packages/neo4j/work/simple.py", line 217, in run
    self._autoResult._run(query, parameters, self._config.database, self._config.default_access_mode, self._bookmarks, **kwparameters)
  File "/Users/djk-tel/Library/Python/3.9/lib/python/site-packages/neo4j/work/result.py", line 101, in _run
    self._attach()
  File "/Users/djk-tel/Library/Python/3.9/lib/python/site-packages/neo4j/work/result.py", line 202, in _attach
    self._connection.fetch_message()
  File "/Users/djk-tel/Library/Python/3.9/lib/python/site-packages/neo4j/io/_bolt4x0.py", line 330, in fetch_message
    response.on_failure(summary_metadata or {})
  File "/Users/djk-tel/Library/Python/3.9/lib/python/site-packages/neo4j/io/_bolt4x0.py", line 529, in on_failure
    raise Neo4jError.hydrate(**metadata)
neo4j.exceptions.ClientError: There is no procedure with the name `gds.graph.exists` registered for this database instance. Please ensure you've spelled the procedure name correctly and that the procedure is properly deployed.

hi @technomorph ,
Yes, you're correct!
You'll have to use a Neo4j installation with the GDS library installed to run the tool.
If you download Neo4j Desktop and set up a database there, you'll be able to install the GDS library with one click:
https://neo4j.com/download/
GDS for Neo4j Aura is still in a preview state, and not available for the public (yet).
Hope this works,
Niels

Thank you.   After I downloaded and installed Neo4J Desktop was able to
get it to work.  I made sure I added the GDS library to the Project so it was available to the dB.
Could you please add that to the README file.
Also note to folks that if you set a password for your dB (I used the default neo4j)
That that should be the password that you enter into your "credentials" in the file.
I started playlist with some of the options like
plot_kmeans_clusters = False                                      	# Whether to plot the kmeans clusters used for playlists. min_playlist_size = 10                                              # Cut off for playlists to be grouped as 'misc' playlist_split_limit = 50                                           # min size for playlists to be chopped up in smaller ones.
and then re-ran and started getting errors about constraint already existing.
I'm learning how to clear out the dB and will start another issue with info.
thanks

Hi @technomorph , thanks for the findings!
I've added a note to the README that the GDS plugin needs to be installed as you suggested.
Thanks,
Niels


Enable METplus to only process certain months of a year

Replace italics below with details for this issue.
Describe the New Feature
Enable METplus to only process certain months out of the year, and then move on to the next year.  For example, only process December, January, and February, skip the other months, and continue on to the next December.  I'm attaching the script below, which regrids data for December, January, February and skips the other months.
comb.txt
Acceptance Testing
Any dataset that runs for multiple years.
Time Estimate
Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.
Sub-Issues
None
Relevant Deadlines
Needed for the UIUC blocking use case
Funding Source
UIUC?
Define the Metadata
Assignee

[ ] Select engineer(s) or no engineer required
[ ] Select scientist(s) or no scientist required

Labels

[ ] Select component(s)
[ ] Select priority
[ ] Select requestor(s)

Projects and Milestone

[ ] Review projects and select relevant Repository and Organization ones
[ ] Select milestone

Define Related Issue(s)
Consider the impact to the other METplus components.

[ ] METplus, MET, METdb, METviewer, METexpress, METcalcpy, METplotpy

New Feature Checklist
See the METplus Workflow for details.

[ ] Complete the issue definition above.
[ ] Fork this repository or create a branch of develop.
Branch name:  feature_<Issue Number>_<Description>
[ ] Complete the development and test your changes.
[ ] Add/update unit tests.
[ ] Add/update documentation.
[ ] Push local changes to GitHub.
[ ] Submit a pull request to merge into develop.
Pull request: feature <Issue Number> <Description>
[ ] Iterate until the reviewer(s) accept and merge your changes.
[ ] Delete your fork or branch.
[ ] Close this issue.

Work done in feature branch: feature_512_skip_months

Merged into develop with this PR: https://github.com/NCAR/METplus/pull/519


SA1013 in collision with SA1010 (closing braces vs opening square brackets)

In the following statement:
new[] { 1, 2, 3 } [new Random().Next(2)]
One cannot reconcile the two rules - the closing braces after number 3 requires a space, opening bracket requires no space.

Duplicate of #1657


Encounter error when using the code in tutorial 3: render with phantomjs

i try to run the demo code in tutorial 3, but encounter error every time. i already installed phantomjs in my ubuntu 14.04 and run pyspider with mongodb.
i run pyspider using command: pyspider  --config .../config.json
is it about the running mode? How to solve this problem?
The error:
[E 150312 17:20:47 base_handler:221] 501 Server Error
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pyspider/libs/response.py", line 175, in raise_for_status
raise http_error
HTTPError: 501 Server Error

You need have phantomjs installed.

i already installed phantomjs ;） 。。。

The logs after pyspider started.

I restart pyspider, and now another error came:
[E 150312 18:23:05 base_handler:221] SSL handshake failed
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pyspider/libs/response.py", line 175, in raise_for_status
raise http_error
HTTPError: SSL handshake failed
besides, where can i find the log file?

It is printed on screen by default.
HTTPError: SSL handshake failed: upgrade pyspider to latest version.

i use command "pip install pyspider --upgrade" to upgrade pyspider, and an exception occurred:
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_universe/pycurl
Storing debug log for failure in ..

Should i just ignore this exception?

The log file content:
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_universe/pycurl
Exception information:
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1229, in prepare_files
req_to_install.run_egg_info()
File "/usr/lib/python2.7/dist-packages/pip/req.py", line 325, in run_egg_info
command_desc='python setup.py egg_info')
File "/usr/lib/python2.7/dist-packages/pip/util.py", line 697, in call_subprocess
% (command_desc, proc.returncode, cwd))
InstallationError: Command python setup.py egg_info failed with error code 1 in

failed with error code 1 in the logs ends here?
You haven't upgrade successfully, why do you ignore it?


proxy.py: set no_reply for KBOOT_BOOT

without this macOS complains about the USB device that goes missing
once m1n1 hands control to Linux
Signed-off-by: Sven Peter<EMAIL_ADDRESS>

This doesn't look right; m1n1 should be sending the reply. Do we have a race where m1n1 does not flush the buffer properly? If so we should fix that (probably worth doing iodev_flush(uartproxy_iodev); after iodev_write(...) at the end of the proxy loop, which shouldn't hurt performance since obviously proxy stuff is synchronous anyway so we might as well wait for the data to go out). If that doesn't fix it then we have a race and the macOS driver does not return buffered data after a device disconnect, so, uh, add a 100ms udelay before USB shutdown in main.c? :/

turns out that this actually had two root causes:

the flush was indeed missing
the usb phy was re-initialized which confused mac os because the usb device reappeared

I guess we're still relying on a race here (Linux will eventually reinit USB), but I guess that's more than enough time for the reply to make it out to python, and in the meantime since m1n1 leaves the USB hardware in limbo macOS won't detect a disconnect. Good enough for now. Might want to explicitly nuke USB (with a bit of a delay) in the future though :)


response.validate! call fails

Hi,
I am trying to use ruby-saml with a custom OmniAuth Strategy. I am using version 0.7.2
In my heroku logs I see
NoMethodError (undefined method `text' for nil:NilClass):

Below is what my OmniAuth callback phase looks like. The failure seems to be around the response.validate! call. Any thoughts on why this is happening? Problem in my code or in ruby-saml? I know I can receive the SAML response if I comment out that
response.validate!

call. But I am trying to understand the validate method. And specifically what steps I need to take to ensure the SAML assertion my application is receiving is properly signed and trustworthy.
      def callback_phase
        unless request.params['SAMLResponse']
          raise OmniAuth::Strategies::Company::ValidationError.new("SAML response missing")
        end

        response = Onelogin::Saml::Response.new(request.params['SAMLResponse'])
        response.settings = Onelogin::Saml::Settings.new(options)
        @name_id = response.name_id
        @attributes = response.attributes

        if @name_id.nil? || @name_id.empty?
          raise OmniAuth::Strategies::Company::ValidationError.new("SAML response missing 'name_id'")
        end
        response.validate!
        super
      rescue OmniAuth::Strategies::Company::ValidationError
        fail!(:invalid_ticket, $!)
      rescue Onelogin::Saml::ValidationError
        fail!(:invalid_ticket, $!)
      end

      uid { @name_id }
      info do
        {
            :company_id  => @attributes[:"CORP ID"],
            :name  => @attributes[:"Given Name"],
            :email  => @attributes[:"Email Address"],
            :account_type  => @attributes[:"Account Type"],
        }
      end

Try upgrading to 0.7.3 or higher if you are running into a similar issue.

@alperkokmen Typo? 0.7.2 is the most recent version available.

@tboyko nope, 0.7.3 was a release. It's fairly out-of-date at this point though. I would recommend using the latest version if you can.


Selective scope

Currently Picus assumes to be the only one adding agents to Woodpecker. This should be changed such that existing agents should be taken into account. So Picus shall support adding an agent when existing agents cannot handle the load. Also Picus shall be able to be configured to listen only to specifc platforms and/or labels and handle agents only for that scope.

I have never written anything in rust so i probably would not be big help in the actual implementation, but how about a design proposal:
Define a kv pair as selector:
Assume i have configured PICUS_AGENT_WOODPECKER_FILTER_LABELS: type=picus so any new agent that will be added gets this label.
In the pipelines:
In the pipeline that should bind to this specific agent one can configure
labels:
    type: picus

the woodpecker endpoint api/queue/info returns these labels:
{
  "pending": [
    {
      "id": "146",
      "data": "REDACTED",
      "labels": {
        "repo": "REDACTED",
        "type": "picus"
      },
      "dependencies": null,
      "run_on": null,
      "dep_status": {},
      "agent_id": 0
    }
  ],
  "waiting_on_deps": null,
  "running": null,
  "stats": {
    "worker_count": 40,
    "pending_count": 1,
    "waiting_on_deps_count": 0,
    "running_count": 0,
    "completed_count": 0
  },
  "paused": false
}

now picus could check for a condition where it has created workers that match this kv combination and create a new agent if there is none or reuse an existing instance if it is still available.
in hetzner and in aws the kv pair could even be added to the instance so the data could directly be matched against the dedicated API. with that picus does not even have to hold any additional state.
wdyt @windsource ?

@Lerentis thanks for the analysis. That sounds like a good plan. I am still planning to implement this feature but I am not sure when I will have the time. So if anyone is willing to create a PR that would help.

Storing the filter labels on instances seems to be not that easy. Hetzner only allows certain characters (, is not allowed) for labels and the length is limited to 63 characters.

Similar limitations apply to k8s as well. I was proposing this as both hetzner and aws are supporting this and kops is using labels similar (as in as a selector). Imo it would be okay to simply fail if a user is configuring something unsupported by the cloud provider

I think there is no need to store the filter labels on the instance when the labels are passed to picus. I am currently preparing a PR.
To understand filter labels I have performed the following tests:
Scenario 1
No WOODPECKER_FILTER_LABELS applied to an agent.
Pipeline:
labels:
  type: picus

Result: Agent will not pick this pipeline.
Scenario 2
WOODPECKER_FILTER_LABELS=type=picus.
Pipeline:
labels:
  type: picus

Result: Agent picks that pipeline.
Scenario 3
WOODPECKER_FILTER_LABELS=type=picus.
Pipeline:
labels:
  type: picus
  foo: bar

Result: Agent will not pick that pipeline.
Scenario 4
WOODPECKER_FILTER_LABELS=type=picus.
Pipeline has no labels set.
Result: Agent will pick that pipeline.
Scenario 5
WOODPECKER_FILTER_LABELS=type=picus and agent has platform=linux/amd64
Pipeline for arm64:
labels:
  type: picus
  platform: linux/arm64

Result: Agent will not pick that pipeline (at least with version next-5b3bba726d)

The release v0.6.0 contains the new feature. Note that you need two more environment variables for Picus:

PICUS_AGENT_WOODPECKER_FILTER_LABELS
PICUS_WOODPECKER_AGENT_ID

See README.md for details.


Translation to Brazilian Portuguese

Translated all files in the resources/lang/en/ directory to Brazilian Portuguese (pt_BR)

Please squash your commits. Thanks for the help!

Alright


media queries not working

I added postcss.config.js - but still no effect, any more configuration steps are required?

yeah, once you have a config you can pass it plugins, this one will recognize the media queries and transform them 👍🏻  you're almost there!

yeah, once you have a config you can pass it plugins, this one will recognize the media queries and transform them 👍🏻 you're almost there!

Well, It comes by default on postcss-preset-env and even so, it is not working for me!

i'm back from vacation, did you find a solution to this? it can be tricky as the plugin needs to have the media queries at build time for many configurations, as they show here.
too bad we dont just have this in CSS yet!


Wrong forecast results

Hello,
Prophet return some strange values
3.1.0 :200 > series
 =>
{#<Date: 2022-01-03 ((2459583j,0s,0n),+0s,2299161j)>=>1.639,
 #<Date: 2022-01-05 ((2459585j,0s,0n),+0s,2299161j)>=>1.649,
 #<Date: 2022-01-06 ((2459586j,0s,0n),+0s,2299161j)>=>1.659,
 #<Date: 2022-01-07 ((2459587j,0s,0n),+0s,2299161j)>=>1.669,
 #<Date: 2022-01-08 ((2459588j,0s,0n),+0s,2299161j)>=>1.659,
 #<Date: 2022-01-10 ((2459590j,0s,0n),+0s,2299161j)>=>1.669,
 #<Date: 2022-01-11 ((2459591j,0s,0n),+0s,2299161j)>=>1.689,
 #<Date: 2022-01-12 ((2459592j,0s,0n),+0s,2299161j)>=>1.679,
 #<Date: 2022-01-13 ((2459593j,0s,0n),+0s,2299161j)>=>1.689,
 #<Date: 2022-01-14 ((2459594j,0s,0n),+0s,2299161j)>=>1.699,
 #<Date: 2022-01-15 ((2459595j,0s,0n),+0s,2299161j)>=>1.699,
 #<Date: 2022-01-18 ((2459598j,0s,0n),+0s,2299161j)>=>1.709,
 #<Date: 2022-01-20 ((2459600j,0s,0n),+0s,2299161j)>=>1.719,
 #<Date: 2022-01-21 ((2459601j,0s,0n),+0s,2299161j)>=>1.729,
 #<Date: 2022-01-22 ((2459602j,0s,0n),+0s,2299161j)>=>1.719,
 #<Date: 2022-01-25 ((2459605j,0s,0n),+0s,2299161j)>=>1.729,
 #<Date: 2022-01-27 ((2459607j,0s,0n),+0s,2299161j)>=>1.739,
 #<Date: 2022-01-29 ((2459609j,0s,0n),+0s,2299161j)>=>1.729}
3.1.0 :201 > Prophet.forecast(series)
 =>
{#<Date: 2022-01-30 ((2459610j,0s,0n),+0s,2299161j)>=>5.547226954196092,
 #<Date: 2022-01-31 ((2459611j,0s,0n),+0s,2299161j)>=>1.7157371604726062,
 #<Date: 2022-02-01 ((2459612j,0s,0n),+0s,2299161j)>=>1.7390083969256263,
 #<Date: 2022-02-02 ((2459613j,0s,0n),+0s,2299161j)>=>1.7340095632954138,
 #<Date: 2022-02-03 ((2459614j,0s,0n),+0s,2299161j)>=>1.7490078239414186,
 #<Date: 2022-02-04 ((2459615j,0s,0n),+0s,2299161j)>=>1.749007500166091,
 #<Date: 2022-02-05 ((2459616j,0s,0n),+0s,2299161j)>=>1.7390047264672899,
 #<Date: 2022-02-06 ((2459617j,0s,0n),+0s,2299161j)>=>5.557226905370591,
 #<Date: 2022-02-07 ((2459618j,0s,0n),+0s,2299161j)>=>1.725737111645919,
 #<Date: 2022-02-08 ((2459619j,0s,0n),+0s,2299161j)>=>1.7490083480986351}

Clearly the first one and the value for the 2022-02-06 are wrong. I suspect a arm64 bug but I don't have an amd64 for test the code.

Hey @blackrez, I'm seeing similar results on x86-64, so don't think it's related to ARM.
It looks like the problem is series doesn't include any Sundays, but it's trying to predict them. If you need predictions for Sundays, make sure to include them in the input. Otherwise, you can filter them from the output.

It looks like the Python library has similar behavior.
import pandas as pd
from prophet import Prophet

df = pd.DataFrame({
  'ds': ["2022-01-03", "2022-01-05", "2022-01-06", "2022-01-07", "2022-01-08", "2022-01-10", "2022-01-11", "2022-01-12", "2022-01-13", "2022-01-14", "2022-01-15", "2022-01-18", "2022-01-20", "2022-01-21", "2022-01-22", "2022-01-25", "2022-01-27", "2022-01-29"],
  'y': [1.639, 1.649, 1.659, 1.669, 1.659, 1.669, 1.689, 1.679, 1.689, 1.699, 1.699, 1.709, 1.719, 1.729, 1.719, 1.729, 1.739, 1.729]
})

m = Prophet()
m.fit(df)

future = m.make_future_dataframe(periods=10, include_history=False)
future.tail()

forecast = m.predict(future)
print(forecast[['ds', 'yhat']])

Output
          ds      yhat
0 2022-01-30 -3.960552
1 2022-01-31  1.720297
2 2022-02-01  1.739000
3 2022-02-02  1.731007
4 2022-02-03  1.749000
5 2022-02-04  1.749000
6 2022-02-05  1.739000
7 2022-02-06 -3.950552
8 2022-02-07  1.730297
9 2022-02-08  1.749000

Thanks for your response and your help, my dataset have a lots of issue and it have a lots of missing days.

Another option is to disable weekly seasonality with the advanced API:
require "prophet"

df = Rover::DataFrame.new({
  "ds" => ["2022-01-03", "2022-01-05", "2022-01-06", "2022-01-07", "2022-01-08", "2022-01-10", "2022-01-11", "2022-01-12", "2022-01-13", "2022-01-14", "2022-01-15", "2022-01-18", "2022-01-20", "2022-01-21", "2022-01-22", "2022-01-25", "2022-01-27", "2022-01-29"],
  "y" => [1.639, 1.649, 1.659, 1.669, 1.659, 1.669, 1.689, 1.679, 1.689, 1.699, 1.699, 1.709, 1.719, 1.729, 1.719, 1.729, 1.739, 1.729]
})

m = Prophet.new(weekly_seasonality: false)
m.fit(df)

future = m.make_future_dataframe(periods: 10, include_history: false)
forecast = m.predict(future)
p forecast[["ds", "yhat"]]

Output
                     ds                yhat
2022-01-30 00:00:00 UTC  1.7360273138060855
2022-01-31 00:00:00 UTC  1.7374329821020085
2022-02-01 00:00:00 UTC  1.7388386503979312
2022-02-02 00:00:00 UTC   1.740244318693854
2022-02-03 00:00:00 UTC  1.7416499869897768
2022-02-04 00:00:00 UTC  1.7430556552856997
2022-02-05 00:00:00 UTC  1.7444613235816226
2022-02-06 00:00:00 UTC  1.7458669918775456
2022-02-07 00:00:00 UTC   1.747272660173468
2022-02-08 00:00:00 UTC  1.7486783284693912

Yeah, it could be the best solution for my very inconsistant dataset. Many thanks for your help.


Ability to connect and read from S3

Want the ability to connect to an S3 bucket and list items within it as if a folder on the local drive.
Should be able to read files from it.
Needs to support AWS Profile and Access Key/Secret Key authentication.
This is a pre-cursor to reading from enso://
This should be the start of Standard.AWS library. Eventually, Redshift should move into here but this is not part of this ticket.
This will eventually be followed by Standard.Azure etc.

@radeusgd - can you separate the Redshift and AWS credentials from Database to their own project please (and leave as a branch).


Bring your own ABI

Sometimes WhatsABI can't get the full ABI, or missing some type annotations. Would be nice to be able to bring your own ABI annotation.
Would be hard to encode this in the link (maybe just the called function hint?).

Make sure we can handle Safe's storage API: https://twitter.com/spengrah/status/1712573701902807314


Make the file inclusion behavior consistent between interactive and batch and get rid of special cases

This point already appeared in #364, #373, FStarLang/fstar-mode.el#13, and https://github.com/FStarLang/fstar-mode.el/issues/20#issuecomment-141713135; let's separate this part on which it seems most of us agree: there should be no special cases in which files are included through the back door (e.g. only from command line and only if one file is passed as argument). This confuses users big time.

+1

I believe this to be fixed since everything is using --auto-deps now.
Closing, feel free to reopen if necessary

I just downloaded the latest fstar.exe for Mac OS X and the fstar-mode for Emacs and am getting the errors reported here: https://github.com/FStarLang/fstar-mode.el/issues/20
for this input file:
module Factorial

(* What are some other types you can give to factorial?
   Try writing them and see if F* agrees with you. *)

val factorial: int -> int   // FStar.All.ML not found
val factorial: int -> ML int // Identifier not found: [ML]
val factorial: x:int{x>=0} -> Tot int
val factorial: x:int{x<0 || x>=0} -> ML int
let rec factorial n = if n = 0 then 1 else n * factorial (n - 1)

This should be fixed on the master branch. You're probably using an old release (they are all old) and

if the release you use is old you might be missing out on new features and bug fixes. Please do not report bugs in old releases until making sure they still exist in the master branch (see Building F* from sources section below).

Source for the quote: https://github.com/FStarLang/FStar/blob/master/INSTALL.md#binary-releases

I downloaded the binaries from the website tonight-- I suppose it hosts binaries that are over 3 months old now. 😀
I will try building from source. Thanks!


Picklist menu items hide menu border when hovered

Is there an existing issue for this?

[X] I have searched the existing issues.

Is this issue related to iced?

[X] My hardware is compatible and my graphics drivers are up-to-date.

What happened?
The title basically says it all; when hovering on an item from the picklist menu, the menu's border gets covered:

)

What is the expected behavior?
The menu border remains visible.
Version
master
Operative System
macOS
Do you have any log output?
No response

I think I've found the code lines responsible of this behaviour: https://github.com/iced-rs/iced/blob/c6218cbc5532551456c0491eb5aa66dfc5f36879/native/src/overlay/menu.rs#L478
And it seems made on purpose.
Any reasons for that?

We are just not accounting for the border width of the background of the menu when rendering the background of the selected option.
I can try to come up with a fix.

Awesome, thanks!

#1831 should fix it.


Sonarr/Radarr Import using script

Sonarr and Radarr now support custom scripts for importing (i.e.. pre-import scripts) https://github.com/Sonarr/Sonarr/pull/5538
All the environmental variables seem the same as OnDownload, and the script should just return exit code 3 after transcoding and they will handle the import.

Shouldn't need any script adjustment then, there was never any specific discrimination between the various calls just the presence of the appropriate environment variables. Are you seeing issues?

https://github.com/Sonarr/Sonarr/blob/60f18249b05daa20523542beef54bc126d963d1e/src/NzbDrone.Core/MediaFiles/ScriptImportDecider.cs#L57

Yeah reading over this it will need its own script I think, a preSonarr.py if you will though I can probably reuse a good portion of postSonarr

https://github.com/Sonarr/Sonarr/pull/5956
Looks like until these features are implemented this new import script probably won't be useful for SMA purposes but I'll keep an eye out for it

Note: I can recommend that you do make some changes to your scripts and remux and transcode from the downloads folder directly to the library folder, as it saves an additional unnecessary copy and thus speeds things up a fair bit(especially for pure remuxing). I can also recommend always requesting a rename, as you don't know whether people might be using say the audio or video codec in their naming scheme. Good to see some potential usages by people other than me though.

Yeah I was mostly referring to these two features

Custom import scripts that generate additional extra files to be imported.
Custom import scripts that remux the original media file to a different container, such that its extension must change

Both of things can and often do happen for SMA processed media, changing containers and ripping or downloading additional subtitles are common so as I understand it that would needed to be supported before adopting the feature
Currently doing things in the download folder is handled by post downloader scripts executed by the downloader before notifying Sonarr/Radarr since they haven't supported this historically, but once this matures it would be nice to offer a script for it

https://github.com/Sonarr/Sonarr/pull/5956 is now merged


Displaying text in pygame window

Hello there,
I'm trying to display text in the pygame window so i used draw_string() function. It works but the text is displayed in the CarlaUE4 not pygame.
Any idea how to display in pygame?
I'm using Carla 0.9.8
OS: Windows

Please, read the documentation of Pygame.


Add :ObsidianTags command

Closes #249.

After not being able to find a neovim tag solution I was happy with, I started writing my own buggy solution last weekend.
I updated the plugin this morning, saw "ObsidianTags", and it felt like Chrsitmas.

Thanks @BrendanRomanDev! :)


trestle remove and testing, with coverage at 100%

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.
[ ] All commits are signed.

Adds trestle remove functionality.
Includes 100% coverage for testing it.

This PR closes #66.

Not quite 100% after the unhandled exceptions were resolved in remove.py.
With the fixes, and added tests, I’m up 95% coverage now, and fixed something that was previously there. The last 5% will probably require a bit more than the prior patterns because simulate() actually catches the problem before execute() does, for a bad remove plan. I think I’ll have to figure out how to feed a bad plan to it.

This is now done with 100% coverage.


canno update from 0.19

Error after click on UPDATE
can't install ghcr.io/donkie/spoolman:0.2: 500 Server Error for http+docker://localhost/v1.45/images/create?tag=0.2&fromImage=ghcr.io%2Fdonkie%2Fspoolman&platform=linux%2Famd64: Internal Server Error ("manifest unknown")

Fixed yesterday - you may have to update add-on sources to see it (should say 0.20.0 is available):


Prompt Too Long

Describe the bug
No matter what I write in prompt, it has started giving me Prompt too long error.
Link to the Bolt URL that caused the error
https://bolt.new/~/sb1-z254bx
Steps to reproduce
When I write instructions whether few words or many, it give me this error every now and then. It started today.
https://github.com/user-attachments/assets/aa7fd0bb-73ce-4134-970e-9bd94fbba31a
Expected behavior
It should give instructions.
Screen Recording / Screenshot
https://github.com/user-attachments/assets/aa7fd0bb-73ce-4134-970e-9bd94fbba31a
Platform
Browser name = Chrome
Full version = <IP_ADDRESS>
Major version = 130
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36
performance.memory = {
  "totalJSHeapSize": 63157931,
  "usedJSHeapSize": 61099191,
  "jsHeapSizeLimit":<PHONE_NUMBER>
}
Username = alirazazain
Chat ID = e69374047358

Additional context
No response

Hi @alirazazain,
We are aware of this issue. We have current workarounds you can try and will be tracking future developments here: #1322


⚠ Young Discoverers has degraded performance

In bda23bb, Young Discoverers (https://youngdiscoverers.org.au) experienced degraded performance:

HTTP code: 200
Response time: 6414 ms

Resolved: Young Discoverers performance has improved in e1b251b.

Degraded, as the cache was purged and is presently rebuilding.


[$250] Android - Track Expenses-The "New workspace" button is displayed behind the keyboard

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 9.0.72-0
Reproducible in staging?: Yes
Reproducible in production?: Yes
If this was caught on HybridApp, is this reproducible on New Expensify Standalone?: Yes
If this was caught during regression testing, add the test name, ID and link from TestRail: https://expensify.testrail.io/index.php?/cases/view/2990404
Email or phone of affected tester (no customers): N/A
Issue reported by: Applause Internal Team
Action Performed:
Setup:
User logged in with an expensifail account.
The user has at least one workspace and needs to delete it just before performing the steps.
Steps:

Submit a track expense in the self DM
On the actionable whisper options of a track expense, click on Share it with your accountant
In the Share page, enter a few characters to trigger the search
Delete all characters
Hide the keyboard

Expected Result:
The "New workspace" button should be displayed above the keyboard all of the time.
Actual Result:
The New Workspace button appears behind and is sometimes cut off by the keyboard.
Workaround:
Unknown
Platforms:

[x] Android: Standalone
[x] Android: HybridApp
[ ] Android: mWeb Chrome
[ ] iOS: Standalone
[ ] iOS: HybridApp
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
https://github.com/user-attachments/assets/a72a45f8-2f86-4b10-be14-2a83aa94de62
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~021866934464307418827
Upwork Job ID:<PHONE_NUMBER>307418827
Last Price Increase: 2024-12-11

Issue OwnerCurrent Issue Owner: @Pujan92

Proposal
Please re-state the problem that we are trying to solve in this issue.
The New Workspace button appears behind and is sometimes cut off by the keyboard.
What is the root cause of that problem?
The empty section has minHeight65 style.
https://github.com/Expensify/App/blob/05b4e8a36647f54bcc8e9a882ef450a2644b0ae1/src/components/EmptySelectionListContent.tsx#L42-L46.
When the keyboard is shown, because the empty section cannot scroll, the new workspace button is down behind the keyboard
https://github.com/Expensify/App/blob/05b4e8a36647f54bcc8e9a882ef450a2644b0ae1/src/components/SelectionList/BaseSelectionList.tsx#L784-L786
What changes do you think we should make in order to solve the problem?
Add a ScrollView to renderListEmptyContent
<ScrollView>
    {renderListEmptyContent()}
</ScrollView>

https://github.com/Expensify/App/blob/05b4e8a36647f54bcc8e9a882ef450a2644b0ae1/src/components/SelectionList/BaseSelectionList.tsx#L784-L786
Optional: In this case, we can also add the footerContent and confirm button to this scroll view
What specific scenarios should we cover in automated tests to prevent reintroducing this issue in the future?

What alternative solutions did you explore? (Optional)
We need to adjust the EmptySelectionListContent to reduce the height of this component. For example, we can introduce props to customize the icon size, text size, and wrap view style here so as not to apply the minHeight65 style when we want. Here is an example of when we reduce the icon size. We need to confirm more information with the design team.

https://github.com/Expensify/App/blob/05b4e8a36647f54bcc8e9a882ef450a2644b0ae1/src/components/EmptySelectionListContent.tsx#L42-L46.

The minHeight65 root cause in EmptySelectionListContent and adding ScrollView for the emptycontent as mentioned in @mkzie2's proposal looks good to me, I suggest to add ScrollView in EmptySelectionListContent component as a generic solution.

@Pujan92 The PR is ready.

Issue not reproducible during KI retests. (First week)


Add support of Swagger docs

Summary
The goal of this PR is to add the support of Swagger docs.

Seems to be a good work, thanks.


Update to compatible node + global @angular/cli version

Bug Report
Expected behaviour
compose_setup and dockstore-ui2 should be using compatible node and @angular/cli versions which at the time of writing is 10.13.0 and 7.0.4 respectively
revert https://github.com/dockstore/dockstore-ui2/commit/be3c0478800dafbc0fb7a074d329338cc4bf240a and the travis wait in the compose_setup
Modify to not use a global @angular/cli version at all
Actual behaviour
It is using node ~8 and 6.0.2 respectively
┆Issue is synchronized with this Jira Bug
┆Issue Type: Bug
┆Fix Versions: Dockstore 1.7
┆Sprint: Seabright Sprint 8 Houseboat
┆Issue Number: DOCK-612

Will soon be made obsolete with cloudfront deployment


Externally applied filter override

I have used filterdatagrid in MainWindow where i have applied filter which i want as a default filter to be applied with other selected filters.
but due to line written below, it is being overridden on Itemsource change
// set Filter, contribution : STEFAN HEIMEL
if (CollectionViewSource.CanFilter) CollectionViewSource.Filter = Filter;

Hi,
You didn't give me enough information to understand your problem

Basically, I am using CollectionViewSource for FIlterDataGrid. This collections contains all records including Active and deleted records.
Now I want to show only active records and i do not want to allow user to filter records based on status.
My application support multi view interface and new records in collectionViewSource will keep adding/updating.
to achieve this, I need an option where i can prepare a custom filter(predicate) which I can bind with Filterdatagrid. so that, this custom filter should work along with filters applied through control (criteria).
I tried to bind the filter but it is replaced due to line written below
// set Filter, contribution : STEFAN HEIMEL
if (CollectionViewSource.CanFilter) CollectionViewSource.Filter = Filter;

What you are asking me is not possible in the sense that "CollectionViewSource.Filter = Filter;" is the heart of datagrid filtering, without it the filters activated by the user on the columns no longer work.
The solution for you would be to use a filter on the source collection to hide the records that you do not want to appear in the datagrid, a bit like the demo application which allows you to filter the collection before being provided to the datagrid. “itemsSource” property of “FilterDatagrid”.

HI,
Actually it is possible. I have applied the changes in FilterDataGrid.cs and external filters are working for me along with criteria. I can raise PR If you want to include that feature.

PR #151 raised.