vagrant up failed. what's up, please give some help. thanks.

macos 10.10
$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
There are errors in the configuration of this machine. Please fix
the following errors and try again:

vm:
* The '' provisioner could not be found.

Vagrant:
* Unknown configuration section 'berkshelf'.
* Unknown configuration section 'omnibus'.

@xiaods Thanks for trying the project!
I've faced this problem before.  At that time, I versionup-ed vagrant to the latest and  I cleanup-ed several caches, those are ~/.vagrant, ~/.berkshelf and ./vagrant.  And then I re-installed vagrant plugins.  This worked for me at that time.  I hope this will help you.

@xiaods Or, you missed some plugins required.  Please check all required plugins installed.

@everpeace yes, it works like a charm. thanks a lot.


Uncaught TypeError: Cannot read property 'offsetWidth' of undefined

[Enter steps to reproduce below:]

...
...

Atom Version: 1.4.0
System: "Arch Linux"
Thrown From: activate-power-mode package, v0.4.1
Stack Trace
Uncaught TypeError: Cannot read property 'offsetWidth' of undefined
At /home/anaibol/.atom/packages/activate-power-mode/lib/activate-power-mode.coffee:126

TypeError: Cannot read property 'offsetWidth' of undefined
    at Object.module.exports.ActivatePowerMode.drawParticles (/home/anaibol/.atom/packages/activate-power-mode/lib/activate-power-mode.coffee:126:35)

Commands
     -1:25.7.0 core:select-down (atom-text-editor.editor.is-focused)
     -1:25.5.0 snippets:previous-tab-stop (atom-text-editor.editor.is-focused)
     -1:25.5.0 editor:outdent-selected-rows (atom-text-editor.editor.is-focused)
     -1:24.9.0 snippets:next-tab-stop (atom-text-editor.editor.is-focused)
     -1:24.9.0 snippets:expand (atom-text-editor.editor.is-focused)
     -1:24.9.0 editor:indent (atom-text-editor.editor.is-focused)
     -1:24.7.0 snippets:next-tab-stop (atom-text-editor.editor.is-focused)
     -1:24.7.0 snippets:expand (atom-text-editor.editor.is-focused)
     -1:24.7.0 editor:indent (atom-text-editor.editor.is-focused)
     -1:23.9.0 core:move-down (atom-text-editor.editor.is-focused)
  2x -1:23.2.0 core:save (atom-text-editor.editor.is-focused)
     -1:22.4.0 command-palette:toggle (atom-text-editor.editor.is-focused)
     -1:21.3.0 editor:consolidate-selections (atom-text-editor.editor.mini.is-focused)
     -1:21.3.0 core:cancel (atom-text-editor.editor.mini.is-focused)
     -1:20.8.0 settings-view:open (atom-text-editor.editor.is-focused)
     -0:51.3.0 activate-power-mode:toggle (button.btn.icon.enablement.icon-playback-pause)

Config
{
  "core": {
    "autoHideMenuBar": true,
    "disabledPackages": [
      "color-picker",
      "auto-indent",
      "fancy-bracket-matcher",
      "git-plus",
      "git-control",
      "import",
      "json-path-finder",
      "livestyle-atom",
      "merge-conflicts",
      "node-debugger",
      "npm-install",
      "nuclide-gadgets",
      "nuclide-move-pane",
      "nuclide-file-tree",
      "nuclide",
      "nuclide-open-filenames-provider",
      "nuclide-quick-open",
      "nuclide-recent-files-provider",
      "nuclide-splits",
      "nuclide-tabs",
      "nuclide-toolbar",
      "pretty-json",
      "tab-control"
    ]
  },
  "activate-power-mode": {
    "particles": {
      "size": {},
      "spawnCount": {},
      "totalCount": {}
    },
    "screenShake": {}
  }
}

Installed Packages
# User
activate-power-mode, v0.4.1
atom-handlebars, v1.3.0
atom-jade, v0.3.0
atom-ternjs, v0.12.2
auto-detect-indentation, v0.4.2
autocomplete-modules, v1.3.1
autocomplete-paths, v1.0.2
hidpi, v0.11.0
json-schema, v0.1.15
minimap, v4.19.0
pigments, v0.19.6
react, v0.14.0

# Dev
No dev packages

[Enter steps to reproduce below:]

...
...

Atom Version: 1.6.1
System: Microsoft Windows 8.1 Enterprise
Thrown From: activate-power-mode package, v0.4.1
Stack Trace
Uncaught TypeError: Cannot read property 'offsetWidth' of undefined
At /C:/Users/Madi/.atom/packages/activate-power-mode/lib/activate-power-mode.coffee:158

TypeError: Cannot read property 'offsetWidth' of undefined
  at Object.module.exports.ActivatePowerMode.drawParticles (C:\Users\Madi\.atom\packages\activate-power-mode\lib\activate-power-mode.coffee:126:35)

Commands
     -0:23.3.0 activate-power-mode:toggle (atom-text-editor.editor.mini.is-focused)

Config
{
  "core": {},
  "activate-power-mode": {
    "particles": {
      "size": {},
      "spawnCount": {},
      "totalCount": {}
    },
    "screenShake": {}
  }
}

Installed Packages
# User
activate-power-mode, v0.4.1
atom-typescript, v8.5.0
linter, v1.11.4
pigments, v0.26.0

# Dev
No dev packages

Seems only to happen when there is no file opened. Same counts for issue #95.

Same as #132, only happens when no file is open.

already fixed, use the latest version


Add support for tessellation.

At long last, tessellation comes to MoltenVK! With this change, clients
will now be able to specify tessellation shaders when creating
pipelines, and then draw tessellated patches with them.
Unfortunately, there seem to be a few gotchas with tessellation in
Metal. For one thing, tessellation pipelines in Metal are structured
very differently from Vulkan. There is no tessellation control or even
vertex stage. Instead, the tessellation evaluation shader takes the
place of the vertex function as a "post-tessellation vertex function."
The tessellation levels are supplied in a buffer to the tessellator,
which you are expected to populate. The most common way to do this is by
running a compute shader. MoltenVK thus runs the vertex shader and
tessellation control shader by themselves; a single VkPipeline object
then requires at least three MTLPipelineState objects.
But wait, there's more! The tessellation-control-as-compute stage uses
Metal's support for vertex-style stage input to a compute shader. But,
this support requires one to declare indexing ahead of time, when the
pipeline state is created. So a single VkPipeline object could have as
many as five MTLPipelineState objects.
Further, if there are more output than input control points for the
tessellation control stage, then later invocations may end up fetching
the wrong attributes! To get around this, this change uses index buffers
to ensure that all tessellation control shaders see the correct input.
Unfortunately, in the indexed draw case, this means that the incoming
index buffer needs to be munged.
Instancing is another pain point here. In Vulkan, as in OpenGL and
Direct3D, instancing is done in the vertex shader; but in Metal, it is
done at the tessellation evaluation stage. For this reason, only the
vertex stage of a tessellated draw supports instancing. Additional
memory is required to hold data for the extra vertices generated by
instancing. This also requires still more munging of index buffers for
indexed draws.
Indirect draws are even more painful. Because the number of vertices and
instances is unknown, storage for the maximum possible number of
vertices must be allocated. This change imposes a totally arbitrary
limit of 131072 vertices from a single draw, including all vertices
generated by instancing. On a Mac, this requires about 194-256 MB of
VRAM for all the temporary buffers.
There are some possible optimizations here. If we could prove that the
vertex shader's output doesn't depend on the instance ID, either
directly or through a per-instance attribute, then we could avoid
running the vertex and tess. control stages per instance, and take
advantage of Metal's support for tess. eval instancing. If we could
also prove that the vertex shader simply passes instance attributes
through (similarly with the tess. control shader), we could do this for
many more instanced draws as well. It should also be possible to cache
the output from the tess. control stage; if the draw comes up again, we
can then skip the vertex and tess. control stages entirely!
Fixes #56 and #501.

This change is incomplete. A bunch of CTS tests fail because shader I/O doesn't work right, and I haven't figured out why this is yet. Also, at lease one of the tests crashes.

@cdavis5e
Thanks for taking this on. Congratulations! Nice work!
Did you intend to leave this PR marked as Draft? It can't be merged into master while it is marked that way.
Are you wanting folks to evaluate, test, and provide feedback? Or are you waiting until more CTS tests pass?

@cdavis5e
Thanks for taking this on. Congratulations! Nice work!
Did you intend to leave this PR marked as Draft? It can't be merged into master while it is marked that way.

Yes. Like I said, it's incomplete.

Are you wanting folks to evaluate, test, and provide feedback? Or are you waiting until more CTS tests pass?

I'd appreciate feedback on this. And maybe with more eyes, it'll be easier to figure out why the CTS tests fail.

For the Vulkan portability initiative...based on your descriptions above...or from the CTS results...it would be helpful if we could identify the Tessellation features that are inherently unsupportable in Metal...and expose them in the Vulkan portability extension VkPhysicalDevicePortabilitySubsetFeaturesEXTX struct.

For the Vulkan portability initiative...based on your descriptions above...or from the CTS results...it would be helpful if we could identify the Tessellation features that are inherently unsupportable in Metal...and expose them in the Vulkan portability extension VkPhysicalDevicePortabilitySubsetFeaturesEXTX struct.

About the only one I can think of is isoline tessellation. It's not directly supported by Metal, and it's difficult to emulate. PointsMode might actually be another. I don't know yet if Metal will see that I set the primitiveTopology in the pipeline state to Point and render accordingly. Probably not.

I'd appreciate feedback on this. And maybe with more eyes, it'll be easier to figure out why the CTS tests fail.

As a start...I blindly extracted all the *.tessellation.* CTS 1.1.4 tests and tried running them.
Some passes. A number of fails (eg. dEQP-VK.draw.shader_layer.tessellation_shader_1) reporting:
Vertex attribute gl_Position(1) is missing from the vertex descriptor.

And dEQP-VK.tessellation.tesscoord.triangles_equal_spacing crashes on EXC_BAD_ACCESS.
Do you want to lay down some direction as to which CTS testing is appropriate for this enhancement PR?

I'd appreciate feedback on this. And maybe with more eyes, it'll be easier to figure out why the CTS tests fail.

As a start...I blindly extracted all the *.tessellation.* CTS 1.1.4 tests and tried running them.
Some passes. A number of fails (eg. dEQP-VK.draw.shader_layer.tessellation_shader_1) reporting:
Vertex attribute gl_Position(1) is missing from the vertex descriptor.

Haven't tried running those yet.

And dEQP-VK.tessellation.tesscoord.triangles_equal_spacing crashes on EXC_BAD_ACCESS.

On one of those testcases, Metal winds up generating far more vertices--about 1000 more--than the test was expecting. Try increasing the slop space to 2048 extra vertices on line 505 in vktTessellationCoordinatesTests.cpp.

Do you want to lay down some direction as to which CTS testing is appropriate for this enhancement PR?

So far, I've been running the dEQP-VK.tessellation.* tests. For me, the tests stop now when I get to  dEQP-VK.tessellation.invariance.tess_coord_component_range.triangles_equal_spacing_ccw_point_mode with:
Assertion 'numVertices <= maxNumVerticesInDrawCall' failed at ../external/vulkancts/modules/vulkan/tessellation/vktTessellationInvarianceTests.cpp:2104

Pulled in your latest...plus latest SPIRV-Cross.
Seeing a number of the following:
Test case 'dEQP-VK.tessellation.tesscoord.triangles_equal_spacing'..
2019-02-22 13:49:31.823096-0500 deqp-vk[13437:9836360] renderPipelineState inputPrimitiveTopology is MTLPrimitiveTopologyClassPoint but primitiveType is not MTLPrimitiveTypePoint

Is that related to the PointMode you mention above?
Also...a number of:
Test case 'dEQP-VK.tessellation.winding.default_domain.glsl_triangles_ccw'..
2019-02-22 13:49:32.418498-0500 deqp-vk[13437:9836360] renderPipelineState inputPrimitiveTopology is MTLPrimitiveTopologyClassTriangle but primitiveType is neither MTLPrimitiveTypeTriangle nor MTLPrimitiveTypeTriangleStrip

Then dEQP-VK.tessellation.shader_input_output.patch_vertices_5_in_10_out crashes with SIGABRT.
Do you want me to aid it debugging and fixing any of these...or is it best to leave you to it for now? You're probably running the same CTS tests that I am.

I'm not seeing any of this.
SIGABRT usually means that some assert(3) failed. Do you have any "Assertion failed" log lines you could show me that would shed light on this?

Strange that we are getting different CTS results.
I'm running the 1.1.4 CTS suite on the portability-dev branch retrieved from Khronos Gerrit. I wouldn't expect that any of that should matter...as portability doesn't touch tessellation right now. But who knows?
What are your results from the 3 tests I mentioned above?
If you let me know which CTS suite and version you are running...I'll give it a go to confirm.
The SIGABRTis originating deep with Metal during the kMVKGraphicsStageVertex stage draw:
Test case 'dEQP-VK.tessellation.shader_input_output.patch_vertices_5_in_10_out'..
2019-02-23 13:35:58.667026-0500 deqp-vk[17922:10359860] renderPipelineState inputPrimitiveTopology is MTLPrimitiveTopologyClassTriangle but primitiveType is neither MTLPrimitiveTypeTriangle nor MTLPrimitiveTypeTriangleStrip
2019-02-23 13:35:58.667118-0500 deqp-vk[17922:10359860] Vertex Function(main0): Bytes are being bound at index 2 to a shader argument with write access enabled.
2019-02-23 13:35:58.678679-0500 deqp-vk[17922:10359860] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[__NSSetM addObject:]: object cannot be nil'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff42cd1ded __exceptionPreprocess + 256
	1   libobjc.A.dylib                     0x00007fff6ed9d720 objc_exception_throw + 48
	2   CoreFoundation                      0x00007fff42d113c0 -[CFPrefsConfigurationFileSource initWithConfigurationPropertyList:containingPreferences:] + 0
	3   CoreFoundation                      0x00007fff42c291ca -[__NSSetM addObject:] + 729
	4   MetalTools                          0x00007fff6217c0c4 +[MTLDebugResourceAccessTracker resourceTrackingRecordAccessesToArguments:bufferFuncArgsPtr:textureFuncArgsPtr:buffersBoundForWrite:texturesBoundForWrite:] + 775
	5   MetalTools                          0x00007fff621a85f9 -[MTLDebugCommandBuffer resourceTrackingRecordAccessesToArguments:bufferFuncArgsPtr:textureFuncArgsPtr:] + 101
	6   MetalTools                          0x00007fff621d2ff0 -[MTLDebugRenderCommandEncoder _resourceTrackingRecordBoundResourceAccesses] + 110
	7   MetalTools                          0x00007fff621d0c0d -[MTLDebugRenderCommandEncoder drawPrimitives:vertexStart:vertexCount:instanceCount:baseInstance:] + 271
	8   libMoltenVK.dylib                   0x0000000109885e7e _ZN10MVKCmdDraw6encodeEP17MVKCommandEncoder + 1246

BTW...just thinking about the upcoming SDK release...
In addition to CTS...are you running this PR against any real-world games that triggered this PR?
If so...and those tessellated games are running fine...and assuming this PR doesn't break anything existing...I'm good with including it in the upcoming release...and continuing to work on completing CTS conformance in the mainline evolution.

Hmm... It looks like you're running with validation enabled, and it doesn't like that I'm calling setVertexBytes:... on a writable buffer. I was afraid of this.
I'm running public HEAD.

Ah! Indeed. Validation on but set to log instead of assert on errors.
With validation off...dEQP-VK.tessellation.* runs to completion:
Test run totals:
  Passed:        61/432 (14.1%)
  Failed:        233/432 (53.9%)
  Not supported: 138/432 (31.9%)
  Warnings:      0/432 (0.0%)

I have a few more changes to push out, but they're dependent on some SPIRV-Cross changes that haven't been merged yet. With these, I have:
Test run totals:
  Passed:        148/432 (34.3%)
  Failed:        146/432 (33.8%)
  Not supported: 138/432 (31.9%)
  Warnings:      0/432 (0.0%)

The remaining failures:

*isoline*: Obviously.
dEQP-VK.tessellation.shader_input_output.cross_invocation_per_vertex_mat4x3: This happens because SPIRV-Cross doesn't translate a shader properly. The temp generated for the matrix input has the wrong name--or it's later using the wrong name to refer to it.
dEQP-VK.tessellation.shader_input_output.cross_invocation_per_patch_mat4x3: This uses an array of matrices, which SPIRV-Cross doesn't support 😠.
dEQP-VK.tessellation.common_edge.*: No idea.
dEQP-VK.tessellation.fractional_spacing.*: Needs isolines.
dEQP-VK.tessellation.primitive_discard.*: No idea.
dEQP-VK.tessellation.user_defined_io.*: Mysterious failures to link the shader program over a missing user(locn0) varying, even though the tessellation evaluation shader writes to it... 😠Bug in Metal?

Some of these don't even make it to that point, since they use a custom block output, and SPIRV-Cross doesn't translate that correctly, either. I suspect this is related to the cross_invocation_per_vertex_mat4x3 failure.

That's good news!
Once the SPIRV-Cross merge is done...and you add them and your other changes to this PR...are you ready to treat this PR as good-to-go for the upcoming SDK release? Or do you want to leave it in Draft mode for a while longer until you can address the leftovers?
Regarding *isoline* & dEQP-VK.tessellation.fractional_spacing.*...looks like there are 90 of those. If this is something that is just not currently supported by Metal...there is the option of adding an isolineTessellation member to the VkPhysicalDevicePortabilitySubsetFeaturesEXTX struct. With that in place I can modify the CTS portability-dev branch to treat those 90 tests as Not supported instead of Failed.

dEQP-VK.tessellation.common_edge.*: No idea.

Figured it out. I was setting the stride wrong.

dEQP-VK.tessellation.user_defined_io.*: Mysterious failures to link the shader program over a missing user(locn0) varying, even though the tessellation evaluation shader writes to it... 😠Bug in Metal?

Gah. I know why this doesn't work. The tessellation evaluation shader writes to a buffer. When you do this in Metal, you can't do rasterization, same as with a vertex shader. So the shader ends up not writing to the given varying, and the link fails. I'm not sure this can easily be made to work.

That's good news!
Once the SPIRV-Cross merge is done...and you add them and your other changes to this PR...are you ready to treat this PR as good-to-go for the upcoming SDK release?

Fine by me. This seems to work OK now, more or less. I'm probably going to try this on a real game tomorrow.

Regarding *isoline* & dEQP-VK.tessellation.fractional_spacing.*...looks like there are 90 of those. If this is something that is just not currently supported by Metal...there is the option of adding an isolineTessellation member to the VkPhysicalDevicePortabilitySubsetFeaturesEXTX struct. With that in place I can modify the CTS portability-dev branch to treat those 90 tests as Not supported instead of Failed.

Sounds good. I guess we could bring this up at the next meeting.

Figured it out.
Gah. I know why this doesn't work.

Nice work!

Fine by me. This seems to work OK now, more or less. I'm probably going to try this on a real game tomorrow.

Happy to hear that! Once you're happy with the CTS and real-game results...release this PR from Draft status. At that point...I'll run some additional smoke tests to make sure it doesn't ambush existing functionality...and then I'll pull it in.

Sounds good. I guess we could bring this up at the next meeting.

Yup...that would be good. So we have isoline at least. If there are others (perhaps the "tessellation evaluation shader writes to a buffer" issue you mentioned?)...maybe you could raise them as possible feature flags too. When you get a chance...perhaps send an email to the portability mail list suggesting additional feature flags a few days before the next meeting (Mar 8) to prep the discussion.

Merged the latest trio of PRs in SPIRV-Cross.

Curiouser and curiouser.
Just spent quite a while beating my head against the proverbial wall running this against published Dota2.
vkCreateDevice() causes _MVKStaticCmdShaderSource to strangely fail to compile with a bunch of:
program_source:160:101: error: unknown type name 'MTLDrawPatchIndirectArguments'; did you mean 'MTLDrawPrimitivesIndirectArguments'?
    device char* dest = destBuff + idx * (sizeof(MTLDispatchThreadgroupsIndirectArguments) + sizeof(MTLDrawPatchIndirectArguments));
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                                                                MTLDrawPrimitivesIndirectArguments

It turns out that MVKDevice::getMTLCompileOptions() is unexpectedly returning a MTLCompileOptions instance whose languageVersion parameter is set to MTLLanguageVersion1_1 instead of MTLLanguageVersion2_1.
This is probably related to a similar item you speculated on in the DXVK issue.
So...a couple of thoughts:

_MVKStaticCmdShaderSource should adapt to MSL versions to conditionally elide MSL artifacts that are MSL version dependent.

MVKDevice::getMTLCompileOptions() should probably force the languageVersion to the latest MSL version available as determined by feature sets. We already track this in MVKDevice so it should be an easy change.

I've confirmed that (2) fixes the issue for Dota2...so I'll polish it up and do a separate PR for that.
Can you take care of (1) as part of this PR...otherwise we might see legitimate crashes if people really run on macOS 10.11 / iOS 9.0?

@cdavis5e

I'm investigating...but nothing obvious yet.

I've narrowed the memory issues down to the [first tessellation commit to SPIRV-Cross](url
https://github.com/KhronosGroup/SPIRV-Cross/commit/eb89c3a428b20627d0f5f02b5f40de2ad7df2a3e). The Dota2 crashes happen with non-tessellation MoltenVK with that SPRIV-Cross commit.

Thanks. Unfortunately, that's a huge change. I need to narrow that down some more.
I'm probably going to run the CTS against Address Sanitizer. Hopefully that will turn up something.

I'm probably going to run the CTS against Address Sanitizer. Hopefully that will turn up something.

I tried running Address Sanitizer against the Cube and Hologram demos with this full PR without finding anything. Unfortunately when I tried doing that against Dota2 it encounters other address issues that trigger before this one.
As is often the case with address issues...the crashes don't occur immediately on the offending code. But as far as I can tell...when it does crash while in SPIRV-Cross...it occurs when several instances of CompilerMSL are running simultaneously on different threads. So it appears to be a threading issue during simultaneous shader conversions...which Dota2 uses aggressively.
But I haven't seen a case yet where it uses the same instance of CompilerMSL...and that would appear to be impossible anyway because ComplerMSL is instantiated dynamically for each conversion by a SPIRVToMSLConverter.

Unfortunately when I tried doing that against Dota2 it encounters other address issues that trigger before this one.

Try running with ASAN_OPTIONS=detect_container_overflow=0. I get a lot of false positives from that check. (At least I think they're false positives... They happen just from the normal operation of libc++'s containers.)

I wonder if Thread Sanitizer will turn something up...

A wild shot in the dark here might be the ClassicLocale class in spirv_common.hpp that's used to force a classic C locale. I suspect it might not be thread safe. Never had any issues with it, but worth a shot to remove it.

A wild shot in the dark here might be the ClassicLocale class in spirv_common.hpp that's used to force a classic C locale. I suspect it might not be thread safe. Never had any issues with it, but worth a shot to remove it. However, it would be very strange if this is only an issue now all of a sudden ...

A good suggestion. But unfortunately removing ClassicLocale does not fix the issue.
However...what threading impact does ClassicLocale have? It seems to be setting and resetting some kind of global configuration during each run. If so...that does sound potentially dangerous for parallel execution!

That's a good question, when I added it I believe this is thread local, but that might not actually be true. I'll need to investigate. The reason for it is that we need C locale when using sprintf because some locales use comma instead of point for decimals. I'll investigate if there's a good stateless alternative we can use, although I guess it won't really affect this issue in particular.

Thread Sanitizer didn't really turn up anything, either.

I did however fix a problem I found with non-tessellated indirect draws. Given that you found that Dota2 crashed with just the tessellation control patch to SPIRV-Cross, I doubt it'll help, but maybe it will?

@cdavis5e
Well...I haven't been able to figure this Dota2 memory issue out yet.
I have measured that SPIRV-Cross with the tessellation enhancements seems to run quite a bit slower than previously. Does that sound reasonable?
I'm wondering if what is happening is that the slower performance is affecting the threading in Dota2...to the point where it is highlighting memory/threading issues in the game that are not actually originating in SPIRV-Cross.
Have you run this PR against any other significant games? What are the results?

@cdavis5e
Well...I haven't been able to figure this Dota2 memory issue out yet.
I have measured that SPIRV-Cross with the tessellation enhancements seems to run quite a bit slower than previously. Does that sound reasonable?

Quite a bit, you say? That's a problem. I was expecting a slight performance hit, but not that much. Shader compilation is already a massive time sink, even with native Vulkan. I need to do some profiling and see if I can make this go faster.

I'm wondering if what is happening is that the slower performance is affecting the threading in Dota2...to the point where it is highlighting memory/threading issues in the game that are not actually originating in SPIRV-Cross.
Have you run this PR against any other significant games? What are the results?

Not yet. I'm trying to run this against Final Fantasy XIV, which is what prompted this in the first place. I've also installed Dota2 from Steam; maybe I'll be able to reproduce the problem locally.

Btw, there is an --iterations argument in SPIRV-Cross CLI which can be used for some crude profiling. Might be worth a look.

@cdavis5e

Quite a bit, you say? That's a problem. I was expecting a slight performance hit, but not that much. Shader compilation is already a massive time sink, even with native Vulkan. I need to do some profiling and see if I can make this go faster.

"Quite a bit" is relative of course. It fluctuates...but seems to take maybe 25%-30% longer.
With a Debug build of MoltenVK, you can use the MVK_CONFIG_PERFORMANCE_TRACKING env var at runtime to have MoltenVK log various shader handling performance. Look for lines like this:
[mvk-debug] Performance to convert SPIR-V to MSL source code curr: 0.506 ms, avg: 7.290 ms, min: 0.506 ms, max: 107.898 ms, count: 252

[mvk-debug] Performance to convert SPIR-V to MSL source code curr: 8.031 ms, avg: 5.807 ms, min: 1.114 ms, max: 64.846 ms, count: 252

The first is from a run that includes SPIRV-Cross tessellation and which crashes, the second is without SPIRV-Cross tessellation, and does not crash. I'm generally comparing the avg...but the max is also interesting...but might be a fluke.
@HansKristian-Work

Btw, there is an --iterations argument in SPIRV-Cross CLI which can be used for some crude profiling. Might be worth a look.

Thanks!

Not yet. I'm trying to run this against Final Fantasy XIV, which is what prompted this in the first place. I've also installed Dota2 from Steam; maybe I'll be able to reproduce the problem locally.

We'll need to get to the bottom of this soon. I'm not comfortable pulling in SPIRV-Cross eb89c3a or later until we understand this...which inhibits our ability to include any SPIRV-Cross enhancements.
Essentially we need to figure out if this is a SPIRV-Cross issue with any sufficiently-complex app...or is specific to Dota2.
An alternate option might be to create a branch in SPIRV-Cross that excludes (or includes) the tessellation commits until we figure it out. But I that would be my last choice.

I added support for ubsan/asan/msan/tsan on my c-api branch, and the test suite runs clean on Linux at least.

I was able to reproduce the crash with commit: aab4a5632b4953d1ce1d20bd2bc0ed5748fe2d07.

@HansKristian-Work

I was able to reproduce the crash with commit: aab4a5632b4953d1ce1d20bd2bc0ed5748fe2d07. As mentioned earlier, it's crashing in a completely weird and unrelated place.

What testing were you doing that caused this crash?

I started the game totally fresh, then entered the 1. tutorial called "Mechanics". During the loading screen, the error would occur.

I started the game totally fresh, then entered the 1. tutorial called "Mechanics". During the loading screen, the error would occur.

Sigh. I hit the close button by mistake with a literal finger twitch as I was moving my mouse. Reopened now. Sorry. 🙄

There seems to have been an update today on DOTA, but it didn't change much, except the shaders seem different. I trigger far fewer warnings.
Been looking into this some more today. This is a serious ghost bug. Some observations.

If I preload the asan library, I cannot reproduce the issue. I cannot find any issues when MoltenVK and the deps are built with asan/ubsan. I get one error in DOTA on boot though, but it seems like a false positive.
With tsan, MoltenVK has a lot of weird errors. I think there were some fixes for that in this PR with some false positive workarounds. Either way, I couldn't see any issues in SPIRV-Cross.
On a crash, no threads are active in SPIRV-Cross. I'm testing this on a dual-core HT macbook pro though.
To debug with lldb you have to do: GAME_DEBUGGER=/Applications/Xcode.app/Contents/Developer/usr/bin/lldb bash "$(pwd)/dota.sh". Just lldb will block DYLD_LIBRARY_PATH or something silly ...

This is a common backtrace, and I see it almost every time:
frame #0: 0x00007fff74277969 libsystem_platform.dylib`_platform_bzero$VARIANT$Haswell + 41
    frame #1: 0x0000000087f0fd61 librendersystemvulkan.dylib`___lldb_unnamed_symbol1018$$librendersystemvulkan.dylib + 353
    frame #2: 0x0000000087f0fbb3 librendersystemvulkan.dylib`___lldb_unnamed_symbol1017$$librendersystemvulkan.dylib + 1395
    frame #3: 0x0000000087f33032 librendersystemvulkan.dylib`___lldb_unnamed_symbol1255$$librendersystemvulkan.dylib + 162
    frame #4: 0x000000009164b34e libscenesystem.dylib`___lldb_unnamed_symbol2365$$libscenesystem.dylib + 286
    frame #5: 0x000000009158e729 libscenesystem.dylib`___lldb_unnamed_symbol1231$$libscenesystem.dylib + 9561
    frame #6: 0x000000009156a4dc libscenesystem.dylib`___lldb_unnamed_symbol1027$$libscenesystem.dylib + 476

If I don't get a crash, I get malloc assertions about broken checksums, etc.

Been looking into this some more today. This is a serious ghost bug. Some observations.

Thanks for keeping on this.

If I preload the asan library, I cannot reproduce the issue. I cannot find any issues when MoltenVK and the deps are built with asan/ubsan. I get one error in DOTA on boot though, but it seems like a false positive.
With tsan, MoltenVK has a lot of weird errors. I think there were some fixes for that in this PR with some false positive workarounds. Either way, I couldn't see any issues in SPIRV-Cross.

How are you loading as asan/tsan? I was finding that difficult. I too hit the false positive...but it wouldn't go any further. Is there a way of having asan log but continue on errors?

On a crash, no threads are active in SPIRV-Cross. I'm testing this on a dual-core HT macbook pro though.

Historically (I haven't tried with the Dota2 update you mention) I sometimes I'll see it crash either in the middle of SPIRV-Cross...or while MoltenVK is logging the shader conversion. And even in the cases where the crash happens with no SPIRV-Cross active...almost every time, the last thing that appears in the log is a shader conversion.
So it may not be happening within SPRIV-Cross...but there may be some conflict between memory that SPRIV-Cross and Dota2 are using.

To debug with lldb you have to do: GAME_DEBUGGER=/Applications/Xcode.app/Contents/Developer/usr/bin/lldb bash "$(pwd)/dota.sh". Just lldb will block DYLD_LIBRARY_PATH or something silly ...

This is a common backtrace, and I see it almost every time:
frame #0: 0x00007fff74277969 libsystem_platform.dylib`_platform_bzero$VARIANT$Haswell + 41
    frame #1: 0x0000000087f0fd61 librendersystemvulkan.dylib`___lldb_unnamed_symbol1018$$librendersystemvulkan.dylib + 353
    frame #2: 0x0000000087f0fbb3 librendersystemvulkan.dylib`___lldb_unnamed_symbol1017$$librendersystemvulkan.dylib + 1395
    frame #3: 0x0000000087f33032 librendersystemvulkan.dylib`___lldb_unnamed_symbol1255$$librendersystemvulkan.dylib + 162
    frame #4: 0x000000009164b34e libscenesystem.dylib`___lldb_unnamed_symbol2365$$libscenesystem.dylib + 286
    frame #5: 0x000000009158e729 libscenesystem.dylib`___lldb_unnamed_symbol1231$$libscenesystem.dylib + 9561
    frame #6: 0x000000009156a4dc libscenesystem.dylib`___lldb_unnamed_symbol1027$$libscenesystem.dylib + 476

If I don't get a crash like this, I get malloc assertions about broken checksums, etc, which then crash.

@cdavis5e Have you tried running this PR against other complex games? You mentioned FF, for example.
@danginsburg Are you able to give us a hand looking into this? I'm hoping that you can run this with additional debugging and inspection capabilities within Data2. To summarize...this PR (actually even just SPIRV-Cross eb89c3a) is encountering random memory conflicts with Dota2. It might bet thread related. It doesn't seem to be happening elsewhere.

@billhollings To disable halt on error, you can pass in:
ASAN_OPTIONS=halt_on_error=0 STEAM_DYLD_INSERT_LIBRARIES=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/10.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib bash "$(pwd)/dota.sh

Yeah, I'm not confident I can find this issue. I've reviewed the PR again, and I can't see anything risky with memory, and given I get no asan/msan errors there isn't much to go on.

All right, I've tested this with FFXIV, and I'm actually able to see the water now on high tessellation! It's kinda slow, though. I haven't seen any crashes so far; I guess DXVK is a bit less aggressive about compiling shaders on other threads.
There is one other problem though, that predates this change. Our testers report that FFXIV has a tendency to crash after about an hour with no explanation or error message. I'm wondering if this is somehow related to the crashes you saw in Dota2--i.e. this is a preexisting problem, and the change to SPIRV-Cross just exposed it. I need to try playing for longer and seeing if I can replicate the crashes that our testers found. Maybe it'll happen sooner with these changes in place.

Well, that crash has nothing to do with the crash in Dota2.
There's got to be a reason for this. I don't want to destabilize Dota2 just so I can use tessellation elsewhere. I wonder if we could get Valve to test a debug version of Dota2 with these changes to MoltenVK.

@cdavis5e I've been looking at it today.  One thing I pointed out to Bill is that this error:
[***MoltenVK ERROR***]: VK_ERROR_INITIALIZATION_FAILED: Under Metal, vertex attribute offsets must not exceed the vertex buffer stride.
is causing pipeline creation to fail for some of our pipelines, which makes the water not render properly in dota.  I wonder if the crash is downstream of this in our handling of pipeline creation failures.  The case in which we're hitting this looks like this:
(const VkVertexInputBindingDescription *) $7 = 0x00007000088a6110 {
  (VkVertexInputBindingDescription) [0] = (binding = 0, stride = 24, inputRate = VK_VERTEX_INPUT_RATE_VERTEX)
  (VkVertexInputBindingDescription) [1] = (binding = 1, stride = 4, inputRate = VK_VERTEX_INPUT_RATE_INSTANCE)
  (VkVertexInputBindingDescription) [2] = (binding = 2, stride = 0, inputRate = VK_VERTEX_INPUT_RATE_INSTANCE)
}
(lldb) parray 4 vertexInputCreateInfo.pVertexAttributeDescriptions
(const VkVertexInputAttributeDescription *) $8 = 0x00007000088a61d0 {
  (VkVertexInputAttributeDescription) [0] = (location = 0, binding = 0, format = VK_FORMAT_R32G32B32_SFLOAT, offset = 0)
  (VkVertexInputAttributeDescription) [1] = (location = 1, binding = 2, format = VK_FORMAT_R32G32B32A32_SFLOAT, offset = 0)
  (VkVertexInputAttributeDescription) [2] = (location = 2, binding = 2, format = VK_FORMAT_R32G32B32A32_SFLOAT, offset = 0)
  (VkVertexInputAttributeDescription) [3] = (location = 3, binding = 1, format = VK_FORMAT_R16G16_SFLOAT, offset = 0)

Is this really a case MoltenVK can't handle?  It's worked on Metal for a long time.  Not sure I understand the MoltenVK change here.
I asked Bill to find out if it still crashes if he avoids that check.
I've run Dota 2 in debug and haven't been able to get it to crash.  The ASAN crash is something I'm looking into, but it looks unrelated to this and is a buffer read overrun so probably shouldn't crash.

@cdavis5e I've been looking at it today. One thing I pointed out to Bill is that this error:
[***MoltenVK ERROR***]: VK_ERROR_INITIALIZATION_FAILED: Under Metal, vertex attribute offsets must not exceed the vertex buffer stride.
is causing pipeline creation to fail for some of our pipelines, which makes the water not render properly in dota. I wonder if the crash is downstream of this in our handling of pipeline creation failures. The case in which we're hitting this looks like this:
(const VkVertexInputBindingDescription *) $7 = 0x00007000088a6110 {
  (VkVertexInputBindingDescription) [0] = (binding = 0, stride = 24, inputRate = VK_VERTEX_INPUT_RATE_VERTEX)
  (VkVertexInputBindingDescription) [1] = (binding = 1, stride = 4, inputRate = VK_VERTEX_INPUT_RATE_INSTANCE)
  (VkVertexInputBindingDescription) [2] = (binding = 2, stride = 0, inputRate = VK_VERTEX_INPUT_RATE_INSTANCE)
}
(lldb) parray 4 vertexInputCreateInfo.pVertexAttributeDescriptions
(const VkVertexInputAttributeDescription *) $8 = 0x00007000088a61d0 {
  (VkVertexInputAttributeDescription) [0] = (location = 0, binding = 0, format = VK_FORMAT_R32G32B32_SFLOAT, offset = 0)
  (VkVertexInputAttributeDescription) [1] = (location = 1, binding = 2, format = VK_FORMAT_R32G32B32A32_SFLOAT, offset = 0)
  (VkVertexInputAttributeDescription) [2] = (location = 2, binding = 2, format = VK_FORMAT_R32G32B32A32_SFLOAT, offset = 0)
  (VkVertexInputAttributeDescription) [3] = (location = 3, binding = 1, format = VK_FORMAT_R16G16_SFLOAT, offset = 0)

Is this really a case MoltenVK can't handle? It's worked on Metal for a long time. Not sure I understand the MoltenVK change here.

I think I got the boundary condition wrong in that check. I added it after encountering some cases in the CTS where offset > stride, which Metal protested loudly against (i.e. killing the process). @billhollings wrote #522 to fix that.

One thing I pointed out to Bill is that this error:...is causing pipeline creation to fail for some of our pipelines

PR #522 fixes this and the pipeline failures in Dota2. The code that broke this appeared prior to the tessellation PR...but after Dota2's MoltenVK version.

Is this really a case MoltenVK can't handle? It's worked on Metal for a long time. Not sure I understand the MoltenVK change here.

No...it's fine. The code that tested for offsets and strides was over-aggressive. It should permit a zero offset and stride combo...and now does.
I'm testing the updated fix now against this tessellation PR.

@cdavis5e
PR #522 conflicts with this PR in MVKPipeline.mm. To merge...accept the changes from #522...but return false instead of nil...and use your pVI instead of pCreateInfo->pVertexInputState.

Sadly #522 does not fix the primary crashes reported above. So it's unrelated to the pipeline failures that were happening in Dota2. 😭
It may be a twisted hint that no crashes or errors appear while ASAN is running. So however ASAN is altering memory access is smothering the memory errors without detecting them. 🤔

I didn't mention it before because ASAN would seem to be more powerful, but if it's not helping… Have you tried using MallocScribble=1, MallocCheckHeapStart=1000, and/or CFZombieLevel=65537 (the scribble bits)?

Valgrind is also an option. However, it does not work yet on macOS 10.14 sadly.

So I'm unfortunately having trouble reproducing the crash in our development branch, but I did repro the crash in Release.  This was the callstack I saw.
0   libsystem_malloc.dylib        	0x00007fff794ff9ca free_tiny + 330
1   libengine2.dylib              	0x00000000112b8b2d operator delete(void*) + 29
2   libMoltenVK.dylib             	0x0000000086a5ca18 spirv_cross::CompilerGLSL::branch_to_continue(unsigned int, unsigned int) + 280
3   libMoltenVK.dylib             	0x0000000086a5d053 spirv_cross::CompilerGLSL::branch(unsigned int, unsigned int, unsigned int, unsigned int) + 531
4   libMoltenVK.dylib             	0x0000000086a5ac19 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 2105
5   libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
6   libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
7   libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
8   libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
9   libMoltenVK.dylib             	0x0000000086a5d005 spirv_cross::CompilerGLSL::branch(unsigned int, unsigned int, unsigned int, unsigned int) + 453
10  libMoltenVK.dylib             	0x0000000086a5ac19 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 2105
11  libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
12  libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
13  libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
14  libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
15  libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
16  libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
17  libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
18  libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
19  libMoltenVK.dylib             	0x0000000086a5aa17 spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 1591
20  libMoltenVK.dylib             	0x0000000086a0571e spirv_cross::CompilerGLSL::emit_function(spirv_cross::SPIRFunction&, spirv_cross::Bitset const&) + 1422
21  libMoltenVK.dylib             	0x000000008697a92c spirv_cross::CompilerMSL::compile() + 2108
22  libMoltenVK.dylib             	0x000000008697ff21 spirv_cross::CompilerMSL::compile(std::__1::vector<spirv_cross::MSLVertexAttr, std::__1::allocator<spirv_cross::MSLVertexAttr> >, std::__1::vector<spirv_cross::MSLResourceBinding, std::__1::allocator<spirv_cross::MSLResourceBinding> >) + 449
23  libMoltenVK.dylib             	0x0000000086950189 mvk::SPIRVToMSLConverter::convert(mvk::SPIRVToMSLConverterContext&, bool, bool, bool) + 1625
24  libMoltenVK.dylib             	0x00000000869410eb MVKShaderModule::convert(mvk::SPIRVToMSLConverterContext*) + 75
25  libMoltenVK.dylib             	0x0000000086940f76 MVKShaderLibraryCache::getShaderLibrary(mvk::SPIRVToMSLConverterContext*, MVKShaderModule*, bool*) + 118
26  libMoltenVK.dylib             	0x0000000086920e19 MVKPipelineCache::getShaderLibrary(mvk::SPIRVToMSLConverterContext*, MVKShaderModule*) + 265
27  libMoltenVK.dylib             	0x0000000086941591 MVKShaderModule::getMTLFunction(mvk::SPIRVToMSLConverterContext*, VkSpecializationInfo const*, MVKPipelineCache*) + 113
28  libMoltenVK.dylib             	0x000000008691f315 MVKGraphicsPipeline::getMTLRenderPipelineDescriptor(VkGraphicsPipelineCreateInfo const*) + 677
29  libMoltenVK.dylib             	0x000000008691ee30 MVKGraphicsPipeline::initMTLRenderPipelineState(VkGraphicsPipelineCreateInfo const*) + 32
30  libMoltenVK.dylib             	0x000000008691e25a MVKGraphicsPipeline::MVKGraphicsPipeline(MVKDevice*, MVKPipelineCache*, MVKPipeline*, VkGraphicsPipelineCreateInfo const*) + 810
31  libMoltenVK.dylib             	0x000000008691a68f VkResult MVKDevice::createPipelines<MVKGraphicsPipeline, VkGraphicsPipelineCreateInfo>(VkPipelineCache_T*, unsigned int, VkGraphicsPipelineCreateInfo const*, VkAllocationCallbacks const*, VkPipeline_T**) + 143
32  librendersystemvulkan.dylib   	0x0000000086272b99 0x8621b000 + 359321
33  librendersystemvulkan.dylib   	0x0000000086273b96 0x8621b000 + 363414
34  librendersystemvulkan.dylib   	0x0000000086280fb5 0x8621b000 + 417717
35  librendersystemvulkan.dylib   	0x0000000086280dd8 0x8621b000 + 417240
36  libclient.dylib               	0x0000000093e85050 0x92f1c000 + 16158800
37  libscenesystem.dylib          	0x000000008f2ccd84 0x8f24a000 + 535940
38  libvstdlib.dylib              	0x00000000117989d5 0x11789000 + 63957
39  libscenesystem.dylib          	0x000000008f45091f 0x8f24a000 + 2124063
40  libscenesystem.dylib          	0x000000008f2a9126 0x8f24a000 + 389414
41  libengine2.dylib              	0x00000000110ef98d 0x1107d000 + 469389
42  libengine2.dylib              	0x00000000110e0d2b 0x1107d000 + 408875
43  libengine2.dylib              	0x00000000110e07b2 0x1107d000 + 407474
44  libengine2.dylib              	0x00000000110d41a0 0x1107d000 + 356768
45  libengine2.dylib              	0x00000000110d48d0 0x1107d000 + 358608
46  libengine2.dylib              	0x00000000110d4b5a Source2Main + 298
47  com.dota2                     	0x000000000c2e2d65 0xc2e2000 + 3429
48  libdyld.dylib                 	0x00007fff793523d5 start + 1

By the way, a thought about what be making this hard to repro: if you want to fully invoke all the MSL compiles run to run you need to delete ~/Library/Application Support/Steam/steamapps/common/dota 2 beta/game/dota/shadercache/vulkan/shaders.cache between runs.

@danginsburg
Thanks for the results. Is that stack consistent? I have not seen that particular one yet.
Can you try creating a few crashes and see if it is consistent or bounces around the code randomly?
Thanks...yes...I have been starting and stopping Data2 via Xcode...so it never takes the step to save the shader cache. And I've been verifying that the cache is clear.

@danginsburg

So I'm unfortunately having trouble reproducing the crash in our development branch, but I did repro the crash in Release.

What is different between your development branch and Release?

What is different between your development branch and Release?

A lot.  If need be I can sync up the release branch and try to repro there.

Can you try creating a few crashes and see if it is consistent or bounces around the code randomly?

Here's one I caught running in Xcode just now.  I did see a couple others that were in more random places, but twice it has failed inside spirv-cross.
(lldb) bt
* thread #10, name = 'GlobPool5', stop reason = EXC_BAD_ACCESS (code=1, address=0xf)
  * frame #0: 0x000000008be68680 libMoltenVK.dylib`spirv_cross::Bitset::merge_or(spirv_cross::Bitset const&) + 64
    frame #1: 0x000000008be56152 libMoltenVK.dylib`spirv_cross::Compiler::combined_decoration_for_member(spirv_cross::SPIRType const&, unsigned int) const + 162
    frame #2: 0x000000008bdff0bc libMoltenVK.dylib`spirv_cross::CompilerMSL::member_is_non_native_row_major_matrix(spirv_cross::SPIRType const&, unsigned int) + 60
    frame #3: 0x000000008be7e6c8 libMoltenVK.dylib`spirv_cross::CompilerGLSL::access_chain_internal(unsigned int, unsigned int const*, unsigned int, unsigned int, spirv_cross::AccessChainMeta*) + 3240
    frame #4: 0x000000008be98468 libMoltenVK.dylib`spirv_cross::CompilerGLSL::access_chain(unsigned int, unsigned int const*, unsigned int, spirv_cross::SPIRType const&, spirv_cross::AccessChainMeta*, bool) + 536
    frame #5: 0x000000008be9d0b6 libMoltenVK.dylib`spirv_cross::CompilerGLSL::emit_instruction(spirv_cross::Instruction const&) + 1382
    frame #6: 0x000000008bdeccf8 libMoltenVK.dylib`spirv_cross::CompilerMSL::emit_instruction(spirv_cross::Instruction const&) + 5096
    frame #7: 0x000000008beb645d libMoltenVK.dylib`spirv_cross::CompilerGLSL::emit_block_chain(spirv_cross::SPIRBlock&) + 4221
    frame #8: 0x000000008be6071e libMoltenVK.dylib`spirv_cross::CompilerGLSL::emit_function(spirv_cross::SPIRFunction&, spirv_cross::Bitset const&) + 1422
    frame #9: 0x000000008bdd592c libMoltenVK.dylib`spirv_cross::CompilerMSL::compile() + 2108
    frame #10: 0x000000008bddaf21 libMoltenVK.dylib`spirv_cross::CompilerMSL::compile(std::__1::vector<spirv_cross::MSLVertexAttr, std::__1::allocator<spirv_cross::MSLVertexAttr> >*, std::__1::vector<spirv_cross::MSLResourceBinding, std::__1::allocator<spirv_cross::MSLResourceBinding> >*) + 449
    frame #11: 0x000000008bdab189 libMoltenVK.dylib`mvk::SPIRVToMSLConverter::convert(mvk::SPIRVToMSLConverterContext&, bool, bool, bool) + 1625
    frame #12: 0x000000008bd9c0eb libMoltenVK.dylib`MVKShaderModule::convert(mvk::SPIRVToMSLConverterContext*) + 75
    frame #13: 0x000000008bd9bf76 libMoltenVK.dylib`MVKShaderLibraryCache::getShaderLibrary(mvk::SPIRVToMSLConverterContext*, MVKShaderModule*, bool*) + 118
    frame #14: 0x000000008bd7be19 libMoltenVK.dylib`MVKPipelineCache::getShaderLibrary(mvk::SPIRVToMSLConverterContext*, MVKShaderModule*) + 265
    frame #15: 0x000000008bd9c591 libMoltenVK.dylib`MVKShaderModule::getMTLFunction(mvk::SPIRVToMSLConverterContext*, VkSpecializationInfo const*, MVKPipelineCache*) + 113
    frame #16: 0x000000008bd7a1f0 libMoltenVK.dylib`MVKGraphicsPipeline::getMTLRenderPipelineDescriptor(VkGraphicsPipelineCreateInfo const*) + 384
    frame #17: 0x000000008bd79e30 libMoltenVK.dylib`MVKGraphicsPipeline::initMTLRenderPipelineState(VkGraphicsPipelineCreateInfo const*) + 32
    frame #18: 0x000000008bd7925a libMoltenVK.dylib`MVKGraphicsPipeline::MVKGraphicsPipeline(MVKDevice*, MVKPipelineCache*, MVKPipeline*, VkGraphicsPipelineCreateInfo const*) + 810
    frame #19: 0x000000008bd7568f libMoltenVK.dylib`VkResult MVKDevice::createPipelines<MVKGraphicsPipeline, VkGraphicsPipelineCreateInfo>(VkPipelineCache_T*, unsigned int, VkGraphicsPipelineCreateInfo const*, VkAllocationCallbacks const*, VkPipeline_T**) + 143
    frame #20: 0x000000008e29fb99 librendersystemvulkan.dylib`___lldb_unnamed_symbol983$$librendersystemvulkan.dylib + 2633
    frame #21: 0x000000008e2a0b96 librendersystemvulkan.dylib`___lldb_unnamed_symbol985$$librendersystemvulkan.dylib + 1638
    frame #22: 0x000000008e2adfb5 librendersystemvulkan.dylib`___lldb_unnamed_symbol1074$$librendersystemvulkan.dylib + 405
    frame #23: 0x000000008e2addd8 librendersystemvulkan.dylib`___lldb_unnamed_symbol1073$$librendersystemvulkan.dylib + 24
    frame #24: 0x000000009c43e050 libclient.dylib`___lldb_unnamed_symbol93245$$libclient.dylib + 672
    frame #25: 0x0000000097861d84 libscenesystem.dylib`___lldb_unnamed_symbol1106$$libscenesystem.dylib + 180
    frame #26: 0x00000000022b15ae libvstdlib.dylib`___lldb_unnamed_symbol281$$libvstdlib.dylib + 910
    frame #27: 0x0000000002232f72 libtier0.dylib`CThread::ThreadProc(void*) + 306
    frame #28: 0x000000000222d89c libtier0.dylib`___lldb_unnamed_symbol360$$libtier0.dylib + 412
    frame #29: 0x00000000003c4dc3 libsystem_pthread.dylib`_pthread_body + 126
    frame #30: 0x00000000003c7e8d libsystem_pthread.dylib`_pthread_start + 66
    frame #31: 0x00000000003c3e11 libsystem_pthread.dylib`thread_start + 13

A very likely explanation is that SPIRV-Cross' memory somehow gets corrupted from the outside, address=0xf looks very odd. I haven't been able to repro a crash inside SPIRV-Cross at least. A couple of times it would crash inside the Metal compiler itself, and usually in librendersystemvulkan.dylib.

I believe I have root caused this and it is a bug in Dota that looks unrelated to this PR, other than bad luck.  It only happened in our RETAIL build, but I was able to get a reliable repro there.  The problem stemmed from some uninitialized memory being used as a count for memset.  The count was clamped to an upper bound, but not a lower bound, so when it went negative it corrupted a potentially large area of memory.
I don't believe this PR needs to be held up for Dota.  I've checked in a fix and I'll integrate it to our release branch for a future Dota update.
Thanks everyone for their diligence and sorry for the inconvenience.

Wonderful! Thanks, @danginsburg !

Phew. Great that we finally figured it out.

Hey @billhollings
Quick question: does this add tessellation control as well as evaluation?

Yes, this adds support for both kinds of tessellation shader.

Excellent. Thanks!
https://media1.giphy.com/media/1pofg0cTz5mak/source.gif


Implement parser handling incorrect input in DateTime scalar (#1035)

Description
Raise GraphQL error (without original error) in Date / DateTime / Time parsers when provided incorrect isoformatted string
Types of Changes

[x] Core
[ ] Bugfix
[x] New feature
[ ] Enhancement/optimization
[ ] Documentation

Issues Fixed or Closed by This PR

#1035

Checklist

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] I have tested the changes and verified that they work and don't break anything (as well as I can manage).

Hi, thanks for contributing to Strawberry 🍓!
We noticed that this PR is missing a RELEASE.md file. We use that to automatically do releases here on GitHub and, most importantly, to PyPI!
So as soon as this PR is merged, a release will be made 🚀.
Here's an example of RELEASE.md:
Release type: patch

Description of the changes, ideally with some examples, if adding a new feature.

Release type can be one of patch, minor or major. We use semver, so make sure to pick the appropriate type. If in doubt feel free to ask :)

Thanks for your first contribution to Strawberry @joeydebreuk ! You've been invited to our organisation, check your emails for the invite 😊
Also, feel free to request a sticker by filling this form 👉
https://forms.gle/dmnfQUPoY5gZbVT67


OS process timed out.

I can't seem to get this working.
I'm on SmartOS. Keep getting this all over my couch log:
[Thu, 20 Jun 2013 18:21:21 GMT] [error] [<0.867.0>] OS Process Error <0.891.0> :: {os_process_error,
"OS process timed out."}
[Thu, 20 Jun 2013 18:21:25 GMT] [error] [<0.842.0>] OS Process Error <0.897.0> :: {os_process_error,
"OS process timed out."}

In cli.js I added (a move will probably do)
process.stdin.resume()

at line 60 such that block of code looks like:
process.stdin.resume()
process.stdin.setEncoding('utf8')
process.stdin.pipe(stdin)
process.stdin.resume()

The clue was in creating the log file in /tmp (touch /tmp/couchjs.log) which collected a message similar to "Error: Cannot switch to old mode now."

Oh, thanks!

@JKDingwall I am not 100% sure I see where you added that code? Would you mind making a PR, or just a link to the source you are referring to would be sufficient.
I am looking at line 60 here: https://github.com/iriscouch/couchjs/blob/master/cli.js#L60
(Another problem that I often have is that I push lots of code to GitHub and forget to npm publish. Might that be it?)

From my npm install -g couchjs I seem to have got a version of cli.js at 5a438debb0a8aec7ed67dd52c751623247651cda but that does not include process.stdin.resume() at all so I'm not sure where that came from.  I will try an install of couchjs with npm install <git:// url> and report my experiences against that.

Thank you very much. I am eager to hear your results.

The same error for me, I did a npm install <git:// url> install on a Linux machine, Ubuntu 15.10 with CouchDB 1.6.0, got another cli.js without resume() at line 62 but nodejs views are still crashing the server:
root@teo-Home:/var/log/couchdb# node -v
v0.10.25
root@teo-Home:/var/log/couchdb# tail couch.log
[Sun, 20 Mar 2016 08:55:34 GMT] [debug] [<0.95.0>] Linked process died abnormally: <0.13034.0> (reason: {badarg,
	 [{erlang,
	   port_command,
	   [#Port<0.11269>,
		[<<"[\"reset\",{\"reduce_limit\":true,\"timeout\":5000}]">>,
		 10]],
	   []},
	  {couch_os_process,
	   writejson,
	   2,
	   [{file,
		 "couch_os_process.erl"},
		{line,
		 93}]},
	  {couch_os_process,
	   handle_call,
	   3,
	   [{file,
		 "couch_os_process.erl"},
		{line,
		 183}]},
	  {gen_server,
	   handle_msg,
	   5,
	   [{file,
		 "gen_server.erl"},
		{line,
		 580}]},
	  {proc_lib,
	   init_p_do_apply,
	   3,
	   [{file,
		 "proc_lib.erl"},
		{line,
		 237}]}]})


Is it possible to add ethernet support ?

Hi, I have an ESP32 bord that is connected via Ethernet instead of WIFI. So NO WIFI is configured, only Ehternet.
First part of my esphome config is:
esphome:
  name: kc-868-a64
  platform: ESP32
  board: esp32dev

logger:
  level: VERBOSE

syslog:
  ip_address: "192.168.255.X"
  port: 514
  min_level: VERY_VERBOSE

api:
  encryption:
    key: <mykey>

ota:
  password: <mypassword>

ethernet:
  type: LAN8720
  mdc_pin: GPIO23
  mdio_pin: GPIO18
  clk_mode: GPIO17_OUT
  phy_addr: 0

  manual_ip:
    static_ip: 192.168.255.X
    gateway: <IP_ADDRESS>
    subnet: <IP_ADDRESS>

I was troubleshooting this because I did not get any syslog message on my syslog server. I used "tcpdump" to see if any syslog packet was received by the syslog server -> the answer was NO :-( (I'm an IT specialist so I know what I"m doing)
Then I added the action "syslog.log" to send explicit a syslog message. Then I received this:
[syslog:058]: Tried to send "custom_action"@"My log message" with level 7 but Wifi isn't connected yet
So my question is: as I'm not using WIFI to connect the ESP32, would it be possible to add support for Ethernet ?
Thanks for reponse and support !

I don't have an ESP32 on me to test this right now, nor do I have an Ethernet module for it. But if you feel up to it, you can try to add a condition to check Ethernet connectivity in this file: https://github.com/TheStaticTurtle/esphome_syslog/blob/main/syslog_component.cpp#L57
I'm not certain how it would work on the esp32 idf platform, though, maybe it's as simple as including the Ethernet.h header and getting the link status.
As an ugly patch just to test that syslog/udp even works (because I'm using WiFiUDP) you could try to remove the check completely
Samuel

Thanks for a quick repply !
After looking at the source code on https://github.com/arcao/Syslog

After looking at the source code on https://github.com/arcao/Syslog, I think this your code is not going to work because it uses the WIFI components.

Actually, no it doesn't, while you can give it an instance of WiFiUDP you can also do it for EthernetUDP see their SimpleLogging example. The only thing that I'm not sure about is how to detect that it's using Ethernet.
Samuel

Hi Samuel,
I did some testing and found out that it works with your code. I removed the "if(WiFi.status() != WL_CONNECTED)" part of your code and it works.
I was reading the source code and searching the internet but I can't find out how to check if ethernet is connected.
For me it works, I receive syslog messages. If you ever find out how to check for etherenet connectivity, please let me know.
Thanks for support !

Same issue, removed the WL_CONNECTED check, but still no logs.
Here's my config file:
`esphome:
name: livingroom
libraries:
- "Syslog"
esp32:
board: esp32-evb
framework:
type: arduino
Enable logging
logger:
hardware_uart: "UART1"
Enable Home Assistant API
api:
password: ""
ota:
password: ""
ethernet:
type: LAN8720
mdc_pin: GPIO23
mdio_pin: GPIO18
clk_mode: GPIO0_IN
phy_addr: 0
uart:
rx_pin: GPIO3
tx_pin: GPIO1
baud_rate: 9600
sensor:

platform: dht
model: DHT11
pin: GPIO14
temperature:
name: "Living Room Temperature"
humidity:
name: "Living Room Humidity"
update_interval: 60s
platform: mhz19
co2:
name: "MH-Z19 CO2 Value"
temperature:
name: "MH-Z19 Temperature"
update_interval: 60s
automatic_baseline_calibration: false

external_components:

source: github://TheStaticTurtle/esphome_syslog
components: [syslog]

syslog:
ip_address: "<IP_ADDRESS>"
port: 514
strip_colors: false`

I have modified syslog/syslog_component.cpp, but during compilation it uses .esphome/build/livingroom/src/esphome/components/syslog/syslog_component.cpp, which has original contents. How do I make esphome use modified source instead of original?

I can confirm on my ESP32 WT32-ETH01 this hack works. I commented out the WiFi connection check and now I get rsyslog messages on my server.
Nice to have if this were implemented in a way so that both Ethernet and WiFi work without having to modify the code and pull it locally :)
But nevertheless, great work, love it!!

Hi,
Nice to know 😀!
Currently, I don't have a working esp32 board with Ethernet as I don't really have any project that needs it, so I can't actually test it.
As you saw, it's easy-ish to switch to Ethernet, almost all the code is already compatible thanks to WifiUDP and EthernetUDP using the same UDP class.
I basically need to do two things:

Check is esphome is using Ethernet or Wi-Fi, if I'm not mistaken, recreating a WifiUdp instance for every log line caused some issue for me in the past. So it needs to change only when the esp switched connection methods.
Check if the Ethernet driver has been initialized and that a link is detected.

The tricky thing is using both Wi-Fi and Ethernet since, especially since the esp32 ip stack is apparently not always stable 😅
Would you mind describing how the module behaves if there is no cable plugged in or if the driver didn't initialize for some reason? My experience with WifiUDP is that it will probably fail silently.
One thing, I'm not clear about the docs is that it states that The Ethernet component and the Wi-Fi component may not be used simultaneously, even if both are physically available.
Does that mean it won't let you compile with both modules, or just not running simultaneously?
But it's of course doable. The thing is, as I said, I don't have the capabilities to do it right now. One thing that could work in the meantime is a secondary branch (or even PR) and users could use the experimental branch/pr-id on the external_components import.
Thanks for the update!

On a side note, esphome evolved a lot since I last looked at how it works internally, I'm amazed that this module still works "fine" after going through so many updates 😀! If I get my hands on an Ethernet board, I'll probably redo of most of the module.
Syslog isn't complicated and ditching the *UDP class entirely and instead using a socket is possible as suggested by esphome docs https://esphome.io/cookbook/lambda_magic.html#send-udp-commands. This would not only reduce overhead (Especially if I also ditch the Syslog library), but would also solve compatibility with esp-idf since it would use anything from the Arduino framework 🤔.
I need to tinker with the esp32 and sendto because if there is no connection it will most likely just do nothing intentionally, and I would prefer to write a warning in the log if there is an issue sending something.

You know what, I'll order a board of AliExpress so that I will at least have it, and I'll work on it when I'll have a bit more time (my month is about to get crazy 😪)

Yeah no worries, it's not urgent for me since it works :)
I just tested it, but only OTA since otherwise, I have to solder again and no time for that :).
While I was connected with "esp logs" it worked. Then I pulled the cable, it said "Connection reset by pear", replugged the cable, it didn't work again. Then I CTRL-C, then re-issued the command esphome logs, but still it didn't connect OTA. I had to unplug the power from the ESP32 before it restarted.
I guess more info will be visible if I connect via serial, but as I said, I'm having a hard time finding time for the project I'm using your custom module in.
With regards to recommendations? I don't know, I just bought an ESP32 WT32-eth01 for around 15$/15eur. I guess most of them are the same but not sure :)
[09:26:25][C][mqtt.light:079]:   Command Topic: '12channel-dimmer/light/flow_heater_l1/command'
[09:26:25][C][mqtt.light:078]: MQTT Light 'Flow heater L2':
[09:26:25][C][mqtt.light:079]:   State Topic: '12channel-dimmer/light/flow_heater_l2/state'
[09:26:25][C][mqtt.light:079]:   Command Topic: '12channel-dimmer/light/flow_heater_l2/command'
[09:26:25][C][mqtt.light:078]: MQTT Light 'Flow heater L3':
[09:26:25][C][mqtt.light:079]:   State Topic: '12channel-dimmer/light/flow_heater_l3/state'
[09:26:25][C][mqtt.light:079]:   Command Topic: '12channel-dimmer/light/flow_heater_l3/command'
WARNING 12channel-dimmer.local: Connection error occurred: [Errno 104] Connection reset by peer
INFO Processing unexpected disconnect from ESPHome API for 12channel-dimmer.local
WARNING Disconnected from API
WARNING 12channel-dimmer.local: Connection error occurred: [Errno 104] Connection reset by peer
WARNING Can't connect to ESPHome API for 12channel-dimmer.local: Read failed (ReadFailedAPIError)
INFO Trying to connect to 12channel-dimmer.local in the background

WARNING 12channel-dimmer.local: Connection error occurred: [Errno 104] Connection reset by peer

WARNING 12channel-dimmer.local: Connection error occurred: [Errno 104] Connection reset by peer
WARNING 12channel-dimmer.local: Connection error occurred: [Errno 104] Connection reset by peer
^C(esphome) userwa@macbookuser:~/Documents/smarthome/esphome/tmp$ esphome logs dimpack-controller.yml
INFO ESPHome 2023.8.3
INFO Reading configuration dimpack-controller.yml...
WARNING GPIO4 is a Strapping PIN and should be avoided.
Attaching external pullup/down resistors to strapping pins can cause unexpected failures.
See https://esphome.io/guides/faq.html#why-am-i-getting-a-warning-about-strapping-pins
Found multiple options for logging, please choose one:
  [1] Over The Air (12channel-dimmer.local)
  [2] MQTT (<IP_ADDRESS>)
(number): 1
INFO Starting log output from 12channel-dimmer.local using esphome API
WARNING 12channel-dimmer.local: Connection error occurred: [Errno 104] Connection reset by peer
WARNING Can't connect to ESPHome API for 12channel-dimmer.local: Read failed (ReadFailedAPIError)
INFO Trying to connect to 12channel-dimmer.local in the background
WARNING 12channel-dimmer.local: Connection error occurred: [Errno 104] Connection reset by peer
WARNING 12channel-dimmer.local: Connection error occurred: [Errno 104] Connection reset by peer
WARNING 12channel-dimmer.local: Connection error occurred: [Errno 104] Connection reset by peer

This should now work (for Ethernet, esp-idf, IPv6) with https://github.com/TheStaticTurtle/esphome_syslog/pull/17


[Profile Page | Desktop] Components Implementation

Components Implementation
Profile Page | Desktop
New components available for anyone that wants to contribute to the new design (kibbeh). Make sure to read the guidelines at the bottom of the issue.

New components

[ ] ProfileHeaderWrapper - https://www.figma.com/file/CS01VVLR7ArQl0afYFkNj3/Web-App?node-id=8277%3A977

Requires none ✅

[ ] ProfileHeader - https://www.figma.com/file/CS01VVLR7ArQl0afYFkNj3/Web-App?node-id=8277%3A978

Requires ProfileHeaderWrapper,  UserBadge (for Follows you), Button, UserAvatar/SingleUser ❌

[ ] ProfileTabs - https://www.figma.com/file/CS01VVLR7ArQl0afYFkNj3/Web-App?node-id=8277%3A989

Requires none ✅

[ ] UserBadgeLg - https://www.figma.com/file/CS01VVLR7ArQl0afYFkNj3/Web-App?node-id=8277%3A990

Requires none ✅

[ ] ProfileAbout - https://www.figma.com/file/CS01VVLR7ArQl0afYFkNj3/Web-App?node-id=8277%3A1048

Requires UserBadgeLg ❌

[ ] BaseDropdownSm

Includes:

CalendarDropdown - https://www.figma.com/file/CS01VVLR7ArQl0afYFkNj3/Web-App?node-id=8277%3A1049
ShareDropdown - https://www.figma.com/file/CS01VVLR7ArQl0afYFkNj3/Web-App?node-id=8277%3A1054

Requires none ✅

[ ] UpcomingRoomCardLg - https://www.figma.com/file/CS01VVLR7ArQl0afYFkNj3/Web-App?node-id=8277%3A1062

Requires CalendarDropdown, ShareDropdown, UserAvatar/MultipleUsers, Tag ❌

Guidelines:

Don't add margin around the components.
The bordered rectangle around each component in Figma is just for delimitation purposes.
Put your component in kibbeh/src/ui and create a story in kibbeh/src/stories.
Use tailwind for styling (see the tailwind config for theme info).
You have to use TypeScript.
Work on the staging branch.
Please comment below if you start to work on a component, but only claim a component if you plan to work on it right away, so it doesn't block other people.
Don't start working on a component if its dependencies aren't done.

Mockup & grid: https://www.figma.com/file/CS01VVLR7ArQl0afYFkNj3/Web-App?node-id=4911%3A0

Most icons should be available under kibbeh/src/icons, but if the one you need isn't, ask on Discord or place a temp one with a comment pointing it out

If you have any questions ask Ben or @ ajb on Discord.

I'm claiming Calendar Dropdown

I'll do UserBadgeLg and possibly do ProfileAbout  once I've finished it.

I'm claiming "Calendar Dropdown" and "Share Dropdown" because both are practically the same.

Merged into BaseDropdown, check my edit

I'm on UserBadgeLg

I'm claiming "Calendar Dropdown" and "Share Dropdown" because both are practically the same.

Merged into BaseDropdownSm, check my edit

Just to clarify, This means that I'll only create "BaseDropdownSm" component right? (The component will accept children)

I'm claiming "Calendar Dropdown" and "Share Dropdown" because both are practically the same.

Merged into BaseDropdownSm, check my edit

Just to clarify, This means that I'll only create "BaseDropdownSm" component right? (The component will accept children)

Yup

I'm done with UserBadgeLg, waiting on @benawad to fix Storybook so I can PR

I will do ProfileHeaderWrapper

Is anyone working on the profileAbout?

I'll do ProfileTabs

I will do CalendarDropdown

@goldyydev there is a storybook issue huh? I thought I was tripping out 😆.

Storybook issue now fixed

I will do UpcomingRoomCardLg

@sahilmob one thing about the ProfileHeaderWrapper, I believe the width is too much. Check out my PR and look at the story: #2111

@sahilmob one thing about the ProfileHeaderWrapper, I believe the width is too much. Check out my PR and look at the story: #2111

I believe ProfileHeaderWrapper width should be bounded by its parent width

Any abandoned components yet?


Cached Key tensors are in float16 for models using RoPE

Not all models apply the position embeddings at the same level, and most recent models apply them on the K projections using RoPE.
This leads to an immediate dequantization of the Key projections.
A possible workaround that issue would be to:

quantize the RoPE sin and cos values with a scale of 1,
support the explicit downscale of the mul operation by 1/max(itype), which would allow to keep cos/sin * k quantized,
support the add feature again when both inputs have the same scale, this time limiting the bitwidth to the target type.

This should be fixed by enhancing dispatched operations for float8 activations.


some updates

Hi @MarkPieszak ,
i started some work in my fork of this repo. main goal is to bring it more up to par with the other aspnetcore starters (knockout/react)
https://github.com/vip32/aspnetcore-Vue-starter

same webpack/npm setup
some folder structure
site.css added
bootstrap added
vs2017 csproj
SampleDataController like other starters

next thing:

bring the demo app alive with bootstrap
typescript
dotnet new capability

for now i have removed the server side rendering, not so convinced it is needed in a non advanced starter. also vue apps should already load fast (due to footprint), compared to ng. for crawlers it is interesting of course.

Just looked at the changes and they look great! Nice job! :100:
Did you want to make a PR so we can implement everything or not yet?

yes let me submit a PR now, then i will continue on the app itself (so it works the same as the other starters)

Merged! 👍👍


Prepare for upcoming breaking changes in FastBoot 1.0

A few things will be gone with the upcoming FastBoot 1.0 release:

no more process.env.EMBER_CLI_FASTBOOT (as there is just one unified build step)
no initializer/(browser|fastboot)/* (again, just one build)

The changes here should continue to work with pre 1.0 and also support post 1.0 versions.

Btw, here is some context: https://github.com/ember-fastboot/ember-cli-fastboot/issues/360
Not sure what's happening with Travis again, cannot install phantomjs?

The failing tests are due to https://github.com/yarnpkg/yarn/issues/3138 😞

@simonihmig should we force travis to use a later node version?

Yes, we could so, as a workaround, but should revert once the yarn bug is fixed!


[Task]: Document application architecture

Task description
We currently have no documentation regarding the overall architecture of Firefly nor how it uses the underlying wallet.rs library. All developers would benefit from understanding this information thoroughly, so it is important to have.
Requirements
TBD
Acceptance criteria
Meets the above (TBD) requirements.
Creation checklist

[X] I have assigned this task to the correct people
[X] I have added the most appropriate labels
[X] I have linked the correct milestone and/or project

This will (most likely) exclude or at least stub out the section for Firefly web (separate task is being tracked here).


fix: add types export

this issue is included in the latest version of Vue & Nuxt.

solved this problem.

Is there anything blocking this PR to be merged? Can this be merged ASAP?


Polymer 3 app on build version fails : Uncaught ReferenceError: Polymer is not defined

I develop an app with polymer 3 starter kit with the general structure :
index.html
src/
|_ my-app.js
|_ my-transaction.js
|_ my-manual.js
modules/
|_ data-transaction.js

index.html imports the my-app.js module like so :
<script type="module" src="src/my-app.js"></script>
my-app.js imports both my-transaction and my-manual dynamically once route changes like so :
case 'transaction':
   import('./my-transaction.js');
   break;
case 'manual':
   import('./my-manual.js');
   break;

both my-transaction.js and my-manual.js import the same module data-transaction like so :
import '../modules/data-transaction.js';
Everything work well on development version and I can operate the app on browser correctly. Then I deploy the production version by "polymer build" and succeed. This is my polymer.json :
{
  "entrypoint": "index.html",
  "shell": "src/my-app.js",
  "sources": [
    "images/**/*"
  ],
  "extraDependencies": [
    "manifest.json",
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>  ],
  "builds": [
    {
      "name": "es6-bundled",
      "browserCapabilities": [
        "es2015"
      ],
      "js": {
        "minify": true,
        "transformModulesToAmd": true
      },
      "css": {
        "minify": true
      },
      "html": {
        "minify": true
      },
      "bundle": true,
      "addServiceWorker": true
    }
  ],
  "moduleResolution": "node",
  "npm": true,
  "lint": {
    "rules": [
      "polymer-3"
    ]
  }
}

When I served the build version and open the app on the browser, there is bug on the browser console with the error report like so :

I checked the my-app.js on the build version refer to the error report and I got the problem from. So there's the data-transaction.js inline script on my-app.js build version. It is supposed on both my-transaction.js & my-manual.js build version, however I can't see it on them.
If I remove data-transaction.js module on my-manual.js then re-build them, It will be different results. data-transaction.js inline script can be found on my-transaction.js build version and nothing in my-app.js either my-manual.js. The interesting one, either the development and build version can be running well on browser with this solution. However I get other problem with the app js threads.
The js threads problem I mean is related to the data-transaction template which be pushed to html of both my-transaction & my-manual. So in the first time app load, my-transaction.js will bind the data-transaction property. Before the binding process completed, I change the route page to my-manual then push bind event to data-transaction property on my-manual.js. The threads run well, the properties update well, however the data-transaction layout doesn't show up on my-manual.js. To show up it, I need to go back the route page to my-transaction and back again to my-manual.
The last problem is nothing on development version of the first one which import data-transaction to both my-transaction and my-manual.
I am not sure my explanation will be clear here, so tell me where the part you lost them. I have researched this issue and seems very new. I hope you have best solution here?
Additional infos I have worked out to this issue with the following resources :
Polymer version 3
polymer-cli version 1.8.0
polymer-cli version 1.8.1
polymer-cli version 1.9.1
polymer-cli version 1.9.2
Thanks

Out of curiosity, what happens if you include import '../modules/data-transaction.js'; in my-app.js as well? Could be a bundler issue when a module is included by multiple fragments but not the shell.

Out of curiosity, what happens if you include import '../modules/data-transaction.js'; in my-app.js as well? Could be a bundler issue when a module is included by multiple fragments but not the shell.

I remove data-transaction module from both my-transaction & my-manual into my-app, then get this problem :

I run it on development version, so I think will be same problem on build version. In polymer 2 we can do that easily, include the element either on the shell or fragments right?

I think I know the problem, but doesn't have time right now. Will do more research after my work.
Thanks

I have to say I was not careful here when define the data-transaction properties. I have a global variable named "Polymer" which async created. There are several data-transaction properties value reference to this variable, here is the problem coming. When my-app import data-transaction, it doesn't found the "Polymer" variable (undefined).
So, I fix the problem by set the properties value on connectedCallback.
Thanks @keanulee had pointed me


verify that all cases in the reducer switch case are used

i suspect there are lots of unused pieces of state currently. we want to remove any cases that are unused and remove the data they touch in initialState
resolved when unused cases and data are rmeoved

Never used:
FALSE
JUMP_SPECIES_PROGRESS
WEATHER_DATA_RESET
CHANGE_ADDRESS_BY_TYPING
UPDATE_LAST_ZONE
UPDATE_MARKER_COPY
UPDATE_ADDRESS_ON_MAP_CLICK
TOGGLE_ADDRESS_CHANGE_BUTTON
TOGGLE_CROP_DETAIL_MODAL
UPDATE_SELCTED_CHECKBOXES
UPDATE_ADDRESS_SEARCH_PREFERENCE
UPDATE_ACTIVE_GROWTH_PERIOD*
Using Change:
TOGGLE
TRUE
Using dispatch:
UPDATE_ACTIVE_CROP_DATA
LAST_ZIP_CODE
UPDATE_CONSENT
UPDATE_PROGRESS
UPDATE_LOCATION
UPDATE_ZONE_TEXT
CHANGE_ADDRESS
CHANGE_ADDRESS_VIA_MAP
RESET
UPDATE_MARKER
ADD_GOALS
SNACK
UPDATE_SELECTED_GOALS
ADD_SELECTED_GOALS
DRAG_GOALS
PULL_CROP_DATA
SELECTED_CROPS_MODIFIER
ACTIVATE_MY_COVER_CROP_LIST_TILE
ACTIVATE_SPECIES_SELECTOR_TILE
UPDATE_WEATHER_CONDITIONS
UPDATE_FROST_FREE_DAYS
UPDATE_AVERAGE_FROST_DATES
UPDATE_AVERAGE_PRECIP_CURRENT_MONTH
UPDATE_AVERAGE_PRECIP_ANNUAL
SET_AJAX_IN_PROGRESS
TOGGLE_SOIL_LOADER
UPDATE_SOIL_DATA
UPDATE_SOIL_DATA_ORIGINAL
UPDATE_DATE_RANGE
UPDATE_ZIP_CODE
UPDATE_DRAINAGE_CLASS
UPDATE_FLOODING_FREQUENCY
UPDATE_COMPARISON_KEYS
Both change and dispatch:
UPDATE_ZONE

How do change and dispatch differ? they seem to be the same so we should eliminate using the other.

all change does is pass in the new state differently depending on if it's an object or not. that could easily be eliminated but might take a bit of refactoring. did i understand the notes above correctly that change is only used for 3 pieces of state?

Yes i believe so.  i may have missed a few but there was very little using change compared to dispatch.  I didn't start keeping track of which were using change till a little later.  I actually meant to omit that piece of info because i wasn't sure how complete it was.


worker page pod additional details

on the workers page, click on an algorithm so see its pods, expand a pod and click on he "Additional Details" tab.
all data there is undefined

Please use the bug template

Fixed on #44 PR @ Dashboard


netbsd add ifconf data

r? @Amanieu
(rust-highfive has picked a reviewer for you, use r? to override)

@bors r+

:pushpin: Commit 0ef1e7130d5f900b0f1c48ef5e65cf672283039d has been approved by Amanieu

:hourglass: Testing commit 0ef1e7130d5f900b0f1c48ef5e65cf672283039d with merge 80f4ca240c0a3fd9bf8acd70b38d456f18981602...

:sunny: Test successful - checks-actions, checks-cirrus-freebsd-11, checks-cirrus-freebsd-12, checks-cirrus-freebsd-13
Approved by: Amanieu
Pushing 80f4ca240c0a3fd9bf8acd70b38d456f18981602 to master...


Expected next release date

There don't seem to have been any releases in 2016 and I don't see any obvious indication of a planned next release.
Is there an expected date for a 0.10.0, or 1.0.0 release?
Looking at the 1.0.0 milestone, it seems there are two remaining issues?
Please excuse me if I'm asking in the wrong place or asking about things that I should have read somewhere. If so please point to the correct location for such information.
Love the software by the way and am interested in contributing

I just saw issue #1782
So I realize that #1690 is the blocker for 0.10.0.  Is 0.10.0 the same as 1.0.0 ?

Yes, graphite developers are agreed to release 0.10.0 branch as graphite 1.0.0, when it will be ready to release. And, indeed, #1690 is the main blocker now.


programmatic spellchecker

Is there a way to turn on/off the spellchecker programmatically?

The spellChecker option. But that's only before initializing. There's not a way once you've initialized.

I think this would be a great enhancement. I would be willing to help on this as well.  Thoughts?

If you'd like, you can submit a PR for this. Be sure to read the contribution guide.

Any pointers?
Luis Majano
CEO
Ortus Solutions, Corp
www.ortussolutions.com
P/F: 1-888-557-8057
On Sat, May 21, 2016 at 6:33 PM -0700, "Wes Cossick"<EMAIL_ADDRESS>wrote:
If you'd like, you can submit a PR for this. Be sure to read the contribution guide.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub

Guys is there a preview mode only?

You'd need to edit the source JS file. You'll also need to fully document the feature in the README.


DELETE method support

I've been using an older intercooler.js for a while now and I noticed that the latest version doesn't support PUT and DELETE requests anymore. Since I have an application that uses DELETE heavily I would have to either rewrite my views or add _method support to the framework I'm using.
Is there any way to go back to the old behaviour with the latest intercooler release? I only need to support modern browsers, so it's not a big deal for me that DELETE is not supported well.
Thanks for your help!

Hi there.
Sorry for breaking that!  In the development branch I've added a meta-tag flag that allows you to enable the actual HTTP method:
<meta name="intercoolerjs:use-actual-http-method" content="true"/>
I'm going to cut another release on Friday if you can wait, or you can grab it out of the dev branch.

No worries. I read the changelog so you didn't break anything ;)
I can totally wait and I'm happy you re-enable the old ways with a setting.


Undesired UX behavior of DataFrame output in IPython Notebook

From my previous line comment at https://github.com/pydata/pandas/commit/be80898920eab8f740cf61019327621a75d73947#commitcomment-11338872. cc @takluyver @jorisvandenbossche
Because DataFrame's _repr_html_ returns hardcoded div tag with style max-height:1000px, if the height of DataFrame's output exceeds 1000px, it will generate a scroll bar.
First, it is unnecessary because IPython Notebook has "toggle output scrolling" feature. Second, it causes UX problem if the height of browser resolution is below 1000px (user must scroll twice: the whole page and the generated div). Third, it can't make use of "toggle output scrolling" feature (if user turn it on, it will generate nested scroll bars (worse UX)). For more detailed explanations and examples, see the steps to reproduce below.
Pandas version: 0.16.1.
Steps to reproduce:

Open an IPython Notebook and paste this code. It will generate an inner scroll bar because the output's height exceeds 1000px.
import pandas as pd
pd.DataFrame(list(range(100)))

If you want to see all rows, first you should scroll the page.

After you reach the bottom of the generated ouput's div, you should scroll again the inner scroll bar.

If you want to toggle the IPython Notebook's "toggle output scrolling" feature, you get this.

Desired behaviors:

When "output scrolling" feature is off

When "output scrolling" feature is on

Solution:
Remove style property from div tag that DataFrame's _repr_html_ returns.
Old: <div style="max-height:1000px;max-width:1500px;overflow:auto;">\n
New: <div>\n
Why should max-width be removed as well? Because it also generates far worse UX. If you have so many columns that the horizontal scroll bar generated, you cannot scroll it. You must refresh the page to be able to scroll it.

Thanks for the clear documentation of the issue with the screenshots. I agree, we should fix this.

Does IPython notebook cleanly handle horizontal scrolling when necessary if max-width is removed?

IIRC, in IPython 3 we put the output in its own scrollable container if it is wider than the notebook. In earlier versions, the page would just get wider, so the whole page would get a horizontal scrollbar.

Indeed, I verified that I get the proper horizontal scrolling behavior in IPython 3.
My only concern here is that IPython 3 may still be a little too new (3 months old) for us to change this -- we might break some folks still using earlier versions. On the other hand, upgrading IPython is pretty easy/seamless (in my experience).
Any other thoughts here?

@shoyer I tested using IPython 2.4.1 and pandas with DataFrame's max-width removed. As @takluyver said, in IPython 2 the page just gets wider. Yup it is ugly, but it works. Is it acceptable?

At least it only makes the one output cell wider -- it could be far worst (e.g., the entire page gets stretched).
For things like this, that work either way but are ugly for some users, I think we probably should make things nicer for users who are using the latest version of IPython. Of users who are using the latest version of pandas, a significant fraction are probably also keeping IPython up to date. So I think this change is a good idea.

closed by #10231

I've encountered a bug which may be related which results in very strange scrollbar behaviour with a particular csv dataset. Unfortunately I wasn't able to generate the problem with a simple script, it occurs with the (anonymised) dataset at this link:
https://www.dropbox.com/s/gdoow6ro8u7yn70/pandas multiple scrollbars bug.csv?dl=0
Steps to reproduce: open a new iPython notebook and paste the following code
import pandas as pd
df = pd.read_csv('pandas multiple scrollbars bug.csv')
df

@jeremybickerstaffe you didn't specify the ipython version. This is fixed in 0.16.2 for ipython >= 3.0.0.

My iPython is version 3.2.1

I can't reproduce this with your file. Can you show the output of pd.show_versions()

Sure, the output is:
INSTALLED VERSIONS
------------------
commit: None
python: 2.7.9.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.16.2
nose: 1.3.4
Cython: 0.22
numpy: 1.9.2
scipy: 0.15.1
statsmodels: 0.6.1
IPython: 3.2.1
sphinx: 1.3.1
patsy: 0.3.0
dateutil: 2.4.2
pytz: 2015.4
bottleneck: None
tables: 3.1.1
numexpr: 2.3.1
matplotlib: 1.4.3
openpyxl: 1.8.5
xlrd: 0.9.3
xlwt: 0.7.5
xlsxwriter: 0.6.7
lxml: 3.4.2
bs4: 4.3.2
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 0.9.9
pymysql: 0.6.6.None
psycopg2: None

Strange, I can't reproduce it with the same version of pandas and IPython. Which browser are you using?

Chrome Version 44.0.2403.125 m (64-bit).
It does appear to be very much an edge case. I found that bizarre changes, like changing the name of the file that's being read, or changing the reading code to this below can remove the rendering problem.
import pandas as pd
pd.read_csv('pandas multiple scrollbars bug.csv')


Prevent client code from linking with incompatible gRPC library (ASAN / TSAN)

The PR is a proposal to address the issue #21838 that can cause unexpected run time errors that are hard to detect and understand.
Principle
The idea is to take advantage of inlined CreateChannel() functions - which are mandatory calls for any grpc client code - to insert a check that makes the client code validate its compatibility with the gRPC library it is about to link with. If not, it will cause a linker error designed to help the user understand what's going wrong.
The runtime impact of those changes are almost null.
This PR also aims at creating an exhaustive list of the type definitions that are subject to change depending on the compiler options. This is done via dedicated macro constants that were added to help the maintainers know about the fact they must update this ODR violation check if they change those structure definitions.
Example
Trying to link non ASAN client code with a gRPC library built with ASAN will create the following link error:
../grpc_services.a(grpc_client.cc.o): In function `grpc_impl::PreventOneDefinitionRuleViolation()':
/usr/local/grpc-asan/include/grpcpp/create_channel_impl.h:46: undefined reference to `grpc_impl::grpc_must_be_compiled_without_asan'
collect2: error: ld returned 1 exit status

What is more likely to happen is trying to link an ASAN client code with a gRPC library that is not ASAN, which will produce the following error with this PR:
../grpc_services.a(grpc_client.cc.o): In function `grpc_impl::PreventOneDefinitionRuleViolation()':
/usr/local/grpc-asan/include/grpcpp/create_channel_impl.h:43: undefined reference to `grpc_impl::grpc_must_be_compiled_with_asan'
/usr/local/grpc-asan/include/grpcpp/create_channel_impl.h:43: undefined reference to `grpc_impl::grpc_must_be_compiled_with_asan'
collect2: error: ld returned 1 exit status

Alternative option
Another option would be to always build the types with the extra fields required to ASAN and TSAN. But I don't know the runtime impact of this approach.
@nicolasnoble

Looks really neat!


Proposal: Implementation of Conventional Commits and Semantic Release

Description
I propose that we implement Conventional Commits for our commit messages. It's a very common convention and easy to learn.
Benefits
By adopting Conventional Commits, we can take advantage of tools such as Semantic Release, which offers the following benefits:

Automated versioning: Semantic Release can automatically increment the version number (semver) according to validation messages.
Change log generation: It can generate a changelog, providing a clear and concise history of changes.
GitHub version generation: It can also generate GitHub versions, making it easier for users to see what's new in each version.

Implementation
If everyone agrees with this proposal, I can implement these tools with Commitlint to enforce commit message conventions.
I think the implementation of these tools could greatly benefit our project. I look forward to hearing from you.

I have considered Semantic Release, but haven't implemented it due to lack of time. I'm also a fan of conventional commits, and my current team uses them religiously. I'd prefer that every merged PR doesn't automatically go out to everyone and introduce potentially breaking changes. My thought was maybe that every merged PR gets pushed to NPM tagged as @Next  or something and then perhaps have a GitHub Action to manually release a specific version on the main channel.
As of now, there is a GitHub Action that does a release when the version in package.json is changed, but a continuous delivery setup would be preferred so folks can have quicker access to changes from NPM in the case that an official release is taking longer than expected. A PR along those lines would certainly be much appreciated.

I think we could do this in several stages.
As you're already used to it, we can already use conventional commit, I can do a PR to add commitlint to the project.
As for semantic-release, I've only used it to make releases on the main branch. I know it can handle pre-releases, but it seems to me that this has to go through a branch (next or beta).
So what we could do, if you agree, is to modify the release workflow to integrate semantic-release with a manual trigger on your part. This is to avoid introducing bugs or releasing an unstable version. I can also take care of this PR.
As far as releasing a pre-release is concerned, I tend to agree, but I don't think it's a good idea to do it every PR. How about using a cron like once a week or once a month?
I'd have to test semantic-release on my side to see if it would be possible to generate pre-releases from the main branch.

So I am a fan of conventional commits, but not positive how it would work with a public repo having commitlint enabled to ensure all contributors are compliant. Maybe there are other projects already doing this and it works well? I know the Dependabot commit prefix is configurable at least.
It’s also possible I’m overcomplicating the release process by insisting on a non-stable channel. I am quite accustomed to anything merging to the main branch being released in my day job. A big part of the problem is that I still feel the need to validate manually before releasing because the integration tests in #148 only ever got halfway completed. Some additional checks might be useful as well, like something looking for warning logs and the like and also validating test coverage. Sometimes, I have merged PRs to the main branch and then done some additional tweaks, although I could merge PRs to another branch, do the tweaks, and only merge once it’s good to be released in such situations.
Once we’re more confident in the checks, Dependabot PRs could possibly automatically merge and release as well if we could automatically respect semver for updated packages, especially Jest, and that would automate a good portion of the maintenance of this package.
The changes from your last PR will be incredibly helpful in this regard as well, since there have been PRs where I’ve had to manually fix code styling issues in the past. For example, #165 is something I’d like to merge that has uncaught style issues. Again, very much appreciated!
So, TL;DR - improving the automated checks is probably the next step, and then we could probably use semantic-release with its out of the box release process assuming enforcing conventional commits is a process that can work well.

Hello,
I agree with you, if a large number of processes are automated it will make your repo management easier.
I agree with you, automating a large number of processes will make your repo management easier.
Regarding tests, yes they are needed to validate the code when merging a PR, what you can do is pin issue #148 so that it is visible and everyone can contribute to improving the tests.
As for using the conventional commit, for me, whether it's a public or private repo, from my point of view you are the project manager and/or technical lead, so if you decide you need to use that commit convention then we'll use it, and you've recently added a contributing.md which is just to explain how to contribute.

Another solution would be to overwrite the commits when merging a PR, which would allow you to format the commit to your liking, but this involves a complex manual action on your part, and I understand that you don't have a lot of time.
For the semantic release part, this can be done a little later (once all the tests have been written and the commit convention has been adopted). I'll look into a solution that might suit you.


add tests.xml sample for different test type

@zqzhang @Shao-Feng @Honry I add the xml samples for different test type, Pls help to review, you could refer them into your developer guide.
and add more samples if any special case.

add the node unit test type tests.xml sample

LGTM

@zqzhang Updated,